AI for Network Engineers: Long Short-Term Memory (LSTM)

 Introduction


As mentioned in the previous chapter, Recurrent Neural Networks (RNNs) can have hundreds or even thousands of time steps. These basic RNNs often suffer from the gradient vanishing problem, where the network struggles to retain historical information across all time steps. In other words, the network gradually "forgets" historical information as it progresses through the time steps.

One solution to address the horizontal gradient vanishing problem between time steps is the use of Long Short-Term Memory (LSTM) based RNN instead of basic RNN. LSTM cells can preserve historical information across all time steps, whether the model contains ten or several thousand time steps. 

Figure 6-1 illustrates the overall architecture of an LSTM cell. It includes three gates: the Forget gate, the Input gate (a.k.a. Remember gate), and the Output gate. Each gate contains input neurons that use the Sigmoid activation function. The reason for employing the Sigmoid function, as shown in Figure 5-4 of the previous chapter, is its ability to produce outputs in the range of 0 to 1. An output of 0 indicates that the gate is "closed," meaning the information is excluded from contributing to the cell's internal state calculations. An output of Continue reading

BGP Updates in 2024

The first part of this annual report on BGP for the year 2024 looked at the size of the routing table and some projections of table growth for both IPv4 and IPv6. However, the scalability of BGP as the Internet’s routing protocol is not just dependant on the number of prefixes carried in the routing table. BGP protocol behaviour in the form of dynamic routing updates are also part of this story. If the update rate of BGP is growing faster than we can deploy processing capability to match, then the routing system will lose coherence, and at that point the network will head into periods of instability. This report looks at the profile of BGP updates across 2023 to assess whether the stability of the routing system, as measured by the level of BGP update activity, is changing.

From Python to Go 009. Interfaces For Data Types In Go (Golang).

Hello my friend,

First of all, Happy New Year! We hope that you had a great festive time with your beloved ones, families and friends. That’s the one of the most important part of our lives and, in our opinion, spending some time off the grid impacts our mental well-being positively and gives us energy to move forward and achieve new heights in professional and business areas.

Talking about the topic of today blog post, we thought it will be useful to show you a concept, which is Go (Golang) specific, as there is no such a need in Python. This concept is called “interfaces”, and it is extremely helpful when you work with external data, which you will face working with external data source, e.g. retrieving data from APIs with JSON/XML encoding.

Disclaimer, we talk about interfaces only in the context of the data types in Go (Golang), as it is also used for class composition (object-oriented programming), so we put it aside for now. We may get back to it later in our blog series.

Automation and AI?

If you follow latest trends, you see that AI in various forms, whether this is agentic AI, Continue reading

Multi-Path TCP: revolutionizing connectivity, one path at a time

The Internet is designed to provide multiple paths between two endpoints. Attempts to exploit multi-path opportunities are almost as old as the Internet, culminating in RFCs documenting some of the challenges. Still, today, virtually all end-to-end communication uses only one available path at a time. Why? It turns out that in multi-path setups, even the smallest differences between paths can harm the connection quality due to packet reordering and other issues. As a result, Internet devices usually use a single path and let the routers handle the path selection.

There is another way. Enter Multi-Path TCP (MPTCP), which exploits the presence of multiple interfaces on a device, such as a mobile phone that has both Wi-Fi and cellular antennas, to achieve multi-path connectivity.

MPTCP has had a long history — see the Wikipedia article and the spec (RFC 8684) for details. It's a major extension to the TCP protocol, and historically most of the TCP changes failed to gain traction. However, MPTCP is supposed to be mostly an operating system feature, making it easy to enable. Applications should only need minor code changes to support it.

There is a caveat, however: MPTCP is still fairly immature, and while it can Continue reading

Kubernetes Security in 2025: The De Facto Platform of GenAI Applications

Over the past year, there has been a culmination of hype and excitement around Generative AI (GenAI). Most organizations initiated proof-of-concept projects for GenAI, eager to reap the technology’s benefits, which range from improved operational efficiency to cost reductions. According to recent research, 88% of organizations are in the midst of actively investigating GenAI, transcending other AI applications. However, the vast majority of organizations have yet to surpass this initial proof-of-concept stage and graduate GenAI applications into production. As we move into 2025, more organizations will begin to formalize their GenAI strategies, creating and deploying a host of new GenAI applications across their infrastructure.

Creating GenAI Applications with Kubernetes

As organizations build out GenAI applications, they will leverage many different GenAI models. To optimize, and derive the most value and accuracy from their GenAI applications, enterprises will utilize proprietary data to create these models, primarily through a Retrieval-Augmented Generation (RAG) architecture. A RAG architecture enables organizations to customize models based on company data, so that GenAI applications are personalized to an enterprise and their specific use cases. Most GenAI applications will contain proprietary company data as a result of this approach, creating many security concerns for organizations.

Consequently, some Continue reading

Behind the scenes with Stream Live, Cloudflare’s live streaming service

Cloudflare announced Stream Live for open beta in 2021, and in 2022 we went GA. While we talked about the experience of using it and the value it delivers to customers, we didn’t talk about how we built it. So let’s talk about Stream Live’s design, and how it leverages the distributed nature of Cloudflare’s network, rather than centralized locations as many other live services do. Ultimately, our goals are to keep our content ingest as close to broadcasters as possible, our content delivery as close to viewers as possible, and to retain our ability to handle unexpected use cases.

At a high level, Stream Live accepts audio/video content from broadcasters and makes that content available to viewers around the world in real time through the Cloudflare network, which reaches more than 330 cities in over 120 countries. Hence, there are two sides to this: ingesting data from broadcasters and delivering encoded content to viewers. Both sides are built on a combination of internal systems and Cloudflare products, including Cloudflare Workers, Durable Objects, Spectrum, and, of course, Cache.

Let’s start on the ingest side.

Ingesting a broadcast

Broadcasters generate content in real time, as a Continue reading

BGP in 2024

At the start of each year, it’s been my habit to report on the behaviour of the Internet’s inter-domain routing system over the previous 12 months, looking in some detail at some metrics from the routing system that can show the essential shape and behaviour of the underlying interconnection fabric of the Internet.

A Year of Consistency, Again

2024 was a year of being busy. You probably noticed as a loyal reader because my output on this blog fell off quite a bit. I wanted to get back on track per my New Year’s Day post. How did I do? Sixteen posts for the whole year. Barely more than one a month.

That doesn’t mean I wasn’t busy. I have been working hard to bring great Tech Field Day events to the community. I’ve become more active on BlueSky as the community shifts there due to the craziness happening on Twitter/X. I have been getting more and more briefings on technology, which I’ve been writing up on LinkedIn. And of course I’ve been active on the Gestalt IT Rundown and the Tech Field Day Podcast

I also ran almost every day in 2024. I mentioned on Facebook that “consistency beats quantity”, which was a phrase that encouraged me to try and run at least one mile a day in 2024. That ended up being 901 miles of running for the year, with November and December having a LOT or running. I plan on keeping that going in 2025, where I’m aiming for 1,000 miles. It will be a Continue reading

Intersection of AI and Web3

Over the past year, AI has taken the world by storm, revolutionizing industries and reshaping technological landscapes. Having been deeply involved in the web3 domain for over two years, I’ve observed a fascinating overlap between these two transformative technologies. This blog explores how AI and blockchain complement each other: AI is opening up new possibilities … Continue reading Intersection of AI and Web3 →

AI Security and Safety Ecosystem

The field of artificial intelligence (AI) has seen explosive growth over the past two years, with its potential for future advancements appearing virtually limitless. However, with this rapid expansion comes a growing wave of challenges and risks. From AI-generated scams to deepfakes and data breaches, many people have either directly experienced or heard about the … Continue reading AI Security and Safety Ecosystem →

The forecast is clear: clouds on e-paper, powered by the cloud

I’ve noticed that many shops are increasingly using e-paper displays. They’re impressive: high contrast, no backlight, and no visible cables. Unlike most electronics, these displays are seamlessly integrated and feel very natural. This got me wondering: is it possible to use such a display for a pet project? I want to experiment with this technology myself.

(source)

My main goal in this project is to understand the hardware and its capabilities. Here, I'll be using an e-paper display to show the current weather, but at its core, I’m simply feeding data from a website to the display. While it sounds straightforward, it actually requires three layers of software to pull off. Still, it’s a fun challenge and a great opportunity to work with both embedded hardware and Cloudflare Workers.

Sourcing the hardware

For this project, I'm using components from Waveshare. They offer a variety of e-paper displays, ranging from credit card-sized to A4-sized models. I chose the 7.5-inch, two-color "e-Paper (G)" display. For the controller, I'm using a Waveshare ESP32-based universal board. With just these two components — a display and a controller — I was ready to get started.

When the components arrived, I carefully Continue reading

Open sourcing h3i: a command line tool and library for low-level HTTP/3 testing and debugging

Have you ever built a piece of IKEA furniture, or put together a LEGO set, by following the instructions closely and only at the end realized at some point you didn't quite follow them correctly? The final result might be close to what was intended, but there's a nagging thought that maybe, just maybe, it's not as rock steady or functional as it could have been.

Internet protocol specifications are instructions designed for engineers to build things. Protocol designers take great care to ensure the documents they produce are clear. The standardization process gathers consensus and review from experts in the field, to further ensure document quality. Any reasonably skilled engineer should be able to take a specification and produce a performant, reliable, and secure implementation. The Internet is central to everyone's lives, and we depend on these implementations. Any deviations from the specification can put us at risk. For example, mishandling of malformed requests can allow attacks such as request smuggling.

h3i is a binary command line tool and Rust library designed for low-level testing and debugging of HTTP/3, which runs over QUIC. h3i is free and open source as part of Cloudflare's quiche project. In this post we'll Continue reading

AI for Network Engineers: Recurrent Neural Network (RNN) – Part II

 Challenges of a RNN Modell


Figure 5-3 shows the last two time steps of our Recurrent Neural Network (RNN). At the time step n (on the left side), there are two inputs for the weighted sum calculation: Xn  (the input at the current time step) and ht−1 (the hidden state from the previous time step).

First, the model calculates the weighted sum of these inputs. The result is then passed through the neuron’s activation function (Sigmoid in this example). The output of the activation function, ht , is fed back into the recurrent layer on the next time step, n+1. At time step n+1, the ht  is combined with the input Xn to calculate weighted sum. This result is then passed through the activation function, which now produces the model's prediction, y ̂ (y hat). These steps are part of the Forward Pass process.

As the final step in the forward pass, we calculate the model's accuracy using the Mean Square Error (MSE) function (explained in Chapter 2).

If the model's accuracy is not close enough to the expected result, it begins the Backward Pass to improve its performance. The most used optimization algorithm for minimizing the loss function during Continue reading

What’s new in Cloudflare: MASQUE now powers 1.1.1.1 & WARP apps, DEX now generally available with Remote Captures

At Cloudflare, we are constantly innovating and launching new features and capabilities across our product portfolio. Today’s roundup blog post shares two exciting updates across our platform: our cross-platform 1.1.1.1 & WARP applications (consumer) and device agents (Zero Trust)  now use MASQUE, a cutting-edge HTTP/3-based protocol, to secure your Internet connection. Additionally, DEX is now available for general availability.

Faster and more stable: our 1.1.1.1 & WARP apps now use MASQUE by default

We’re excited to announce that as of today, our cross-platform 1.1.1.1 & WARP apps now use MASQUE, a cutting-edge HTTP/3-based protocol, to secure your Internet connection.

As a reminder, our 1.1.1.1 & WARP apps have two main functions: send all DNS queries through 1.1.1.1, our privacy-preserving DNS resolver, and protect your device’s network traffic via WARP by creating a private and encrypted tunnel to the resources you’re accessing, preventing unwanted third parties or public Wi-Fi networks from snooping on your traffic.

There are many ways to encrypt and proxy Internet traffic — you may have heard of a few, such as IPSec, WireGuard, or OpenVPN. There are many tradeoffs Continue reading

1 … 44 45 46 47 48 … 3,799