Opensource LLM Models – Meta llama / Meta Codellama ? Deploying In-house ? Context of Networking!

Disclaimer: All Writings And Opinions Are My Own And Are Interpreted Solely From My Understanding. Please Contact The Concerned Support Teams For A Professional Opinion, As Technology And Features Change Rapidly.

In a world where even your toaster might soon have a PhD in quantum physics, LLMs are taking over faster than a cat video going viral! LLMs are becoming increasingly powerful and are being integrated into various business and personal use cases. Networking is no different. Due to reasons like privacy, connectivity, and cost, deploying smaller form factor models or larger ones (if you can afford in-house compute) is becoming more feasible for faster inference and lower cost.

The availability and cost of model inference are improving rapidly. While OpenAI’s ChatGPT-4 is well-known, Meta and other firms are also developing LLMs that can be deployed in-house and fine-tuned for various scenarios.

Let’s explore how to deploy an open-source model in the context of coding. For beginners, ease of deployment is crucial; nothing is more off-putting than a complicated setup.

Reference : Ollama.com (https://github.com/ollama/ollama?tab=readme-ov-file) simplifies fetching a model and starting work immediately.

Visit ollama.com to understand what a codellama model looks like and what Continue reading

Weekend Reads 071924


On the other hand, these same minimal overheads imply that DNS over UDP cannot perform prompt detection of packet loss and cannot efficiently defend itself against various approaches to tampering with the DNS, such as source address spoofing, payload alteration and third-party packet injection. Perhaps most importantly, the way UDP handles large payloads is a problem.


User interface (UI) design is currently experiencing a transition from traditional graphical user interfaces (GUIs) to systems designed to recognize a personﾒs gestures and movements.


The U.S. Federal Trade Commission (FTC), along with two other international consumer protection networks, announced on Thursday the results of a study into the use of “dark patterns” — or manipulative design techniques — that can put users’ privacy at risk or push them to buy products or services or take other actions they otherwise wouldn’t have.


One of the most concerning aspects of social media is that much of its influence evades our notice. We don’t realize that we’re being influenced, or shaped to think a certain way, or view the world through a specific lens.


Chinese automobile conglomerate Geely has made significant strides since I last wrote about their Geesat LEO constellation for mobile vehicle connectivity.


Continue reading

Terraform for Network Engineers: Part Four

Terraform for Network Engineers: Part Four

Welcome back to the final part of my Terraform for Network Engineers series! In this installment, we're diving into the often tricky territory of managing the state file in Terraform. I'll also walk you through the concept of remote state management and show you how it can streamline your infrastructure management. Let's get started!

Managing the State file

The state file is the backbone of Terraform. It keeps a detailed record of the resources you've created and their current state. Terraform relies on this file to figure out what changes need to be made whenever you run terraform apply. By default, Terraform saves this file locally as terraform.tfstate.

While this works fine if you're the only one handling the infrastructure, it can quickly become a headache for a team. Multiple engineers working on the same setup can run into conflicts and inconsistencies. The solution? Store the state file remotely in a shared location that everyone on the team can access. This way, everyone stays on the same page, and your infrastructure management becomes much smoother.

Remote State Management

Remote state management is all about storing the state file in a place that everyone on the team can access. This Continue reading

Why Your Mobile App Needs Client-Side Network Monitoring

The vast majority of mobile applications rely on making network requests to deliver a successful user experience. However, many engineering teams do not have client-side network monitoring. Instead, they rely exclusively on network performance is from a backend perspective. Not All Requests Make It to Your Backend Servers Your backend can only measure the behavior of network requests that actually reach your servers. Below are a few reasons why requests would fail to make it there. No Internet Connection There are scenarios where it is not obvious to mobile users that they don’t have a connection. For example, a user can be connected to a WiFi access point, but the upstream connection from the access point is down or has intermittent connectivity. Interrupted Connection Even if you initially make a successful connection to a backend server, there’s no guarantee that the request will complete successfully. This is more common with mobile Continue reading

Exploring Internet traffic during the 2024 U.S. Republican National Convention

Internet traffic typically mirrors human behavior, with significant fluctuations during large political events. This comes during a time when the United States is in election mode, as political campaigns are in full swing and candidates for various offices, primaries and caucuses make their case to voters and debates are being held. This week, the Republican National Convention was hosted in Milwaukee, Wisconsin from July 15 to 18, 2024. We examined traffic shifts and cyberattacks since June 2024 to see how these events have impacted the Internet.

Attacks on political related websites

Cyberattacks are a constant threat, and aren't necessarily driven by elections. With that said, notable trends can often be observed, and we’ve seen before how specific geopolitical events can trigger online attacks. For example, we saw cyberattacks at the start of the war in Ukraine to more recently in the Netherlands, when the June 2024 European elections coincided with cyberattacks on Dutch political-related websites that lasted two days — June 5th and 6th. The main DDoS (Distributed Denial of Service attack) attack on June 5, the day before the Dutch election, reached 73,000 requests per second (rps).

Shifting our focus to the United States in particular, Continue reading

The Hedge 235: Copyrights and Centralization

Join us as Tom, Eyvonne, and Russ hang out for another roundtable. We start the show talking about Tom’s plant (is it real or … ??). What does copyright have to do with Internet Service Providers? Should the two topics be related at all? What can the IETF do about Internet centralization?

Thanks for listening—and please reach out if you have a topic you’d like to hear about, or a guest you’d like to hear.

 

 

download

AI/ML Networking: Part-III: Basics of Neural Networks Training Process

Neural Network Architecture Overview

Deep Neural Networks (DNN) leverage various architectures for training, with one of the simplest and most fundamental being the Feedforward Neural Network (FNN). Figure 2-1 illustrates our simple, three-layer FNN.

Input Layer: 

The first layer receives the initial data, consisting of parameters X1, X2, and X3. Each neuron in the input layer passes these data parameters to the next hidden layer.

Hidden Layer: 

The neurons in the hidden layer calculate a weighted sum of the input data, which is then passed through an activation function. In our example, we are using the Rectified Linear Unit (ReLU) activation function. These calculations produce activation values for neurons. The activation value is modified input data value received from the input layer and published to upper layer.

Output Layer: 

Neurons in this layer calculate the weighted sum in the same manner as neurons in the hidden layer, but the result of the activation function is the final output.


The process described above is known as the Forwarding pass operation. Once the forward pass process is completed, the result is passed through a loss function, where the received value is compared to the expected value. The difference between these two values Continue reading

Some History on VLAN 1 in Cisco Switches

Over the years, there has been a lot of discussion on if VLAN 1 in Cisco switches is special or not. Does it have any characteristics that other VLANs don’t? I covered some of this in the Is VLAN 1 Special in Cisco Networks. This time I thought it would be interesting to give some historical perspective on VLAN 1 and describe some of the implementation details that I learned from Francois Tallet. Francois was heavily involved in L2 and STP when at Cisco.

The 802.1Q standard was released at the end of 1998. Several years before that, Cisco had introduced Inter-Switch Links (ISL) and Dynamic ISL (DISL) to support VLANs. The main difference between ISL and 802.1Q is that ISL encapsulates the entire frame as opposed to 802.1Q that adds a field to the existing frame. DISL was a method of forming trunks dynamically, a predecessor to Dynamic Trunking Protocol (DTP) if you will.

Before VLANs and before ISL, it was simple to send control plane protocol frames such as CDP, PAgP, STP, etc. There was no concept of VLANs so there was no relation to VLANs or encapsulating/tagging the frames. When VLANs were introduced, now Continue reading

AMD’s Long And Winding Road To The Hybrid CPU-GPU Instinct MI300A

Back in 2012, when AMD was in the process of backing out of the datacenter CPU business and did not really have its datacenter GPU act together at all, the US Department of Energy exhibited the enlightened self-interest that is a strong foundation of both economics and politics and took a chance and invested in AMD to do research in memory technologies and hybrid CPU-GPU computing at exascale. …

AMD’s Long And Winding Road To The Hybrid CPU-GPU Instinct MI300A was written by Timothy Prickett Morgan at The Next Platform.

VMware’s ‘Private Cloud’ Solution Emerges Under Broadcom

VMware Cloud Foundation’s (VCF) new configuration was eagerly awaited. There have been many questions about what VCF would look like exactly and, more importantly, what it would mean for DevOps customers now that VCF is under the Broadcom umbrella. While there has been a lot of discussion about price increases for some customers following licensing changes and other attributes of VMware honing its product portfolio under Broadcom, we have now seen, during the past few days, releases detailing what VCF now means, what it has to offer and what is planned for the future. To that end, a lot of care has been taken to accommodate more emerging needs, especially for private cloud ownership involving large, geographically distributed operations across many different sectors. This often includes IoT and edge applications that private cloud is configured for. There is also a simplification in VCF’s portfolio now under Broadcom, which we will detail below. The company detailed several features, including VCF’s management in line with hyper-convergence and combining storage operations environments under a single umbrella, uniting or “de-siloing” them. This offers many advantages and accounts for much of the hype surrounding VCF. At the same time, the development of VCF’s offering Continue reading

TL000: Announcing Technically Leadership, a New Podcast for the Next Phase of Your Career

Technically Leadership is a brand new podcast on the Packet Pushers network. Host Laura Santamaria explores leadership in the tech industry, with conversations and insights to help you development your management skills. Whether you’re considering your first management role or you’re an experienced manager working your way to the C-suite, this podcast is for you.... Read more »

eBPF: Enabling Security and Performance to Co-Exist

Today, most organizations and individuals use Linux and the Linux kernel with a “one-size-fits-all” approach. This differs from how Linux was used in the past–for example, 20 years ago, many users would compile their kernel and modify it to fit their specific needs, architectures and use cases. This is no longer the case, as one-size-fits-all has become good enough. But, like anything in life, “good enough” is not the best you can get.

Enter: Extended Berkeley Packet Filter (eBPF). eBPF allows users to modify one-size-fits-all to fit their specific needs. While this was not impossible before, it was cumbersome and often unsecure.

eBPF is a feature available in Linux kernels that allows users to safely load programs into the kernel, to customize its operation. With eBPF, the kernel and its behavior become highly customizable, instead of being fixed.

Utilizing eBPF, users can load a program into the kernel and instruct the kernel to execute their program if, for example, a certain packet is seen or another event occurs. eBPF lets programs run without needing to add additional modules or modify the kernel source code. Users can think of it as a lightweight, sandboxed virtual machine (VM) within the Linux kernel Continue reading

Eliminating hardware with Load Balancing and Cloudflare One

In 2023, Cloudflare introduced a new load balancing solution supporting Local Traffic Management (LTM). This year, we took it a step further by introducing support for layer 4 load balancing to private networks via Spectrum. Now, organizations can seamlessly balance public HTTP(S), TCP, and UDP traffic to their privately hosted applications. Today, we’re thrilled to unveil our latest enhancement: support for end-to-end private traffic flows as well as WARP authenticated device traffic, eliminating the need for dedicated hardware load balancers! These groundbreaking features are powered by the enhanced integration of Cloudflare load balancing with our Cloudflare One platform, and are available to our enterprise customers. With this upgrade, our customers can now utilize Cloudflare load balancers for both public and private traffic directed at private networks.

Cloudflare Load Balancing today

Before discussing the new features, let's review Cloudflare's existing load balancing support and the challenges customers face.

Cloudflare currently supports four main load balancing traffic flows:

  1. Internet-facing load balancers connecting to publicly accessible endpoints at layer 7, supporting HTTP(S).
  2. Internet-facing load balancers connecting to publicly accessible endpoints at layer 4 (Spectrum), supporting TCP and UDP services
  3. Internet-facing load balancers connecting to private endpoints at layer 7 HTTP(S) via Cloudflare Tunnels.
  4. Continue reading

Q2 2024 Internet disruption summary

Cloudflare’s network spans more than 320 cities in over 120 countries, where we interconnect with over 13,000 network providers in order to provide a broad range of services to millions of customers. The breadth of both our network and our customer base provides us with a unique perspective on Internet resilience, enabling us to observe the impact of Internet disruptions. Thanks to Cloudflare Radar functionality released earlier this year, we can explore the impact from a routing perspective, as well as a traffic perspective, at both a network and location level.

As we have seen in previous years, nationwide exams take place across several MENA countries in the second quarter, and with them come government directed Internet shutdowns. Cable cuts, both terrestrial and submarine, caused Internet outages across a number of countries, with the ACE submarine cable being a particular source of problems. Maintenance, power outages, and technical problems also disrupted Internet connectivity, as did unknown issues. And as we have frequently seen in the two-plus years since the conflict began, Internet connectivity in Ukraine suffers as a result of Russian attacks.

As we have noted in the past, this post is intended as a summary overview Continue reading

AI/ML Networking: Part-II: Introduction of Deep Neural Networks

Machine Learning (ML) is a subset of Artificial Intelligence (AI). ML is based on algorithms that allow learning, predicting, and making decisions based on data rather than pre-programmed tasks. ML leverages Deep Neural Networks (DNNs), which have multiple layers, each consisting of neurons that process information from sub-layers as part of the training process. Large Language Models (LLMs), such as OpenAI’s GPT (Generative Pre-trained Transformers), utilize ML and Deep Neural Networks.

For network engineers, it is crucial to understand the fundamental operations and communication models used in ML training processes. To emphasize the importance of this, I quote the Chinese philosopher and strategist Sun Tzu, who lived around 600 BCE, from his work The Art of War.

If you know the enemy and know yourself, you need not fear the result of a hundred battles.

We don’t have to be data scientists to design a network for AI/ML, but we must understand the operational fundamentals and communication patterns of ML. Additionally, we must have a deep understanding of network solutions and technologies to build a lossless and cost-effective network for enabling efficient training processes.

In the upcoming two posts, I will explain the basics of: 

a) Data Models: Continue reading

Encapsulation of PDUs On Trunk Ports

When I studied for my CCIE almost 15 years ago, I recall that I was fascinated by how different PDUs such as CDP, DTP, STP would have different encapsulations on a trunk depending on the configuration of it. What happens when you change the native VLAN? What happens if the native VLAN is not allowed on the trunk? What happens if you tag the native VLAN? There aren’t many resources describing this as most people don’t care for this level of detail, but there are situations where this is important. The goal of this post is to configure different protocols and see how they are encapsulated using different trunk configurations. You don’t need to consume this entire post, rather use it as a reference for different scenarios. Just be aware that some of this may be platform/OS specific.

The protocols we’ll cover for this post are:

  • CDP.
  • LLDP.
  • DTP.
  • PAgP.
  • LACP.
  • PVST+.
  • RPVST+.
  • MST.

The topology is going to be very simple, two switches connected by a single link:

These are IOSv-L2 devices:

SW1#show version
Cisco IOS Software, vios_l2 Software (vios_l2-ADVENTERPRISEK9-M), Experimental Version 15.2(20200924:215240) [sweickge-sep24-2020-l2iol-release 135]
Copyright (c) 1986-2020 by Cisco Systems, Inc.
Compiled Tue 29-Sep-20 11:53 by sweickge


 Continue reading

Torero – Boots on the Ground Framework for Automation Sharing

Background One of the many useful things I came away with from AutoCon1 in Amsterdam was a "to-do" to investigate torero. Launched by Itential at NAF's AutoCon1 as a community based product and presented to the AutoCon community by Peter Sprygada in Amsterdam, one has to take notice and I did. You can see Mr. READ MORE

The post Torero – Boots on the Ground Framework for Automation Sharing appeared first on The Gratuitous Arp.

Ongoing Saga: How Much Money Will Be Spent On AI Chips?

Everybody knows that companies, particularly hyperscalers and cloud builders but now increasingly enterprises hoping to leverage generative AI, are spending giant round bales of money on AI accelerators and related chips to create AI training and inference clusters. …

Ongoing Saga: How Much Money Will Be Spent On AI Chips? was written by Timothy Prickett Morgan at The Next Platform.

1 2 3 4 … 3,632