Containerized Python Development – Part 2

This is the second part of the blog post series on how to containerize our Python development. In part 1, we have already shown how to containerize a Python service and the best practices for it. In this part, we discuss how to set up and wire other components to a containerized Python service. We show a good way to organize project files and data and how to manage the overall project configuration with Docker Compose. We also cover the best practices for writing Compose files for speeding up our containerized development process.

Managing Project Configuration with Docker Compose

Let’s take as an example an application for which we separate its functionality in three-tiers following a microservice architecture. This is a pretty common architecture for multi-service applications. Our example application consists of:

  • a UI tier – running on an nginx service
  • a logic tier – the Python component we focus on
  • a data tier – we use a mysql database to store some data we need in the logic tier

The reason for splitting an application into tiers is that we can easily modify or add new ones without having to rework the entire project.

A good way to Continue reading

Publish-Subscribe: Introduction to Scalable Messaging

Matthew O’Riordan A serial entrepreneur and seasoned developer with over 15 years of hands-on development experience. Matthew is the CEO of Ably, an Infrastructure-as-a-Service (IaaS) provider. He was co-founder and technical director of Aqueduct, a leading digital agency in London and Founder of easyBacklog, a SaaS agile backlog management tool. Matthew co-founded Econsultancy, a global digital marketing publishing, training and research business, with Ashley Friedlein and exited via a £25m trade sale to Centaur Media plc in 2012. The publish-subscribe (or pub/sub) messaging pattern is a design pattern that provides a framework for exchanging messages that allows for loose coupling and scaling between the sender of messages (publishers) and receivers (subscribers) on topics they subscribe to. Messages are sent (pushed) from a publisher to subscribers as they become available. The host (publisher) publishes messages (events) to channels (topics). Subscribers can sign up for the topics they are interested in. This is different from the standard request/response (pull) models in which publishers check if new data has become available. This makes the pub/sub method the most suitable framework for streaming data in real-time. It also means that dynamic networks can be built at internet scale. However, building a messaging infrastructure at Continue reading

U.S. Tribes Have until August 3rd to Apply to Help Bring Internet to Their Communities

See how the Makah Tribe launched an emergency network on EBS spectrum during COVID-19

The Makah Tribe has lived around Neah Bay at the northwest tip of what is now Washington State since time immemorial. It is a breathtaking landscape of dense rainforest and steep hills, far removed from any major urban center.

But for all its beauty, the hills, forests, and remoteness have made it difficult for the community to access quality high-speed Internet – and even cell and radio service.

In some areas, cell service was so poor that only certain spots worked: one community member had to go outside and stand beside a rhododendron bush to make a call or send a text. While Facebook is the main way people stay connected, many couldn’t access it. The local clinic struggled to use electronic records – it sometimes took upwards of 40 minutes just to get into the system. Even emergency responders, such as police and the fire department, couldn’t rely on the dispatch system that required Internet connectivity to operate.

And then the coronavirus began to sweep the world. The Makah closed the reservation to outsiders to protect the community. And its connectivity challenges became even more problematic. Continue reading

Jinja2 Tutorial – Part 4 – Template filters

This is part 4 of Jinja2 tutorial where we continue looking at the language features, specifically we'll be discussing template filters. We'll see what filters are and how we can use them in our templates. I'll also show you how you can write your own custom filters.

Jinja2 Tutorial series

Contents

Overview of Jinja2 filters

Let's jump straight in. Jinja2 filter is something we use to transform data held in variables. We apply filters by placing pipe symbol | Continue reading

Data-center survey: IT seeks faster switches, intelligent computing

The growth in data use and consumption means the needs of IT managers are changing, and a survey from Omdia (formerly IHS Markit) found data-center operators are looking for intelligence of all sorts, not just the artificial kind.Omdia analysts recently surveyed IT leaders from 140 North American enterprises with at least 101 employees working in North American offices and data centers and asked them what features they wanted the most in their networking technology.The results say respondents expect to more than double their average number of data-center sites between 2019 and 2021, and the average number of servers deployed in data centers is expected to double over the same timeline.To read this article in full, please click here

Tradeoffs Come in Threes

On a Spring 2019 walk in Beijing I saw two street sweepers at a sunny corner. They were beat-up looking and grizzled but probably younger than me. They’d paused work to smoke and talk. One told a story; the other’s eyes widened and then he laughed so hard he had to bend over, leaning on his broom. I suspect their jobs and pay were lousy and their lives constrained in ways I can’t imagine. But they had time to smoke a cigarette and crack a joke. You know what that’s called? Waste, inefficiency, a suboptimal outcome. Some of the brightest minds in our economy are earnestly engaged in stamping it out. They’re winning, but everyone’s losing. —Tim Bray

This, in a nutshell, is what is often wrong with our design thinking in the networking world today. We want things to be efficient, wringing the last little dollar, and the last little bit of bandwidth, out of everything.

This is also, however, a perfect example of the problem of triads and tradeoffs. In the case of the street sweeper, we might thing, “well, we could replace those folks sitting around smoking a cigarette and cracking jokes with a robot, making things Continue reading

The Week in Internet News: Hackers Target COVID-19 Research

Hacking the research: Intelligence agencies from the U.S., U.K., and Canada have accused a Russian hacking group of targeting organizations conducting COVID-19 research, the Washington Post reports. The so-called Cozy Bear hacking group is trying to steal vaccine research specifically, the intelligence groups say.

Hacking the tweets: Meanwhile, 130 of Twitter’s most high-profile accounts were targeted by hackers recently, with a few of them compromised, in an apparent bitcoin scam, the New York Post writes. Among the accounts targeted were Kanye West, Elon Musk, Barack Obama, and Warren Buffett. The hackers reportedly paid a Twitter employee to help them with the attack.

No data collection, please: The government of China is cracking down on apps that collect what it considers too much personal data, the South China Morning Post says. The government has ordered several tech companies, including Alibaba Group and Tencent, to remove non-compliant apps as soon as possible.

Broadband is fundamental: Microsoft CEO Satya Nadella has called broadband a “fundamental right” in an interview with CNN. Many rural areas in the U.S. still lack broadband, and that needs to change, he said. “If you think about the rural community today, they are going to Continue reading

Top Questions for Getting Started with Docker

Does Docker run on Windows?

Yes. Docker is available for Windows, MacOS and Linux. Here are the download links:

What is the difference between Virtual Machines (VM) and Containers?

This is a great question and I get this one a lot. The simplest way I can explain the differences between Virtual Machines and Containers is that a VM virtualizes the hardware and a Container “virtualizes” the OS. 

If you take a look at the image above, you can see that there are multiple Operating Systems running when using Virtual Machine technology. Which produces a huge difference in start up times and various other constraints and overhead when installing and maintaining a full blow operating system. Also, with VMs, you can run different flavors of operating systems. For example, I can run Windows 10 and a Linux distribution on the same hardware at the same time. Now let’s take a look at the image for Docker Containers.

As you can see in this image, we only have one Host Operating System installed on our infrastructure. Docker sits “on top” of the host operating system. Each application is then bundled in an Continue reading

Network Break 293: HPE Acquires Silver Peak; Dell Teases VMware Sale

Today's Network Break scrutinizes HPE's big payout for Silver Peak and Dell's plans for a possible sale of VMware. We also discuss new capabilities in VMware Cloud on AWS, a new synthetic monitoring service from Kentik, how NIST thinks "giga" is pronounced, and more.

The post Network Break 293: HPE Acquires Silver Peak; Dell Teases VMware Sale appeared first on Packet Pushers.

Phased Approach to Securing a Data Center

In the fight against relentless cyberattacks, organizations have long relied on traditional perimeter firewalls to protect sensitive workloads and information in the data center. But today, in the era of distributed applications and hybrid cloud environments, we know that perimeter defenses are not enough to stop cybercriminals.  

To improve security postures inside corporate networks — which means protecting against both bad actors who penetrate perimeter defenses and malicious insiders — organizations must monitor, detect, and block hostile east-west (internal) traffic using internal firewalls.  

To datenetwork and security professionals have generally viewed securing east-west traffic as too complex, expensive, and time-consuming for their brownfield, and even greenfield, data centers. At VMware, we agree with that perception: itcertainly true for organizations trying to detect and prevent the lateral movement of attackers by employing traditional, appliance-based perimeter firewalls as internal firewalls.  

There’s a Better Way to Secure the Data Center 

Instead of awkwardly forcing appliance-based firewalls to serve as internal firewallsorganizations should emploa distributed, scale-out internal firewall specifically Continue reading

Why I’m Helping Cloudflare Grow in Japan

Why I’m Helping Cloudflare Grow in Japan

If you'd like to read this post in Japanese click here.

Why I’m Helping Cloudflare Grow in Japan

I’m excited to say that I’ve recently joined the Cloudflare team as Head of Japan. Cloudflare has had a presence in Japan for a while now, not only with its network spanning the country, but also with many Japanese customers and partners which I’m now looking forward to growing with. In this new role, I’m focused on expanding our capabilities in the Japanese market, building upon our current efforts, and helping more companies in the region address and put an end to the technical pain points they are facing. This is an exciting time for me and an important time for the company. Today, I’m particularly eager to share that we are opening Cloudflare’s first Japan office, in Tokyo! I can’t wait to grow the Cloudflare business and team here.

Why I’m Helping Cloudflare Grow in Japan

Why Cloudflare?

The web was built 25 years ago. This invention changed the way people connected—to anyone and anywhere—and the way we work, play, live, learn, and on. We have seen this become more and more complex. With complexities come difficulties, such as ensuring security, performance, and reliability while online. Cloudflare is helping to solve these challenges that businesses Continue reading

How CEOs think

Recently, Twitter was hacked. CEOs who read about this in the news ask how they can protect themselves from similar threats. The following tweet expresses our frustration with CEOs, that they don't listen to their own people, but instead want to buy a magic pill (a product) or listen to outside consultants (like Gartner). In this post, I describe how CEOs actually think.


The only thing more broken than how CEOs view cybersecurity is how cybersecurity experts view cybersecurity. We have this flawed view that cybersecurity is a moral imperative, that it's an aim by itself. We are convinced that people are wrong for not taking security seriously. This isn't true. Security isn't a moral issue but simple cost vs. benefits, risk vs. rewards. Taking risks is more often the correct answer rather than having more security.

Rather than experts dispensing unbiased advice, we've become advocates/activists, trying to convince people that Continue reading

EIGRP Behavior with IP Unnumbered

Carl Zellers asked an excellent question on how EIGRP works when run over FlexVPN with IP unnumbered, considering that routers will not be on a common subnet. I thought this was a great question so I took some help from my great friend, the EIGRP guru, Peter Palúch.

First, let’s examine behavior when EIGRP is run on numbered interface. I have built a very simple lab consisting of three routers, R1, R2, and R3, where R1 and R3 are separated by R2. To demonstrate that EIGRP checks that incoming hellos are received on a common subnet, the following simple configurations were applied to R1 and R2:

R1:

interface GigabitEthernet1
 ip address 10.0.0.1 255.255.255.0
!
router eigrp LAB
 !
 address-family ipv4 unicast autonomous-system 64512
  !
  topology base
  exit-af-topology
  network 10.0.0.0 0.0.0.255
 exit-address-family

R2:

interface GigabitEthernet1
 ip address 10.0.1.1 255.255.255.0
!
router eigrp LAB
 !
 address-family ipv4 unicast autonomous-system 64512
  !
  topology base
  exit-af-topology
  network 10.0.1.0 0.0.0.255
 exit-address-family

This results in the well familiar messages on the console:

*Jul 15 08:53:20.966: %DUAL-6-NBRINFO: EIGRP-IPv4 64512: Neighbor 10.0.0.1 (GigabitEthernet1) is blocked: not  Continue reading

Cloudflare outage on July 17, 2020

Cloudflare outage on July 17, 2020

Today a configuration error in our backbone network caused an outage for Internet properties and Cloudflare services that lasted 27 minutes. We saw traffic drop by about 50% across our network. Because of the architecture of our backbone this outage didn’t affect the entire Cloudflare network and was localized to certain geographies.

The outage occurred because, while working on an unrelated issue with a segment of the backbone from Newark to Chicago, our network engineering team updated the configuration on a router in Atlanta to alleviate congestion. This configuration contained an error that caused all traffic across our backbone to be sent to Atlanta. This quickly overwhelmed the Atlanta router and caused Cloudflare network locations connected to the backbone to fail.

The affected locations were San Jose, Dallas, Seattle, Los Angeles, Chicago, Washington, DC, Richmond, Newark, Atlanta, London, Amsterdam, Frankfurt, Paris, Stockholm, Moscow, St. Petersburg, São Paulo, Curitiba, and Porto Alegre. Other locations continued to operate normally.

For the avoidance of doubt: this was not caused by an attack or breach of any kind.

We are sorry for this outage and have already made a global change to the backbone configuration that will prevent it from being able to occur Continue reading