How do you explain the unreasonable effectiveness of cloud security?

With the enormous attack surface of cloud providers like AWS, Azure, and GCP, why aren't there more security problems? Data breaches and cyber attacks occur daily. How do you explain the unreasonable effectiveness of cloud security?

Google has an ebook on their security approach; Microsoft has some web pages. Both are the equivalent of that person who is disgustingly healthy and you ask them how they do it and they say "I don't know. I just eat right, exercise, and get plenty of sleep." Not all that useful. Most of us want a hack, a trick to good health. Who wants to eat right? 

I'm sure Amazon also eats right, exercises, and gets plenty of sleep (probably not the people who work there), but AWS also has a secret that when that disgustingly healthy person starts talking about at a party, you just can't help leaning in and listening. 

What's the trick to 6-pack security? Proving systems correct. Does your datacenter do that? I didn't think so. AWS does. 

Dr. Byron Cook gave an enthusiastic talk on Formal Reasoning about the Security of Amazon Web Service. He's clearly excited about finally applying his research in a Continue reading

Sponsored Post: Twitch, InMemory.Net, Triplebyte, Etleap, Stream, Scalyr, MemSQL

Who's Hiring? 

• Twitch's commerce team in San Francisco is looking to hire senior developers to keep up with rapidly increasing demand for our Subscriptions and Payment platform. Engineers will be tasked with building new products and features to solve business and ecommerce challenges as we're dealing with engaging problems at a massive scale and will create solutions that impact millions of people around the world. Apply here. 


• Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.


• Need excellent people? Advertise your job here! 

Fun and Informative Events

• Advertise your event here!

Cool Products and Services

• InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net

• For heads of IT/Engineering responsible for building an analytics infrastructure, Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike Continue reading

Stuff The Internet Says On Scalability For September 14th, 2018

Hey, it's HighScalability time:

The Cloud Native Interactive Landscape is fecund. You are viewing 581 cards with a total of 1,237,157 stars, market cap of $6.86T and funding of $20.1B. (changelog)

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

• 72: signals sensed from a distant galaxy using AI; 12M: reddit posts per month; 10 trillion: per day Google generated test inputs with 100s of servers for several months using OSS-Fuzz; 200%: growth in Cloud Native technologies used in production; $13 trillion: potential economic impact of AI by 2030; 1.8 trillion: plastic pieces eaten by giant garbage pac-man; 100: min people needed to restart humanity; 


• Quotable Quotes:
  • Joel Hruska: Farmers in California have lost the fight to be allowed to repair their own tractors and equipment thanks to the capitulation Continue reading

Public Cloud Postcentralization is the Thin Edge of the Wedge into the Enterprise

Like an amoeba the public cloud is extending fingerlike projections to the edge in a new kind of architecture that creates a world spanning distributed infrastructure under one centralized management, billing, and security domain.

This issue—the deep nature of centralization—came up as a comment on my article What Do You Believe Now That You Didn't Five Years Ago? Centralized Wins. Decentralized Loses.

Centralization can refer to the locus of computation, but it also refers to a boundary, to a domain of control.

Facebook, Netflix, and Google are all distributed across much of the world, but they are still centralized services because control is centralized. You know this because in a browser, no matter where you are in the world, you navigate to facebook.com, netflix.com, or google.com, you never enter the URL for independent shards, yet all your data and services magically follow you around like a hyperactive puppy. 

That's the world we've come to expect. That's how services built on a cloud work.

In an unexpected development, the public cloud is expanding control out to the edge. As I wrote in Stuff The Internet Says On Scalability For July 27th, 2018:

The Anna Key-Value Store Now Has 355x the Performance of DynamoDB for the Dollar

 

New databases used to be announced seemingly every week. While database neogenesis has slowed down considerably, it has not gone necrotic.

RISELabs, those wonderfully innovative folks over at Berkeley, have uplifted their Anna datatabase—a shared-nothing, thread-per-core architecture to achieve lightning-fast speeds by avoiding all coordination mechanisms—to become cloud-aware.

What's changed?

Anna is not only incredibly fast, it’s incredibly efficient and elastic too: an autoscaling, multi-tier, selectively-replicating cloud service. All that adaptivity means that Anna ramps down resource consumption for cold things, and ramps up consumption for hot things. You get all the multicore Anna performance you want, but you don’t pay for what you don’t need.

Just to throw out some numbers, we measured Anna providing 355x the performance of DynamoDB for the dollar. No, I don’t think that is because AWS is earning a 355x margin on DynamoDB! The issue is that Anna is now orders of magnitude more efficient than competing systems, in addition to being orders of magnitude faster.

Using Anna v0 as an in-memory storage engine, we set out to address the cloud storage problems described Continue reading

Stuff The Internet Says On Scalability For September 7th, 2018

Hey, it's HighScalability time:

Get antsy waiting 60 seconds for a shot? Imagine taking over 300,000 photos over 14 years, waiting for Mount Colima to erupt. Sergio Tapiro studied, waited, and snapped.

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

• 3.5 Pflop/s: fully synchronous tensorflow data-parallel training; 3.3 million: new image/caption training set; 32,408,715: queries sent to Pwned Passwords; 53%: Memory ICs Total 2018 Semi Capex; 11: story Facebook datacenter prison in Singapore; $740,357: ave cost of network downtime; 


• Quotable Quotes:
  • @BenedictEvans: Recorded music: $18 billion. Cars: $1 trillion. Retail: $20 trillion.
  • @JoeEmison: Lies that developers tell (themselves): (1) This is the best stack/IaaS for us to use [reality: I know it and want to start now] (2) DevOps doesn’t matter until scaling [you’ll spend 30% of your time dealing with ops Continue reading

Sponsored Post: NationBuilder, Twitch, InMemory.Net, Triplebyte, Etleap, Stream, Scalyr, MemSQL

Who's Hiring? 

• NationBuilder — if you’re a systems engineer, SRE or DevOps focused developer and have been looking for a place where you can help other people while still working in tech? We can give that opportunity. Please apply here. 


• Twitch's commerce team in San Francisco is looking to hire senior developers to keep up with rapidly increasing demand for our Subscriptions and Payment platform. Engineers will be tasked with building new products and features to solve business and ecommerce challenges as we're dealing with engaging problems at a massive scale and will create solutions that impact millions of people around the world. Apply here. 


• Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.


• Need excellent people? Advertise your job here! 

Fun and Informative Events

• Advertise your event here!

Cool Products and Services

• InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use Continue reading

Stuff The Internet Says On Scalability For August 31st, 2018

Hey, it's HighScalability time:

This mind blowing creation is from John Williamson. It's the first million integers, represented as binary vectors indicating their prime factors, laid out with UMAP. No, I really have no idea what that means either, but it did make me consider that our universe could be created by an algorithm. What are the wiggly cycles on the periphery? Groups of numbers that share a minimum amount of prime factors, further out groups are numbers that have increasing amounts of shared prime factors. So the primes are at the core, ungrouped as they have no prime factors to use to join groups. Primorials should be furthest out.

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

• 800 million: internet users in China; 19,000: hours of audio from historic Apollo 11 mission; 2: mph Frodo walking pace to Mordor;  Continue reading

Auth0 Architecture: Running In Multiple Cloud Providers And Regions

This is article was written by Dirceu Pereira Tiegs, Site Reliability Engineer at Auth0, and originally was originally published in Auth0.

Auth0 provides authentication, authorization, and single sign-on services for apps of any type (mobile, web, native) on any stack. Authentication is critical for the vast majority of apps. We designed Auth0 from the beginning so that it could run anywhere: on our cloud, on your cloud, or even on your own private infrastructure.

In this post, we'll talk more about our public SaaS deployments and provide a brief introduction to the infrastructure behind auth0.com and the strategies we use to keep it up and running with high availability. 

A lot has changed since then in Auth0. These are some of the highlights:

• We went from processing a couple of million logins per month to 1.5+ billion logins per month, serving thousands of customers, including FuboTV, Mozilla, JetPrivilege, and more.

• We implemented new features like custom domains, scaled bcrypt operations, vastly improved user search, and much more.

• The number of services that compose our product in order to scale our organization and handle the increases in traffic went from under Continue reading

Stuff The Internet Says On Scalability For August 24th, 2018

Hey, it's HighScalability time:

 

Images from a far flung galaxy? Nope. It's the mind blowing swirling beauty of ink in motion. 

 

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

• $10 billion: damages in worlds largest cyberattack; .5%: bitcoins use of all the electricity on earth; 1/200th: Verizon throttling California firefighters for leverage; 4.6%: YC companies reaching $100M+ valuation; 45: ave age of successful startup founder; $250,000: monthly take from browser-based Monero mining; 300+: 3D digitizatized Greek and Roman sculptures; 80: employees are chipped at a company; 100k: bike graveyard from failed startups; 70%: executives think they are block chain experts; $7 billion: Slack valuation; 120: AWS instance types; 27.6 petabytes: Microsoft’s undersea data center webcam of swimming fish; 42%: product is the reason startups fail; $334bn: Continue reading

What do you believe now that you didn’t five years ago? Centralized wins. Decentralized loses.

Decentralized systems will continue to lose to centralized systems until there's a driver requiring decentralization to deliver a clearly superior consumer experience. Unfortunately, that may not happen for quite some time.

I say unfortunately because ten years ago, even five years ago, I still believed decentralization would win. Why? For all the idealistic technical reasons I laid out long ago in Building Super Scalable Systems: Blade Runner Meets Autonomic Computing In The Ambient Cloud.

While the internet and the web are inherently decentralized, mainstream applications built on top do not have to be. Typically, applications today—Facebook, Salesforce, Google, Spotify, etc.—are all centralized.

That wasn't always the case. In the early days of the internet the internet was protocol driven, decentralized, and often distributed—FTP (1971), Telnet (<1973), FINGER (1971/1977),  TCP/IP (1974), UUCP (late 1970s) NNTP (1986), DNS (1983), SMTP (1982), IRC(1988), HTTP(1990), Tor (mid-1990s), Napster(1999), and XMPP(1999).

We do have new decentalized services: Bitcoin(2009), Minecraft(2009), Ethereum(2104), IPFS(2015), Mastadon(2016), and PeerTube(2018). We're still waiting on Pied Piper to deliver the decentralized internet. 

On an evolutionary timeline decentralized systems are neanderthals; centralized systems are the humans. Neanderthals came first. Humans may have interbred with neanderthals, humans may have even killed off the neanderthals, but Continue reading

Sponsored Post: NationBuilder, Twitch, InMemory.Net, Triplebyte, Etleap, Stream, Scalyr, MemSQL

Who's Hiring? 

• NationBuilder — if you’re a systems engineer, SRE or DevOps focused developer and have been looking for a place where you can help other people while still working in tech? We can give that opportunity. Please apply here. 


• Twitch's commerce team in San Francisco is looking to hire senior developers to keep up with rapidly increasing demand for our Subscriptions and Payment platform. Engineers will be tasked with building new products and features to solve business and ecommerce challenges as we're dealing with engaging problems at a massive scale and will create solutions that impact millions of people around the world. Apply here. 


• Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.


• Need excellent people? Advertise your job here! 

Fun and Informative Events

• Advertise your event here!

Cool Products and Services

• InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use Continue reading

Stuff The Internet Says On Scalability For August 17th, 2018

Hey, it's HighScalability time:

 

The amazing Zoomable Universe from 10^27 meters—about 93 billion light-years—down to the subatomic realm, at 10^-35 meters.

 

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

• 2.24x10^32T: joules needed by the Death Star to obliterate Alderaan, which would liquify everyone in the Death Star; 13 of 25: highest paying jobs are in tech; 70,000+: paid Slack workspaces; 13: hours ave american sits; $13.5 million: lost in ATM malware hack; $1.5 billion: cryptocurrency gambling ring busted in China; $8.5B: Auto, IoT, Security startups; 10x: infosec M&A; 1,000: horsepower needed to fly a jet suit; 30% Google's energy savings from AI control of datacenters;


• Quotable Quotes:
  • The Jury Is In: From the security point of view, the monolithic OS design is flawed and a root cause of the Continue reading

What do you believe now that you didn’t five years ago?

Decentralized systems will continue to lose to centralized systems until there's a driver requiring decentralization to deliver a clearly superior consumer experience. Unfortunately, that may not happen for quite some time.

I say unfortunately because ten years ago, even five years ago, I still believed decentralization would win. Why? For all the idealistic technical reasons I laid out long ago in Building Super Scalable Systems: Blade Runner Meets Autonomic Computing In The Ambient Cloud.

While the internet and the web are inherently decentralized, mainstream applications built on top do not have to be. Typically, applications today—Facebook, Salesforce, Google, Spotify, etc.—are all centralized.

That wasn't always the case. In the early days of the internet the internet was protocol driven, decentralized, and often distributed—FTP (1971), Telnet (<1973), FINGER (1971/1977),  TCP/IP (1974), UUCP (late 1970s) NNTP (1986), DNS (1983), SMTP (1982), IRC(1988), HTTP(1990), Tor (mid-1990s), Napster(1999), and XMPP(1999).

We do have new decentalized services: Bitcoin(2009), Minecraft(2009), Ethereum(2104), IPFS(2015), Mastadon(2016), and PeerTube(2018). We're still waiting on Pied Piper to deliver the decentralized internet. 

On an evolutionary timeline decentralized systems are neanderthals; centralized systems are the humans. Neanderthals came first. Humans may have interbred with neanderthals, humans may have even killed off the neanderthals, but Continue reading

Stuff The Internet Says On Scalability For August 10th, 2018

Hey, it's HighScalability time (out Thur-Fri, so we're going early):

 

London Maker Faire 1851—The Great Exhibition—100,000 objects, displayed along more than 10 miles, by over 15,000 contributors.

 

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

• 90%: accuracy predicting gender from retinal image; $1 billion: Ebay sales per quarter from AI; $78 billion: global AI software market by 2025; $75m: penalty for botched SAP upgrade; 35 million: m^3 of mud dredged out of the Dutch waterways; 138 terabytes: memory per square inch; 500 million: Uber metrics per second; 22x: new faster JSON Sparser; 


• Quotable Quotes:
  • @IanColdwater: The JIRA tickets will continue until morale improves
  • @david_perell: Three crazy stats from @mikedariano’s newsletter. 1. People watch more Minecraft hours than the NBA, NHL, NFL, and MLB combined.  2. Only 26 countries have more people than Continue reading

Case Study: Pokémon GO on Google Cloud Load Balancing

There are a lot of cool nuggets in Google's New Book: The Site Reliability Workbook. If you haven't put it on your reading list, here's a tantalizing excerpt from CHAPTER 11 Managing Load by Cooper Bethea, Gráinne Sheerin, Jennifer Mace, and Ruth King with Gary Luo and Gary O’Connor.

Niantic launched Pokémon GO in the summer of 2016. It was the first new Pokémon game in years, the first official Pokémon smartphone game, and Niantic’s first project in concert with a major entertainment company. The game was a runaway hit and more popular than anyone expected—that summer you’d regularly see players gathering to duel around landmarks that were Pokémon Gyms in the virtual world.

Pokémon GO’s success greatly exceeded the expectations of the Niantic engineering team. Prior to launch, they load-tested their software stack to process up to 5x their most optimistic traffic estimates. The actual launch requests per second (RPS) rate was nearly 50x that estimate—enough to present a scaling challenge for nearly any software stack. To further complicate the matter, the world of Pokémon GO is highly interactive and globally shared among its users. All players in a given area see the same view of the game Continue reading

Sponsored Post: NationBuilder, Twitch, InMemory.Net, Triplebyte, Etleap, Scalyr, MemSQL

Who's Hiring? 

• NationBuilder — if you’re a systems engineer, SRE or DevOps focused developer and have been looking for a place where you can help other people while still working in tech? We can give that opportunity. Please apply here. 


• Twitch's commerce team in San Francisco is looking to hire senior developers to keep up with rapidly increasing demand for our Subscriptions and Payment platform. Engineers will be tasked with building new products and features to solve business and ecommerce challenges as we're dealing with engaging problems at a massive scale and will create solutions that impact millions of people around the world. Apply here. 


• Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.


• Need excellent people? Advertise your job here! 

Fun and Informative Events

• Advertise your event here!

Cool Products and Services

• InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use Continue reading

Stuff The Internet Says On Scalability For August 3rd, 2018

Hey, it's HighScalability time:

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

• $1 trillion: Apple; 45: mean founder age for fastest growing new ventures; 1 trillion: files created by Trinity in 2 minutes;  $3.93 billion: Baidu's AI driven quarterly revenues; 13.7: microsecond error in GPS timestamps for nearly 14 hours; 240 million: Apple CarPlay cars by 2023; 99%: confidence setting Continue reading

Stuff The Internet Says On Scalability For July 27th, 2018

Hey, it's HighScalability time:

 

Startup opportunity? Space Garbage Collection service. 18,000+ known Near-Earth Objects. (NASA)

 

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

• 143 billion: daily words Google Translated; 73%: less face-to-face interaction in open offices; 10 billion: Uber trips; 131M: data breach by Exactis; $123 billion: Facebook value loss is 4 Twitters and 7 snapchats; $9.1B: spent on digital gaming across all platforms; 20-km: width of lake on mars; 1 billion: Google Drive users; $32.7 billion: Alphabet revenues; $110bn: Microsoft total revenue; $1.9 million: buy the Brady Bunch house; 5.623 trillion: Amazon Sable requests handle on prime day; 91%: Facebook's advertising revenue on mobile; $18 billion: deep learning market by 2024; 


• Quotable Quotes:
  • Ryan Cash: The App Store has changed the Continue reading

Google’s New Book: The Site Reliability Workbook

Google has released a new book: The Site Reliability Workbook — Practical Ways to Implement SRE.

It's the second book in their SRE series. How is it different than the previous Site Reliability Engineering book?

David Rensin, a SRE at Google, says:

It's a whole new book.  It's designed to sit next to the original on the bookshelf and for folks to bounce between them -- moving between principle and practice.

And from the preface:

The purpose of this second SRE book is (a) to add more implementation detail to the principles outlined in the first volume, and (b) to dispel the idea that SRE is implementable only at “Google scale” or in “Google culture.”

The Site Reliability Workbook weighs in at a hefty 508 pages and roughly follows the structure of the first book. It's organized into three different parts: Foundations, Practices, and Processes. There are three appendices: Example SLO Document, Example Error Budget Policy, and Results of Postmortem Analysis.

The table of content is quite detailed, but here are the chapter titles:

1. How SRE Relates to DevOps.  
2. Implementing SLOs.
3. SLO Engineering Case Studies.
4. Monitoring.
5. Alerting on SLOs.
6. Eliminating Toil.
7. Simplicity.
8. On-Call.
9. Incident Response.
10. Postmortem Continue reading