New Book: Explain the Cloud Like I’m 10

What is the cloud? Why is it called a cloud? How does the cloud work? What does it mean when something is 'in the cloud'?

I wrote a new book: Explain the Cloud Like I'm 10, answering those questions for the complete beginner. It makes the perfect gift for Halloween. And Thanksgiving. And Christmas. Oh, and birthdays too!

The irony is, if you read HighScalability, you're not the target audience :-) Explain the Cloud Like I'm 10 is for people who hear about the cloud everyday and have wondered what it is.

Talking with people outside the tech bubble I've found the cloud is still a mystery. I think that's because almost every explanation of the cloud I could find was a rewording of the same unhelpful technobabble.

In Explain the Cloud Like I'm 10 I've used a lot of pictures and a lot of examples. I go slow and easy. I try really hard to build up an intuitive understanding of what the cloud is and how it works.

If you know of anyone who might benefit from a book like this, I'd appreciate it if you'd pass it on.

thanks! 

Stuff The Internet Says On Scalability For October 20th, 2017

Hey, it's HighScalability time: 

 

Cassini's last image of Saturn, stitched together from 11 color composites, each a stack of three images taken in red, green, and blue channels. (Jason Major)

• 21 billion: max bitcoins ever; #2: Alibaba's cloud?; 1M MWh: Amazon Wind Farm Texas with 100 Turbines is live; $1000: cost to track someone with mobile ads; 20%: ebook sales of total; 17: qubit chip; 30%: Uber deep learning speedup using RDMA; 


• Quoteable Quotes:
  • Tim O'Reilly: So what makes a real unicorn of this amazing kind? 1.  It seems unbelievable at first. 2.  It changes the way the world works. 3.  It results in an ecosystem of new services, jobs, business models, and industries.
  • @rajivpant: AlphaGo has already beaten two of the world's best players. But the new AlphaGo Zero began with a blank Go board and no data apart from the rules, and then played itself. Within 72 hours it was good enough to beat the original AlphaGo by 100 games to zero!
  • @swardley: Containers aka winning battle but losing the war. AMZN joining Continue reading

ButterCMS Architecture: a Mission-Critical API Serving Millions of Requests per Month

This is a guest post by Jake Lumetta, co-founder and CEO of ButterCMS.

ButterCMS lets developers add a content management system to any website in minutes. Our business requires us to deliver near-100% uptime for our API, but after multiple outages that nearly crippled our business, we became obsessed with eliminating single points of failure. In this post, I’ll discuss how we use Fastly’s edge cloud platform and other strategies to make sure we keep our customers’ websites up and running.

At its core, ButterCMS offers:

• A dashboard for content editors

• A JSON API for fetching content

• SDK’s for integrating ButterCMS into native code

ButterCMS Tech Stack

Stuff The Internet Says On Scalability For October 13th, 2017

Hey, it's HighScalability time: 

 

Tech is transforming how food is being grown. Lots of opportunity for local nerdy production. Greenhouses even look like dartacenters! (This Tiny Country Feeds the World)

• 320 trillion: ops/second in Nvidia driverless-car computer; .25%: Lambda invocations impacted by cold starts; $30,000: monthly take hijacking computers to mine cryptocurrency; 400 gbps: Ethernet standard to be ratified this year; 2.1 million: MySQL 8.0 query/second; 100,000: Kiva robots owned by Amazon; 50,000: greenhouses in Egypt's new farm city; 100 petabytes: new hard drives ordered by Backblaze; 20 million: max Bitcoin users per month; 662 million: unused vacation days in US; 92 billion: Pornhub views per year; 1,000: new Facebook hires to review ads; 12 milion: Tinder matches per day; $1 billion: Google training grants; 


• Quotable Quotes: 
  • @toddmotto: Space X sends a rocket up into space. Lands back on its feet back on earth 7minutes later. I can't even run an npm install in that time.
  • nappy-doo: Years ago, I started at Google, and was in Charlie's Continue reading

Sponsored Post: Loupe, Etleap, Aerospike, Stream, Scalyr, VividCortex, Domino Data Lab, MemSQL, InMemory.Net, Zohocorp

Who's Hiring? 

• Need excellent people? Advertise your job here! 

Fun and Informative Events

• October 10 Live Webinar. Fast & Frictionless - The Decision Engine for Seamless Digital Business. Join us for a live webinar on Tuesday, October 10 at 11:00 am Pacific Time featuring guest speakers Michele Goetz, Principal Analyst at Forrester Research, and Matthias Baumhof, VP Worldwide Engineering at ThreatMetrix®. A positive customer experience is required for successful enterprise digital transformation. Digital businesses depend on speed and efficiency to drive operational decisions. Making faster, accurate, and real-time customer trust decisions removes friction and delivers superior business outcomes. In this session, you’ll learn: How risk-based authentication leveraging digital identities is key to empowering customer transactions; How real-time customer trust decisions can reduce fraud and improve customer satisfaction; How a high performance Hybrid Memory Architecture (HMA) database helps continuously evaluate across a multitude of factors to drive decisioning at the lowest operational cost. Register now.


• Advertise your event here!

Cool Products and Services

• .NET developers dealing with Errors in Production: You know the pain of troubleshooting errors with limited time, limited information, and limited tools. Managers want to know what’s wrong right away, users don’t want to provide Continue reading

What will programming look like in the future?

Maybe programming will look something like the above video. Humans and AIs working together to produce software better than either can separately.

The computer as a creative agent, working in tandem with a human partner, to produce software, in a beautiful act of co-creation.

The alternative vision—The Coming Software Apocalypse—is a dead end. Better requirements and better tools have already been tried and found wanting. Requirements are a trap. They don't work. Requirements are no less complex and undiscoverable than code. Tools are another trap. Tools are just code that encode an inflexible solution to a problem that's already been solved.

Admittedly, I'm cheating. I have no idea how any of this will work, but here are the seeds of how it has already started:

• Jeff Dean On Large-Scale Deep Learning At Google 
• Machine Learning Driven Programming: A New Programming For A New World

Here's what we do know: neither tools or requirements are a silver bullet, they are a method of incrementally improving software quality. Software production quantity is not increased at all.

What we need is a manufacturing process that puts software production on an exponential curve. The only conceivable tool we have at Continue reading

Stuff The Internet Says On Scalability For October 6th, 2017

Hey, it's HighScalability time: 

 

LiDAR sees an enchanted world. (Luminar)

 

If you like this sort of Stuff then please support me on Patreon.

• 14TB: Western Digital Hard Drive; 3B: Yahoo's perfidy; ~80%: companies traded on U.S. stock market 1950-2009 were gone by 2009; 21%: conversion increase with AI-enabled site personalisation; $1 billion: US Air Force jets off the cloud; 1 billion: iOS devices in use; 1000x: new DeepMind WaveNet model produces 20 seconds of higher quality audio in 1 second; 96: vCPUs on new GCE machine type, with 624GB of memory; 


• Quotable Quotes:
  • fusiongyro: The amount of incipient complexity in programming has been growing, not going down. What's more complex, "hello, world" to the console in Python, or "hello world" in a browser with the best and newest web stack? Mobility and microservices create lots of new edge cases and complexity—do non-programmers seem particularly well-equipped to handle edge cases to you? The problem has never really been the syntax—if it were, non-programmers would have made great strides with Applescript and SQL, and we'd all be building PowerBuilder libraries for a living. The problem is Continue reading

Ripple: The Most (Demonstrably) Scalable Blockchain

Ripple’s XRP Ledger is a blockchain-based payment network that transfers funds between any type of currency within a few seconds with average transaction costs of a fraction of a penny. The core of this peer-to-peer network is an open source C++ application called rippled. Ripple’s goal is to supplant the world’s existing legacy payment networks. As such, scalability is a continuous goal. This document describes how the rippled team has integrated performance engineering into its development processes, and how this has contributed to throughput gains of over 1000%.

Performance engineering practices deliver benefits in addition to measurable performance gains. These include the ability to report on the capabilities of the software so that users can feel confident that their needs will be met by the system. Performance engineering informs capacity planning and optimal configuration of environments to support the application. Many performance problems are caught and addressed before customers notice them. As process automation improves, each change to the software can be quickly assessed for improvement or regression. This methodology also makes better use of developer time by helping choose the most effective tasks for improving performance. Any software project serious about supporting global scale should integrate performance engineering Continue reading

Stuff The Internet Says On Scalability For September 29th, 2017

Hey, it's HighScalability time: 

 

Latency Numbers Every Programmer Should Know plotted over time. Click on and move the slider to see changes. There were a lot more blocks in 1990.

 

If you like this sort of Stuff then please support me on Patreon.

• 1040: undergrads enrolled in Stanford's machine learning class; 39: minutes to travel from New York to Shanghai on Elon's rocket ride; $625,000: in stolen electronic-grade polysilicon; 160: terabits of data per second for Microsoft's new Trans-Atlantic Subsea Cable; 8K: people in Microsoft's AI group; 110%: increase in ICS/SCADA attacks from 2016 to 2017; 2 million: advertisers on Instagram; ~70%: savings using new Spot instance checkpointing; 10,000: nuts a year stored by a fox squirrel; $22.1 billion: IaaS market in 2016; 


• Quotable Quotes:
  • @patio11: Wife: "Hold hands when crossing the street." *2 year old grabs own hands* "OK Mommy." Me: "Oh you're going to be so good at programming."
  • Charlie Demerjian: Intel’s “new” 8th Gen CPUs are a stopgap OEM placation to cover for a failed process, but they do bring some advances. As SemiAccurate sees it, Intel took .023 steps forward with Continue reading

Aligning Your Team around Microservices When There’s No Precise Definition

This is a guest post by Roger Jin, Software Architect at ButterCMS and co-author of Microservices for Startups.

For a profession that stresses the importance of naming things well, we've done ourselves a disservice with microservices. The problem is that that there is nothing inherently "micro" about microservices. Some can be small, but size is relative and there's no standard of unit of measure across organizations. A "small" service at one company might be one million lines of code while far less at another.

Some argue that microservices aren’t a new thing at all and rather a rebranding of Service Oriented Architectures, while others advocate for viewing microservices as an implementation of SOA similar to how Scrum is an implementation of Agile.

How do you align your team when no precise definitions of microservices exist? The most important thing when talking about microservices on a team is to ensure that you are grounded in a common starting point.

But ambiguous definitions don’t help with this. It would be like trying to put Agile into practice without context for what you are trying to achieve, or an understanding of precise methodologies like Scrum.

Finding common ground 

Sponsored Post: Loupe, Etleap, Aerospike, Stream, Scalyr, VividCortex, Domino Data Lab, MemSQL, InMemory.Net, Zohocorp

Who's Hiring? 

• Advertise your job here! 

Fun and Informative Events

• October 10 Live Webinar. Fast & Frictionless - The Decision Engine for Seamless Digital Business. Join us for a live webinar on Tuesday, October 10 at 11:00 am Pacific Time featuring guest speakers Michele Goetz, Principal Analyst at Forrester Research, and Matthias Baumhof, VP Worldwide Engineering at ThreatMetrix®. A positive customer experience is required for successful enterprise digital transformation. Digital businesses depend on speed and efficiency to drive operational decisions. Making faster, accurate, and real-time customer trust decisions removes friction and delivers superior business outcomes. In this session, you’ll learn: How risk-based authentication leveraging digital identities is key to empowering customer transactions; How real-time customer trust decisions can reduce fraud and improve customer satisfaction; How a high performance Hybrid Memory Architecture (HMA) database helps continuously evaluate across a multitude of factors to drive decisioning at the lowest operational cost. Register now.


• Advertise your event here!

Cool Products and Services

• .NET developers dealing with Errors in Production: You know the pain of troubleshooting errors with limited time, limited information, and limited tools. Managers want to know what’s wrong right away, users don’t want to provide log data, and Continue reading

Stuff The Internet Says On Scalability For September 22nd, 2017

Hey, it's HighScalability time: 

 

Ever feel like howling at the universe? (Greg Rakozy)

 

If you like this sort of Stuff then please support me on Patreon.

• 10 billion: API calls made every second in Google datacenters; $767,758,000,000: collected by Apple on iPhones sold to the end of June; 20: watts of power consumed by human brain, autonomous vehicles peak at 3000 watts; 59%: drop in leads using AMP; 27%: success rate of AIs guessing passwords; 2.8 kilometers: distance devices running on almost zero power can xmit using backscatter; 96: age at which Lotfi Zadeh, inventor of Fuzzy Logic, passed away; 35%: store time series data in a RDBMS; $1.1 billion: Google's spend on self-driving tech;  $5.1 billion: Slack valuation; 15%: bugs reduced by strong typing; ~1 ft: new smartphone GPS accuracy; 


• Quotable Quotes:
  • Napoleon: [Sir Hudson Lowe] was a man wanting in education and judgment. He was a stupid man, he knew nothing at all of the world, and like all men who knew nothing of the world, he was suspicious and jealous.
  • Rich Werner: Data center operations, to me, is 362 Continue reading

Stuff The Internet Says On Scalability For September 15th, 2017

Hey, it's HighScalability time: 

 

Earth received Cassini’s final signal at 7:55am ET. Let's bid a fond farewell. After a 13-year tour of duty, job well done!

 

If you like this sort of Stuff then please support me on Patreon.

• 12.9 million: DynamoDB requests per second on Prime Day; 4 billion: transistors on Apple's A11 Bionic chip; 4x: extreme weather events since 1970; 51: qubit device; 50%: Messenger.com converted to Reason; 56.6 million: US cord cutters; 5000: bikes abandoned at Burning Man; 500 million: yearly visitors to Apple stores; 30 min: time to send one HD color image from Mars to Earth; 


• Quoteable Quotes:
  • @randyshoup: Interesting idea of a *Negative* MTTR by @adrianco: notice something is going to fail and proactively fix it before it breaks!
  • @rob_pike: "The Equifax executives who let my data be stolen will probably suffer fewer consequences than I will for an overdue library book." @nytimes
  • @avantgame: on weaponized social media: "We’re in an information war with Russia. It’s time we started acting like it."
  • Jamie Dimon: It's [Bitcoin] worse than tulip bulbs. It won't end well. Someone is going Continue reading

Have you noticed there’s a lot more collaboration going on these days? Why?

Thanks to zero marginal cost digital production methods, we're seeing content markets—for the first time—develop in conditions free from supply and price constraints.

In the process we've learned something: consumers have an unquenchable thirst for new content; content creators are willing to oblige with an equally prodigious stream of new content; platforms that best control access to the customer are the biggest winners; the reward for content creators varies drastically by medium and platform.

For consumers, life is now a streaming fixed priced buffet of unending variety and diversion.

For producers, the changes have been terrifying. Old modes have crumbled, leaving everyone scrambling to figure out what, if anything, comes next.

To adapt, content creators are learning to exploit capture loops, bundling, and collaboration to extract money from a digital economy that has collectively decided it rarely wants to pay artists directly for their content anymore.

The most highly evolved form of digital content platform strategies can be found in the book market. Why? Because Amazon.

Kindle Unlimited is the Clear Platform Winner

Sponsored Post: Close.io, Loupe, Etleap, Aerospike, Stream, Scalyr, VividCortex, Domino Data Lab, MemSQL, InMemory.Net, Zohocorp

Who's Hiring? 

• Close.io is a ~25 person fully remote team that is profitable and building a product our customers love! We’re hiring Senior Backend Developers to join our team. Our backend tech stack currently includes Python (Flask, Gunicorn, TaskTiger), Elasticsearch, MongoDB, Postgres, and Redis running in Docker/Kubernetes on AWS. Learn more and apply here!


• Advertise your job here! 

Fun and Informative Events

• Advertise your event here!

Cool Products and Services

• .NET developers dealing with Errors in Production: You know the pain of troubleshooting errors with limited time, limited information, and limited tools. Managers want to know what’s wrong right away, users don’t want to provide log data, and you spend more time gathering information than you do fixing the problem. To fix all that, Loupe was built specifically as a .NET logging and monitoring solution. Loupe notifies you about any errors and tells you all the information you need to fix them. It tracks performance metrics, identifies which errors cause the greatest impact, and pinpoints the root causes. Learn more and try it free today.


• Enterprise-Grade Database Architecture. The speed and enormous scale of today’s real-time, mission critical applications has exposed gaps in legacy database technologies. Continue reading

Stuff The Internet Says On Scalability For September 8th, 2017

Hey, it's HighScalability time: 

 

May you live in interesting times. China games swarming drone attacks. Portable EMP anyone? (Tech in Asia)

 

If you like this sort of Stuff then please support me on Patreon.

• 100GB: entire corpus of articles written at the NY Times; 80GB: data for one human genome; 3%: Linux desktop market share; 3.5M: fake Wells Fargo accounts; $18,000: world’s most expensive vacuum; 2000: Netflix recommender taste groups; 27%: year-over year-growth rate of Python on SO; 4M: Time Warner hacked; 143M: Equifax hacked; $800M: ICO funding in Q2; $257M: Filecoin ICO; 


• Quotable Quotes:
  • Brendan Gregg: jobs are also migrating from both Solaris and Linux to cloud jobs instead, specifically AWS. The market for OS and kernel development roles is actually shrinking a little. The OS is becoming a forgotten cog in a much larger cloud-based system. The job growth is in distributed systems, cloud SRE, data science, cloud network engineering, traffic and chaos engineering, container scheduling, and other new roles. 
  • @DrQz: The Performance Paradox: The better u do ur job, the more invisible u become. https://goo.gl/1aTRvw  ? ?
  • @kennwhite: $100,000+ spent Continue reading

Stuff The Internet Says On Scalability For September 1st, 2017

Hey, it's HighScalability time: 

Obviously, cloud native is simplicity itself. (Cloud Native Landscape Project)

• $10: price for IPv4 address (and falling); 10-15%: better IPv6 network performance; 711M: Record Onliner Spambot Dump; 41.4-tesla: strongest resistive magnet; 85%: cell towers offline due to Hurricane Harvey; 1M: Facebook accounts turned off every single day; 700K: Lyft drivers; $160B: Crypto Market Cap; $1.5M: bounty for iPhone jail break; 2.93M: pirated views of Mayweather-McGregor fight; 465k: people need to update pacemaker firmware; 70,065,920: views of Taylor Swift in 2 days; 2 trillion: kafka messages per day at LinkedIn; $35 billion: saved when planes fly themselves; 990: bird species in North America; 108B: number of people who have ever lived; 500M: DuckDuckGo anonymous searches in one month; 


• Quotable Quotes:
  • @asymco: The iPhone is not only the best selling product of all time but it may be the most used. 60 trillion minutes a year.
  • @Nick_Craver: Stack Overflow Questions (last 30 days): 1,280,911,699 Hits 26.92 ms Avg Render Time 11,040,996,940 SQL Queries Continue reading

Sponsored Post: Close.io, Loupe, Etleap, Aerospike, Stream, Scalyr, VividCortex, Domino Data Lab, MemSQL, InMemory.Net, Zohocorp

Who's Hiring? 

• Close.io is a ~25 person fully remote team that is profitable and building a product our customers love! We’re hiring Senior Backend Developers to join our team. Our backend tech stack currently includes Python (Flask, Gunicorn, TaskTiger), Elasticsearch, MongoDB, Postgres, and Redis running in Docker/Kubernetes on AWS. Learn more and apply here!


• Advertise your job here! 

Fun and Informative Events

• Advertise your event here!

Cool Products and Services

• .NET developers dealing with Errors in Production: You know the pain of troubleshooting errors with limited time, limited information, and limited tools. Managers want to know what’s wrong right away, users don’t want to provide log data, and you spend more time gathering information than you do fixing the problem. To fix all that, Loupe was built specifically as a .NET logging and monitoring solution. Loupe notifies you about any errors and tells you all the information you need to fix them. It tracks performance metrics, identifies which errors cause the greatest impact, and pinpoints the root causes. Learn more and try it free today.


• Grokking the System Design Interview is an interactive course that explains the Large Scale Distributed System Design problems in detail. System Continue reading

Stuff The Internet Says On Scalability For August 25th, 2017

Hey, it's HighScalability time: 

 

View of the total solar eclipse from a hill top near Madras Oregon, August 21, 2017. As totality approaches, dragons gorge on sun flesh; darkness cleaves the day; a chill chases away the heat; all becomes still. Contact made! Diamonds glitter; beads sparkle; shadow band snakes slither across pale dust; moon shadow races across the valley, devouring all in wonder. Inside a circle of standing stones, obsidian knives slash and stab. Sacrifices offered, dragons take flight. In awe we behold the returning of the light.

• ~5: ethereum transactions per second; 29+M: Snapchat news viewers; 100K: largest Mastodon instance; 2x: Alibaba's cloud base growth; 1B: trees planted by a province in Pakistan; 90.07%: automated decoding of honey bee waggle dances; $86.4B: Worldwide Information Security Spending; 1200: db migrations from Mysql to Postgres; $7B: Netflix content spend (most not original); 13%: increased productivity by making vacation mandatory; 75%: US teens use iPhones; 30,000x: energy use for Bitcoin transaction compared to Visa; ~1 trillion: observations processed for Gaia mission; 50% Continue reading

Stuff The Internet Says On Scalability For August 18th, 2017

Sorry about missing last week, but my birthday won out over working: 



 

Ouch! @john_overholt: My actual life is now a science exhibit about the primitive conditions of the past.

If you like this sort of Stuff then please support me on Patreon.

 

• 1PB: SSD in 1U chassis; 90%: savings using EC2 Spot for containers; 16: forms of inertia; $2.1B: Alibaba’s profit; 22.6B: app downloads in Q2; 25%: Google generated internet traffic; 20 by 20 micrometers: quantum random number generators; 16: lectures on Convolutional Neural Networks for Visual Recognition; 25,000: digitized gramophone records; 280%: increase in IoT attacks; 6.5%: world's GDP goes to subsidizing fossil fuel; 832 TB: ZFS on Linux;  $250,000: weekly take from breaking slot machines; 30: galatic message exchanges using artificial megastructures in 100,000 years; 


• Quotable Quotes:
  • @chris__martin: ALIENS: we bring you a gift of reliable computing technol--
    HUMANS: oh no we have that already but JS is easier to hire for
  • @rakyll: "You woman, you like intern." I interned on F-16's flight computer. Even my internship was 100x more legit than any job you will have.
  • @CodeWisdom Continue reading