Author Archives: Todd Hoff
Author Archives: Todd Hoff
Hi everyone,
First, I’d like to sincerely thank everyone who has supported Highscalability over the years.
We’ve grown together through some interesting times. Massive changes have occurred in how systems are built, and my goal has always been to help people learn how to build them better. I’d like to think that goal was accomplished.
Obviously, for that last several years, things have been quiet around here. This has always been a one programmer show, and for everything, there is a season. The season for me is changing. That means http://highscalability.com is for sale.
My preference is for someone to take HS over and keep it going in some form or another. If you are a content farm, please don’t reply. I’ve never once taken payment for a post and I don’t plan on changing that now.
You get the site, the domain, and the twitter account. If you are not a scam artist and have a reasonable offer, please contact me at [email protected].
Thanks!
DALL-E picture generated from the answer.
I was curious, given all the ChatGPT love, what it would make of some of our favorite topics. I was both impressed and unimpressed.
Impressed. ChatGPT instantly generated an response that might make a good answer in a Miss Universe contest. You know, the kind of answer delivered with a radiant smile calculated to offend no one yet still sounds comprehensive and deep.
Unimpressed. The answers are a mile wide and an inch deep. There's not a lot of insight. It's more of a zeitgeist oracle. Nothing wrong with that, but I don't think we'll be unemployed soon. It's as many have predicted about AI/ML. These are tools to augment us, flawed humans, not replace us...yet.
You're welcome! I'm happy to answer any questions you have. Please go ahead and ask your first question.
Cloud computing is a model of computing that delivers computing services over the internet, including storage, data processing, and networking. It allows users to access and use shared computing resources, such as servers, storage, and applications, on demand and without the need Continue reading
Never fear, HighScalability is here!
1958: An engineer wiring an early IBM computer 2021: An engineer wiring an early IBM quantum computer. @enclanglement
There's a move to regulate cloud providers by vertically separating the services they offer.
Like railroads of yore, who were not allowed to provide freight services on top of their base services, cloud providers would not be allowed to provide services on top of their base platform services.
Vertical separation would be new to the cloud industry. Is it a good idea? Would it actually solve any problems? My answers are no and no, but probably not for the reasons you think. Let's dive in.
Here are a few useful resources for exploring this argument:
Now Cory is about 100x times smarter than I am, but this analogy is wrong:
That's why the trustbusters were so big on "structural separation": the principle that a business can own a platform or use the platform, but not both.
So this remedy is also wrong:
Rather than trying to construct rules that kept the referee honest even when they played on one of the teams, they Continue reading
I've been working hard on a completely updated version of Explain the Cloud Like I'm 10 and I'm happy to announce it's ready!
You can buy version 2.0 at: https://www.amazon.com/Explain-Cloud-Like-Im-10-ebook/dp/B0765C4SNR.
If you've already bought, Explain the Cloud Like I'm 10, then you should just be able to update to the new version for free.
What's new? This version has about 2x the material. I go in-depth on:
In version 1 of the book, I didn't cover cloud providers in any detail. Almost all the new content is platform related, so that's fixed. And I say platform rather than cloud provider on purpose.
The biggest change in the book is it is now oriented around the "cloud model" as the ultimate goal for software development, deployment, and use, not just the Continue reading
Never fear, HighScalability is here!
Every cell a universe. Most detailed image of a human cell to date. @microscopicture
Other images considered: one byte of RAM in 1946; visual guide on troubleshooting Kubernetes; Cloudflare using lava lamps to generate cryptographic keys; 5MB of data looked like in 1966
Never fear, HighScalability is here!
This is a guest post by Ankit Sirmorya. Ankit is working as a Machine Learning Lead/Sr. Machine Learning Engineer at Amazon and has led several machine-learning initiatives across the Amazon ecosystem. Ankit has been working on applying machine learning to solve ambiguous business problems and improve customer experience. For instance, he created a platform for experimenting with different hypotheses on Amazon product pages using reinforcement learning techniques. Currently, he is in the Alexa Shopping organization where he is developing machine-learning-based solutions to send personalized reorder hints to customers for improving their experience.
This is a guest post by Ankit Sirmorya. Ankit is working as a Machine Learning Lead/Sr. Machine Learning Engineer at Amazon and has led several machine-learning initiatives across the Amazon ecosystem. Ankit has been working on applying machine learning to solve ambiguous business problems and improve customer experience. For instance, he created a platform for experimenting with different hypotheses on Amazon product pages using reinforcement learning techniques. Currently, he is in the Alexa Shopping organization where he is developing machine-learning-based solutions to send personalized reorder hints to customers for improving their experience.
Design a location-based social search application similar to Tinder which if often used as a dating service. It allows users to use a swiping motion to like (swipe right) or dislike (swipe left) other users, and allows users to chat if both parties like each other(a “match”).
This is a guest post by Ankit Sirmorya. Ankit is working as a Machine Learning Lead/Sr. Machine Learning Engineer at Amazon and has led several machine-learning initiatives across the Amazon ecosystem. Ankit has been working on applying machine learning to solve ambiguous business problems and improve customer experience. For instance, he created a platform for experimenting with different hypotheses on Amazon product pages using reinforcement learning techniques. Currently, he is in the Alexa Shopping organization where he is developing machine-learning-based solutions to send personalized reorder hints to customers for improving their experience.
Design a photo-sharing platform similar to Instagram where users can upload their photos and share it with their followers. Subsequently, the users will be able to view personalized feeds containing posts from all the other users that they follow.
The application should be able to support the following requirements.
This is a guest post by Ankit Sirmorya. Ankit is working as a Machine Learning Lead/Sr. Machine Learning Engineer at Amazon and has led several machine-learning initiatives across the Amazon ecosystem. Ankit has been working on applying machine learning to solve ambiguous business problems and improve customer experience. For instance, he created a platform for experimenting with different hypotheses on Amazon product pages using reinforcement learning techniques. Currently, he is in the Alexa Shopping organization where he is developing machine-learning-based solutions to send personalized reorder hints to customers for improving their experience.
Design an instant messenger platform such as WhatsApp or Signal which users can utilize tosend messages to each other. An essential aspect of the application is that the chat messageswon’t be permanently stored in the application.
FUN FACT: Some of the chat messengers such as FB Messenger stores the chat messages unless the users explicitly delete it. However, instant messengers such as WhatsApp don’t save the messages permanently on their server.
This is a guest post by Ankit Sirmorya. Ankit is working as a Machine Learning Lead/Sr. Machine Learning Engineer at Amazon and has led several machine-learning initiatives across the Amazon ecosystem. Ankit has been working on applying machine learning to solve ambiguous business problems and improve customer experience. For instance, he created a platform for experimenting with different hypotheses on Amazon product pages using reinforcement learning techniques. Currently, he is in the Alexa Shopping organization where he is developing machine-learning-based solutions to send personalized reorder hints to customers for improving their experience.
Design a video streaming platform similar to Netflix where content creators can upload their video content and viewers are able to play video on different devices. We should also be able to store user statistics of the videos such as number of views, video watched duration, and so forth.
Today I have the pleasure of announcing my new app—Max reHIT Workout—on Product Hunt. Max reHIT Workout is an exercise app that guides you through interval workouts.
I won’t pitch the app here. I'll just say I’m proud of how it turned out and if you want an optimal algorithm for exercising, you might like it.
I know I haven’t been writing much lately. That's because there’s been very little evolution in software system architecture. It’s pretty much same thing, different day. In many ways that’s good, but it’s not interesting to write about.
This article, while definitely self serving, targets the choice of using a native iOS environment versus a cloud environment for an app. It’s a choice every developer must make. How do you make that choice? What are the implications? What choice would I make next time?
What would a totally new search engine architecture look like? Who better than Julien Lemoine, Co-founder & CTO of Algolia, to describe what the future of search will look like. This is the second article in a series. Here's Part 1.
Search engines need to support fast scaling for both Read and Write operations. Rapid scaling is essential in most use cases. For example, adding a vendor in a marketplace generates a spike of indexing operations (Write), and a marketing campaign generates a spike of queries (Read). In most use cases, both Read and Write operations scale but not at the exact same moment. The architecture needs to handle efficiently all these situations as the scaling of Read and Write operations varies over time in most use cases.
Until now, search engines were scaling with Read and Write operations colocated on the same VMs. This scaling method brings drawbacks, such asWrite operations unnecessarily hurting the Read performance and using a significant amount of duplicated CPU at indexing. This article explains those drawbacks and introduces a new way to scale more quickly and efficiently by splitting Read and Write operations.