High Scalability Archives - Page 4 of 30

Who's Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

P2Location is IP address geolocation service provider since 2002. The geolocation database or API detects location, proxy and other >20 parameters. The technology has been cited in more than 700 research papers and trusted by many Fortune 500 companies. Try it today!

ipdata is a reliable IP Address Geolocation API that allows you to lookup the approximate location of any IP Address, detect proxies and identify a company from an IP Address. Trusted by 10,000+ developers. Try it now!

Developers care about shipping secure applications. Application security products and processes, however, have not kept up with advances in software development. There are a new breed of tools hitting the market that enable developers to take the lead on AppSec. Learn how Continue reading

Moving HPC to the Cloud: A Guide for 2020

This is a guest post by Limor Maayan-Wainstein, a senior technical writer with 10 years of experience writing about cybersecurity, big data, cloud computing, web development, and more.

High performance computing (HPC) enables you to solve complex problems which cannot be solved by regular computing. Traditionally, HPC solutions provided mainly supercomputers. Today, HPC is typically a mix of resources, including supercomputing and virtualized and bare metal servers, platforms for management, sharing and integration capabilities, and more. When coupled with the cloud, HPC is made more affordable, accessible, efficient and shareable.

What Is HPC?

Who's Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

ipdata is a reliable IP Address Geolocation API that allows you to lookup the approximate location of any IP Address, detect proxies and identify a company from an IP Address. Trusted by 10,000+ developers. Try it now!

Developers care about shipping secure applications. Application security products and processes, however, have not kept up with advances in software development. There are a new breed of tools hitting the market that enable developers to take the lead on AppSec. Learn how engineering teams are using products like StackHawk and Snyk to add security bug testing to their CI pipelines.

Learn the stuff they don't teach you in the AWS docs. Filter out the distracting hype, and focus on the parts of AWS that Continue reading

MySQL on Azure Performance Benchmark – ScaleGrid vs. Azure Database

Microsoft Azure is one of the most popular cloud providers in the world, and a natural fit for database hosting on applications leveraging Microsoft across their infrastructure. MySQL is the number one open source database that’s commonly hosted through Azure instances. While Microsoft offers their own Azure Database product, there are other alternatives available that may be able to help you improve your MySQL performance. In this blog post, we compare Azure Database for MySQL vs. ScaleGrid MySQL on Azure so you can see which provider offers the best throughput and latency performance. We measure latency in ms 95th percentile latency.

Who's Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

ipdata is a reliable IP Address Geolocation API that allows you to lookup the approximate location of any IP Address, detect proxies and identify a company from an IP Address. Trusted by 10,000+ developers. Try it now!

Developers care about shipping secure applications. Application security products and processes, however, have not kept up with advances in software development. There are a new breed of tools hitting the market that enable developers to take the lead on AppSec. Learn how engineering teams are using products like StackHawk and Snyk to add security bug testing to their CI pipelines.

Learn the stuff they don't teach you in the AWS docs. Filter out the distracting hype, and focus on the parts of AWS that Continue reading

Stuff The Internet Says On Scalability For July 31st, 2020

Hey, it's HighScalability time!

Serverless is really complex. Or is it? @paulbiggar sparked a thoughtful Twitter thread.

Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen.

Usually I pitch my book Explain the Cloud Like I'm 10 here, but if you're an author with books on Amazon I'd like to ask you to give Best Sellers Rank a try. I started it to send data to me about my own books, but in the best bootstrapping tradition I extended it to work for the entire planet.

Number Stuff:

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Who's Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

Developers care about shipping secure applications. Application security products and processes, however, have not kept up with advances in software development. There are a new breed of tools hitting the market that enable developers to take the lead on AppSec. Learn how engineering teams are using products like StackHawk and Snyk to add security bug testing to their CI pipelines.

Learn the stuff they don't teach you in the AWS docs. Filter out the distracting hype, and focus on the parts of AWS that you'd be foolish not to use. Learn the Good Parts of AWS. Created by former senior-level AWS engineers of 15 years.

Stateful JavaScript Apps. Effortlessly add state to your Javascript apps with FaunaDB. Generous free tier. Try Continue reading

Stuff The Internet Says On Scalability For June 26th, 2020

Hey, it's HighScalability time!

Line noise? Perl? Still uncertain? It's how you program a quantum computer. Silq.

Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the smart and thoughtful world.

Know someone who could benefit from understanding the cloud? Of course you do. I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 115 mostly 5 star reviews. Here's a 100% gluten-free review:

Number Stuff:

How TripleLift Built an Adtech Data Pipeline Processing Billions of Events Per Day

This is a guest post by Eunice Do, Data Engineer at TripleLift, a technology company leading the next generation of programmatic advertising.

What is the name of your system and where can we find out more about it?

The system is the data pipeline at TripleLift. TripleLift is an adtech company, and like most companies in this industry, we deal with high volumes of data on a daily basis. Over the last 3 years, the TripleLift data pipeline scaled from processing millions of events per day to processing billions. This processing can be summed up as the continuous aggregation and delivery of reporting data to users in a cost efficient manner. In this article, we'll mostly be focusing on the current state of this multi-billion event pipeline.

To follow the 5 year journey leading up to the current state, check out this talk on the evolution of the TripleLift pipeline by our VP of Engineering.

Why did you decide to build this system?

Who’s Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

Developers care about shipping secure applications. Application security products and processes, however, have not kept up with advances in software development. There are a new breed of tools hitting the market that enable developers to take the lead on AppSec. Learn how engineering teams are using products like StackHawk and Snyk to add security bug testing to their CI pipelines.

Learn the stuff they don’t teach you in the AWS docs. Filter out the distracting hype, and focus on the parts of AWS that you’d be foolish not to use. Learn the Good Parts of AWS. Created by former senior-level AWS engineers of 15 years.

Stateful JavaScript Apps. Effortlessly add state to your Javascript apps with FaunaDB. Generous free tier. Try Continue reading

Who's Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems: large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. - this is going to be a challenging journey for any backend engineer! Please apply here.

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

Learn the stuff they don't teach you in the AWS docs. Filter out the distracting hype, and focus on the parts Continue reading

A Short on How Zoom Works

Zoom scaled from 20 million to 300 million users virtually over night. What's incredible is from the outside they've shown little in the way of apparent growing pains, though on the inside it's a good bet a lot of craziness is going on.

Sure, Zoom has made some design decisions that made sense as a small spunky startup that don't make a lot of sense as a defacto standard, but that's to be expected. It's not a sign of bad architecture as many have suggested. It's just realistically how products evolve, especially when they must uplift over weeks, days, and even hours.

Sudden success invites scrutiny, so everyone wants to know how Zoom works. The problem is we don't know much, but we do have a few information sources:

Stuff The Internet Says On Scalability For May 14th, 2020

Hey, it's HighScalability time!

LOL. Who knew a birthday service could lead to an existential crisis?

Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the smart and thoughtful world.

Know someone who wants to understand the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 111 mostly 5 star reviews. Here's a recent 100% organic non-GMO review:

Number Stuff:

Who's Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems: large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. - this is going to be a challenging journey for any backend engineer! Please apply here.

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

Learn the stuff they don't teach you in the AWS docs. Filter out the distracting hype, and focus on the parts Continue reading

Reducing Your Database Hosting Costs: DigitalOcean vs. AWS vs. Azure

If you’re hosting your databases in the cloud, choosing the right cloud service provider is a significant decision to make for your long-term hosting costs. This is especially apparent in today's world where organizations are doing whatever they can to optimize and reduce their costs. Over the last few weeks, we have been inundated with requests from SMB customers looking to improve the ROI on their database hosting. In this article, we are going to compare three of the most popular cloud providers, AWS vs. Azure vs. DigitalOcean for their database hosting costs for MongoDB® database to help you decide which cloud is best for your business.

Comparing Cloud Instance Costs

Part 2: The DynamoDB Book: An Interview With Alex DeBrie On His New Book

This is the second part of my interview with Alex DeBrie on his new insta-classic: The DynamoDB Book.

To read the first part of the interview please mosey on over to The DynamoDB Book: An Interview With Alex DeBrie On His New Book. Go ahead. Take your time. It's worth it.

Quick Fire Round. What's your quick reaction to:

Post: InterviewCamp.io, Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

Who's Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems: large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. - this is going to be a challenging journey for any backend engineer! Please apply here.

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

Learn to balance architecture trade-offs and design scalable enterprise-level software. Check out Educative.io's bestselling new 4-course learning track: Scalability and Continue reading

The DynamoDB Book: An Interview With Alex DeBrie On His New Book

You know nothing about DynamoDB. At least that’s what I realized the first time I heard Rick Houlihan give his now infamous talk at AWS re:Invent 2018 on Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB.

In that talk Rick revealed for the first time the inner arcana of single-table design. Minds were blown. Weaknesses were revealed. Futures were changed.

As a mere novice in the ways of DynamoDB I realized there were many levels of understanding needed before one could become a true AWS Data Hero. For that we need a guide.

Our guide on the Hero’s Journey that is mastering DynamoDB is a wise young wizard named Alex DeBrie. Alex wrote what you might consider to be the Gnostic Gospels of DynamoDB: The DynamoDB Book.

You will know something after reading this book

But it's more than just a book. You can’t buy it on Amazon. Instead, Alex uses Gumroad to offer packages at three different price points along with a team option. Each level provides additional content:

Basic package ($79). A 450 page book and six cheatsheets.
Plus package ($129). 60-page Analytics supplement guide. 60-page Operations supplement guide. Five deployable code implementations.
Premium Continue reading

HubSpot Migrating ZooKeeper into Kubernetes

We recently migrated hundreds of ZooKeeper instances from individual server instances to Kubernetes without downtime. Our approach used powerful Kubernetes features like endpoints to ease the process, so we’re sharing the high level outline of the approach for anyone who wants to follow in our footsteps. See the end for important networking prerequisites.

Click to read more

Post: InterviewCamp.io, Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

Who's Hiring?

InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems: large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. - this is going to be a challenging journey for any backend engineer! Please apply here.

Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

Need excellent people? Advertise your job here!

Cool Products and Services

Learn to balance architecture trade-offs and design scalable enterprise-level software. Check out Educative.io's bestselling new 4-course learning track: Scalability and Continue reading

« Previous 1 2 3 4 5 6 … 30 Next »