A few days ago I wrote a Twitter thread about the global Fastly outage that was taking place at the moment. Following my trend of recycling content for this blog, I’ll repost it here.
Right now there is a global outage that is affecting some of the most used websites, making them partially or completely unavailable. The reason is a failure in a company that you may have never heard of called Fastly. This is a story about modern web centralization.
In 1992, Tim Berners-Lee defined the HTTP protocol while working at CERN, giving birth to the Web. In practical terms, this meant that users could retrieve information from the internet with a web browser, instead of having to use client side programs tailored to each server. As we know today, the world was craving for huge amounts of cute cat pictures (and porn), so the number of websites steadily increased for the next decade. At that time things were simple. A web page was backed up by a web server hosted in a computer in someone’s home/office. When you connected to the site, your computer sent a request to the site owner’s computer, which answered with the page contents. Easy.
However, this didn’t scale well with the exponential growth of web adoption. There is only so much that an individual computer can handle, after which point users begin to receive the dreaded “Connection timed out” errors. And there is a much bigger problem derived from having lots of users: your internet bill goes crazy. The same way you have to pay to your internet provider for usage, the sites you visit do too. The bandwidth cost of a content-heavy site can bankrupt you quite easily.
That is how Content Delivery Networks (CDN) appeared. These are huge companies that deliver your static content (everything that doesn’t need to be personalized) to the user on your behalf. They make use of economies of scale to offer a lower bill than if you did it yourself. Also, they have servers located all around the world, so the content is sent to the user from the closest location, reducing the latency. It’s a win-win-win situation: website wins, because cheap; user wins because fast; CDN wins because huge amounts of money.
The problem is that there are not so many reliable CDN providers, because the entry barrier to the market is quite high. It is just too difficult to build the needed infrastructure and negotiate worldwide cheap fees with internet providers. We have a beautiful oligopoly market. Fastly is one of those big players, along with Amazon, Google, Microsoft, Cloudflare, Akamai… Don’t get me wrong, they are amazing at what they do. Huge respect for keeping the web blazing fast. But they have become centralized points of failure for the modern internet. The issue with complex systems is that you can’t achieve 100% uptime. So from time to time, even though these hardworking teams of professionals do their best, a big CDN provider is going to suffer an outage. This will affect a big percentage of the webpages we use in a daily, as we have just witnessed.
Our (my) job as technology professionals is to design systems in a way that these incidents don’t become life endangering situations for people (e.g., online emergency services of health organizations). It is a fascinating job, but also a huge responsibility.
Oh, and this situation also applies to non-static web content (when you need a server personalizing the response for the user). The only difference is that the group of companies is slightly different and Amazon is a terrifying monopoly.
Welcome to cloud!