Modern Web Serving in 2020 (Part 1)
Web application is fascinating. It stood the test of the dot-com bubble, the mobile revolution, and it is still here. Web application is not going anywhere, at least in the foreseeable future.
Just like how web technologies has iterated over the past decades, web serving has also developed quite a lot in order to cope with the ever-increasing scale, demand for security, reliability and performance. By web serving, what I really mean is the infrastructures which support the running of web applications on 1 or more servers.
In this blog, I try to summarize a few ways to serve a web application, from really simple one(one website, one server) to more sophisticated one(think 1 website, multiple data centers), and to explain why more sophisticated architectures are necessary for modern websites, what problems they are solving, etc.
Dummy web serving in localhost
Let’s start with the simple one. For those of you who have created your own little toy website with NodeJS/Django/etc, you’ve probably had some experience with serving web app locally.
To start serving your static assets in NodeJS, we’d use a simple line of code like this:
app.use(express.static(‘public’))
Once starting up http server, usually provided by frameworks like NodeJS/Django as a dev http server, static assets in the public folder is ready to be accessed, e.g
And the APIs of your web app could be accessed like this:
Assuming above is what you already know 🙂 Let’s discuss something you might not know
There are many problems with serving your website this way in production, in terms of many aspects
Performance
Serving static assets will be really slow. Trust me I’ve done it on GCP, it’s super slow..
Reasons could be:
- User is too far away from the server geographically
- Lack of web-acceleration features(data compression, caching, etc)
- Server is overloaded by requests
Security
IP address of your web server is directly exposed to everyone so it’s vulnerable to both DDOS attack and things like 0day bug exploitation.
Flexibility & Scalability
This simple architecture is not flexible/scalable, what if we want more servers to handle increasing traffic?
Availability
There ’s only 1 server, what if there’s a million user making requests to your server per minute. In this case, your server will definitely not be available to handle all requests due to limiting CPU resources, connection limit, memory, etc.
Single Point of Failure
If the server is down, the whole website cease to exists..User will be very sad if they can’t use your dating site 🙂
Therefore, we definitely need something better than this dummy example!
Web Serving with Nginx
With slightly more sophisticated websites than toy project which only lives in localhost, a better web server is needed.
Nginx is typically used in production environment to serve your web contents, be it static assets or backend API micro services.
Now you may be wondering, why do we need Nginx for production web serving?
Well, there are actually many use cases.
1 Use nginx as a Reverse Proxy
The definition of Reverse Proxy is
A reverse proxy accepts a request from a client, forwards it to a server that can fulfill it, and returns the server’s response to the client.
OK, so a reverse proxy is just a middle guy that sits between client and server, why would that be useful? Here are the reasons
1a) Increased security
No information about your real servers are exposed, so malicious clients cannot access them directly to exploit any vulnerabilities. Also nginx provides security features that protect servers from DDOS attacks by blacklisting malicious IP and limiting number of connections accepted from each client.
1b) Increased scalability and flexibility
Clients only knows the IP address of the Nginx server, so we’re free to take servers down for maintenance. For the same reason, number of servers can be scaled up and down while keeping the website running.
1c) Compression
Before returning server response to client, nginx can compress the response with gzip so that it can be transferred through the public network more quickly!
Don’t worry! The browser knows how to decompress the response because nginx will put a note in the http response:
Content-Encoding: gzip
1d) SSL termination
When https is used, the requests sent by users are encrypted. In this case Nginx, instead of your backend servers, are responsible for decrypting the https request, which is computationally intensive. This helps free up computation resources on your backend servers so that it can focus on serving! Similarly, when your backend server returns a response, Nginx is also responsible for encrypting it.
1e Caching
This one is simple, Nginx caches the response for some request and return the cached response upon same request.
2 Using nginx as Load Balancer
Definition for load balancer:
A load balancer distributes incoming client requests among a group of servers, in each case returning the response from the selected server to the appropriate client.
Load balancers are commonly needed when the amount of requests a site receives is just beyond what one server can handle.
In this case, multiple backend servers are deployed. Nginx acts as a load balancer that distribute load evenly among these servers, and also persist sessions between client and servers.
Now that we know nginx provides so many benefits, let’s see a how nginx is used!
In this picture, all users access the website by first sending a /GET request to the Nginx box, the box then forwards the /GET request to one of the web server it thinks is up and ready to handle request. The selected app server queries the database and return the response to the nginx box, which then gets returned to the user.
Actually, for serving static assets, an even easier setup(not described in the picture above) is to mount a remote folder with static assets onto the nginx box, and the nginx would simply fetch the static files there and return to the user.
This architecture solves many of the problems appeared in the first dummy web serving example.
With web acceleration and load balancing features, performance is greatly improved
All app servers are now hiding behind the nginx box, so they are more secure.
The architecture is also more scalable, as servers could be brought up and down as needed.
Your website will be much more available as we could scale it up to, say using 100 servers instead of just 1 server.
Serving static assets with Content Delivery Network(CDN)
Now there is one more performance problem that needs to be tackled. Imagine your dating web app is located in the US, and your user Goku is located in Japan. It takes way too long for him to load a girl’s profile image, and Goku sadly closes your website and swear never to use it again.
And.. CDN is here to save your dating site!
Definition of CDN
A content delivery network, or content distribution network, is a geographically distributed network of proxy servers and their data centers. The goal is to provide high availability and performance by distributing the service spatially relative to end users
With CDN, owner of a website could distribute its static assets around the world in servers deployed by cloud providers such as Amazon CloudFront or Cloudflare CDN.
Thus, ideally, user will be able to download the static assets from the closest server, significantly reducing latency.
With CDN, our architecture now looks like this
In the client side, user X always try to access the CDN(closest server) for a static file, if it’s not cached in CDN, then the CDN server will pull it from the Nginx box.
After the first time, everyone who is in user X’s region will get the static file from the cloud CDN because it’s now cached.
But, how does this all work?
To illustrate this with an example, let’s say your static assets are served at this url
To setup CDN, you will provide two pieces of info to a CDN provider(there are many other info a CDN provider may need, but these two are most important)
1 Domain name: “https://www.datingapp.com"
2 Folder: “public/img”
With these information, your CDN provider gives you a unique URL such as
Then, Modify all references of
to
in your webiste’s codebase
Now, when your user hits
The CDN server will look into its cache to see if it has a copy of user.png, if it does, it’ll simply return it. Otherwise it’ll pull the user.png from
and stores a copy of it in cache
One thing to note is that the domain name
cdn.datingapp.com
will always resolve to a closest server according to your location so that you’re always downloading the files as fast as possible.
How does it know how to resolve to a CDN server closest to your location? Well that’s another topic for DNS, for now you can just think of it as some blackbox magic.
Conclusion
With current setup, the website has nginx as load balancer in front of any number of servers in a cluster, and meanwhile using CDN as a way to distribute static contents, we can now proudly announce that our website is ready to serve a million users with great security, reliability and browsing experience. For those who are even more ambitious, this is not enough for serving hundreds of millions if not billions of users.
Why? At the end of the day, there’s only one cluster of servers sitting in one place, latency will still suffer if the site needs to handle huge amount of API requests which can’t be handled by CDN. In this case, multiple data centers and even a dozens of Points of Presence are strategically placed at different parts of the world to help get user requests quickly routed and responded! Interested to know how that works? See my next blog, coming up soon..
