Summary
A web app that covers the most notable architectural works in the
city of Zagreb.
With more than 1000 detailed descriptions of buildings and their
architects, most of them from the 19th and early 20th centuries, the
app serves as an online catalog of the early architectural
development of the city of Zagreb.
Challenge
The biggest challenge on this project was architecting it for high availability while keeping the cost of running the app in the Cloud low. The website is non-profit and does not generate any revenue. It uses a wide set of Cloud Services, from MySQL database, file storage, virtual machines to Redis cache. The operational costs of running it have to be minimized without sacrificing performance.
Outcome
The website can serve at least 500.000 monthly visitors, most of the time free of charge or less than $1 at max. Since the website targets Croatian (around 4M population) users, it never exceeds that cost.
Before we dig into the details let's explain the complete app stack. On the frontend, a Vue.js app is talking to a Python Django Backend. Database duties are handled by MySQL. Redis memory cache is used to cache HTTP responses, database queries, and Google Maps API calls. Cloud file storage acts as CDN. Everything except the Redis cache runs on the Google Cloud Platform.
Now how is that possible, to run all these services almost free of charge? Just running the cheapest Google Cloud SQL database instance would incur at least a $20 monthly bill. $20 would be incurred for the cheapest Redis instance running on the Google Cloud Memorystore. So, only these two services would be around $40 monthly.
Well, we do not use any of them.
To provision the database we use the Google Compute Engine virtual
machine. This means setting up everything manually (installing the
database, setting up networking and VPC). Also, backups have to be
done manually. Since there is no automatic backup that the Cloud SQL
offers. But since the database is not that big, we can run it on the
free Compute Engine e2-micro instance. And data it holds is fairly
static, so we can use the Redis cache to cache the queries for
longer periods and reduce the database load significantly. Also,
since the data is static we can do backups only a couple of times
per month.
Redis handles the caching. We run it as a serverless cache on
Upstash. In the serverless model you pay for what you use, and Upstash
offers free 10k commands per day. Once you exceed it, it is $0.2 per
100k commands.
The Django backend runs on the Google App Engine standard
environment. We can get away with the smallest F1 instance because
it only serves HTTP requests. Automatic scaling is configured, we
start with 1 instance and can scale up to more if needed. Google App
Engine offers daily 28h of F instance runtime for free. With that
setup and normal load, we rarely exceed the free quota.
Google Cloud Storage acts as CDN for the static files on the
frontend. Besides JavaScript files and CSS, this includes all the
photos of buildings. Again thanks to the free tier from Google
Cloud, we get 5GB of storage for free.
Thanks to caching, most of the requests have a cache hit and end up
with fast response times. This allows the backend to serve more
requests without Google App Engine entering auto-scaling mode and
starting more instances. Almost all the traffic is handled by a
single App Engine F1 instance. Which runs in the free tier.
By leveraging the free tier from Google Cloud, and setting up some of the services manually, medium traffic websites can utilize a lot of Cloud Services and run almost for free.