Services
As I already mentioned in the Introduction, this is not a simple static website or WordPress page. Instead, I am using multiple microservices and event-driven mechanisms.
This post and all the other Services posts are rather technical, so if you were more interested in the design of
individual pages, have a look at the Design posts.
If you want to learn about all the moving parts that make this website run however - grab yourself a coffee, get comfy &
enjoy reading! :)
Let's start with a simple list of all the services that work together in harmony! Many of which I built myself and others that run in docker containers alongside them.
This section gives a brief description, and you can find more details on each of the services in their designated posts.
Frontend
The actual website that you are currently interacting with to read this post. It is built with the Svelte Kit framework, Svelte 5, Tailwind CSS and written in TypeScript.
Most of the data like texts, images, files & more are saved in the database, cache and Static File Server. All requests go through the API Gateway.
API Gateway
I am using Kong's API Gateway to orchestrate requests from the frontend to the backend
services.
It can manage traffic with rate limiting, handle authentication, and also supports a bunch of plugins from
Kong and the community! It is built on top of NGINX, utilizing its reverse proxy capabilities.
Cache API
This is the main API for when data is requested from the frontend! It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route GET-Requests to the Cache API first because the cache stores entire data objects for faster lookups compared to the database!
It can also request data from the database via the Database API in case of a Cache-Miss (more details in Application Flow).
Cache
I am using a Redis cache to store already built data like JSON objects, Strings, Sets and more. It is a "dumb" storage that doesn't handle any calculations or joins.
Database API
This is the main API for when data is created, updated and deleted. It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route POST-, PUT-, PATCH-, and DELETE-Requests to the Database API first because the database is the single source of truth - so any data modification should happen here first!
A message is published via the Message Queue in most cases after data has been modified to let the Cache-Sync-Service invalidate connected cache keys (more details in Application Flow).
It can also fetch from the database to build objects that are saved in the cache - this usually happens in case of a Cache-Miss.
Database
I am using a Postgres database which is the single source of truth for all the data. It made most sense to use a relational database for the website because a lot of the data is actually connected to one another.
Postgres also allows the use of functions on a database-level & more to tailor the saved data to my exact expectations - so even if unwanted data were to come through the frontend and APIs, the database would check it a third time before saving.
Cache-Sync-Service
A small microservice written in Rust. It subscribes to the Message Queue and looks for cache invalidation events. Each message holds a list of values that indicate which cache keys should be invalidated.
It connects to the Cache API whenever it receives a message and sends an invalidation request for all given keys.
Those messages are usually sent by the Database API after modifying the database.
Message Queue
I chose RabbitMQ for the Message Queue to handle events like cache invalidation. It includes a dead letter exchange out of the box, which improves data consistency and helps with error-handling.
Everytime a message is sent, there has to happen an acknowledgement before the message is finalized or else it is sent to the dead letter exchange. This way messages that couldn't process successfully aren't just dropped and can be retried instead.
Protobuf
I decided to use Protobuf as part my data serialization for DTOs (Data Transfer Objects).
Reason being that multiple services are using the same DTOs.
Let's take this Project Post for example:
- It is received in the frontend from the Cache API,
- saved in the cache as JSON object,
- returned by the Database API in case of a Cache-Miss.
So this means that the Database API, Cache API and frontend all have to know about the same DTOs. Thankfully I can automatically generate Rust structs and TypeScript interfaces for this, so that they are all in-sync!
Static File Server
NGINX is often used as a reverse proxy, but it can be configured as a file server too. So files like Images and PDF documents that are saved on my server can be safely accessed through the Static File Server from their specified directory.
As I already mentioned in the Introduction, this is not a simple static website or WordPress page. Instead, I am using multiple microservices and event-driven mechanisms.
This post and all the other Services posts are rather technical, so if you were more interested in the design of
individual pages, have a look at the Design posts.
If you want to learn about all the moving parts that make this website run however - grab yourself a coffee, get comfy &
enjoy reading! :)
Let's start with a simple list of all the services that work together in harmony! Many of which I built myself and others that run in docker containers alongside them.
This section gives a brief description, and you can find more details on each of the services in their designated posts.
Frontend
The actual website that you are currently interacting with to read this post. It is built with the Svelte Kit framework, Svelte 5, Tailwind CSS and written in TypeScript.
Most of the data like texts, images, files & more are saved in the database, cache and Static File Server. All requests go through the API Gateway.
API Gateway
I am using Kong's API Gateway to orchestrate requests from the frontend to the backend
services.
It can manage traffic with rate limiting, handle authentication, and also supports a bunch of plugins from
Kong and the community! It is built on top of NGINX, utilizing its reverse proxy capabilities.
Cache API
This is the main API for when data is requested from the frontend! It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route GET-Requests to the Cache API first because the cache stores entire data objects for faster lookups compared to the database!
It can also request data from the database via the Database API in case of a Cache-Miss (more details in Application Flow).
Cache
I am using a Redis cache to store already built data like JSON objects, Strings, Sets and more. It is a "dumb" storage that doesn't handle any calculations or joins.
Database API
This is the main API for when data is created, updated and deleted. It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route POST-, PUT-, PATCH-, and DELETE-Requests to the Database API first because the database is the single source of truth - so any data modification should happen here first!
A message is published via the Message Queue in most cases after data has been modified to let the Cache-Sync-Service invalidate connected cache keys (more details in Application Flow).
It can also fetch from the database to build objects that are saved in the cache - this usually happens in case of a Cache-Miss.
Database
I am using a Postgres database which is the single source of truth for all the data. It made most sense to use a relational database for the website because a lot of the data is actually connected to one another.
Postgres also allows the use of functions on a database-level & more to tailor the saved data to my exact expectations - so even if unwanted data were to come through the frontend and APIs, the database would check it a third time before saving.
Cache-Sync-Service
A small microservice written in Rust. It subscribes to the Message Queue and looks for cache invalidation events. Each message holds a list of values that indicate which cache keys should be invalidated.
It connects to the Cache API whenever it receives a message and sends an invalidation request for all given keys.
Those messages are usually sent by the Database API after modifying the database.
Message Queue
I chose RabbitMQ for the Message Queue to handle events like cache invalidation. It includes a dead letter exchange out of the box, which improves data consistency and helps with error-handling.
Everytime a message is sent, there has to happen an acknowledgement before the message is finalized or else it is sent to the dead letter exchange. This way messages that couldn't process successfully aren't just dropped and can be retried instead.
Protobuf
I decided to use Protobuf as part my data serialization for DTOs (Data Transfer Objects).
Reason being that multiple services are using the same DTOs.
Let's take this Project Post for example:
- It is received in the frontend from the Cache API,
- saved in the cache as JSON object,
- returned by the Database API in case of a Cache-Miss.
So this means that the Database API, Cache API and frontend all have to know about the same DTOs. Thankfully I can automatically generate Rust structs and TypeScript interfaces for this, so that they are all in-sync!
Static File Server
NGINX is often used as a reverse proxy, but it can be configured as a file server too. So files like Images and PDF documents that are saved on my server can be safely accessed through the Static File Server from their specified directory.
Services
As I already mentioned in the Introduction, this is not a simple static website or WordPress page. Instead, I am using multiple microservices and event-driven mechanisms.
This post and all the other Services posts are rather technical, so if you were more interested in the design of
individual pages, have a look at the Design posts.
If you want to learn about all the moving parts that make this website run however - grab yourself a coffee, get comfy &
enjoy reading! :)
Let's start with a simple list of all the services that work together in harmony! Many of which I built myself and others that run in docker containers alongside them.
This section gives a brief description, and you can find more details on each of the services in their designated posts.
Frontend
The actual website that you are currently interacting with to read this post. It is built with the Svelte Kit framework, Svelte 5, Tailwind CSS and written in TypeScript.
Most of the data like texts, images, files & more are saved in the database, cache and Static File Server. All requests go through the API Gateway.
API Gateway
I am using Kong's API Gateway to orchestrate requests from the frontend to the backend
services.
It can manage traffic with rate limiting, handle authentication, and also supports a bunch of plugins from
Kong and the community! It is built on top of NGINX, utilizing its reverse proxy capabilities.
Cache API
This is the main API for when data is requested from the frontend! It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route GET-Requests to the Cache API first because the cache stores entire data objects for faster lookups compared to the database!
It can also request data from the database via the Database API in case of a Cache-Miss (more details in Application Flow).
Cache
I am using a Redis cache to store already built data like JSON objects, Strings, Sets and more. It is a "dumb" storage that doesn't handle any calculations or joins.
Database API
This is the main API for when data is created, updated and deleted. It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route POST-, PUT-, PATCH-, and DELETE-Requests to the Database API first because the database is the single source of truth - so any data modification should happen here first!
A message is published via the Message Queue in most cases after data has been modified to let the Cache-Sync-Service invalidate connected cache keys (more details in Application Flow).
It can also fetch from the database to build objects that are saved in the cache - this usually happens in case of a Cache-Miss.
Database
I am using a Postgres database which is the single source of truth for all the data. It made most sense to use a relational database for the website because a lot of the data is actually connected to one another.
Postgres also allows the use of functions on a database-level & more to tailor the saved data to my exact expectations - so even if unwanted data were to come through the frontend and APIs, the database would check it a third time before saving.
Cache-Sync-Service
A small microservice written in Rust. It subscribes to the Message Queue and looks for cache invalidation events. Each message holds a list of values that indicate which cache keys should be invalidated.
It connects to the Cache API whenever it receives a message and sends an invalidation request for all given keys.
Those messages are usually sent by the Database API after modifying the database.
Message Queue
I chose RabbitMQ for the Message Queue to handle events like cache invalidation. It includes a dead letter exchange out of the box, which improves data consistency and helps with error-handling.
Everytime a message is sent, there has to happen an acknowledgement before the message is finalized or else it is sent to the dead letter exchange. This way messages that couldn't process successfully aren't just dropped and can be retried instead.
Protobuf
I decided to use Protobuf as part my data serialization for DTOs (Data Transfer Objects).
Reason being that multiple services are using the same DTOs.
Let's take this Project Post for example:
- It is received in the frontend from the Cache API,
- saved in the cache as JSON object,
- returned by the Database API in case of a Cache-Miss.
So this means that the Database API, Cache API and frontend all have to know about the same DTOs. Thankfully I can automatically generate Rust structs and TypeScript interfaces for this, so that they are all in-sync!
Static File Server
NGINX is often used as a reverse proxy, but it can be configured as a file server too. So files like Images and PDF documents that are saved on my server can be safely accessed through the Static File Server from their specified directory.
As I already mentioned in the Introduction, this is not a simple static website or WordPress page. Instead, I am using multiple microservices and event-driven mechanisms.
This post and all the other Services posts are rather technical, so if you were more interested in the design of
individual pages, have a look at the Design posts.
If you want to learn about all the moving parts that make this website run however - grab yourself a coffee, get comfy &
enjoy reading! :)
Let's start with a simple list of all the services that work together in harmony! Many of which I built myself and others that run in docker containers alongside them.
This section gives a brief description, and you can find more details on each of the services in their designated posts.
Frontend
The actual website that you are currently interacting with to read this post. It is built with the Svelte Kit framework, Svelte 5, Tailwind CSS and written in TypeScript.
Most of the data like texts, images, files & more are saved in the database, cache and Static File Server. All requests go through the API Gateway.
API Gateway
I am using Kong's API Gateway to orchestrate requests from the frontend to the backend
services.
It can manage traffic with rate limiting, handle authentication, and also supports a bunch of plugins from
Kong and the community! It is built on top of NGINX, utilizing its reverse proxy capabilities.
Cache API
This is the main API for when data is requested from the frontend! It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route GET-Requests to the Cache API first because the cache stores entire data objects for faster lookups compared to the database!
It can also request data from the database via the Database API in case of a Cache-Miss (more details in Application Flow).
Cache
I am using a Redis cache to store already built data like JSON objects, Strings, Sets and more. It is a "dumb" storage that doesn't handle any calculations or joins.
Database API
This is the main API for when data is created, updated and deleted. It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route POST-, PUT-, PATCH-, and DELETE-Requests to the Database API first because the database is the single source of truth - so any data modification should happen here first!
A message is published via the Message Queue in most cases after data has been modified to let the Cache-Sync-Service invalidate connected cache keys (more details in Application Flow).
It can also fetch from the database to build objects that are saved in the cache - this usually happens in case of a Cache-Miss.
Database
I am using a Postgres database which is the single source of truth for all the data. It made most sense to use a relational database for the website because a lot of the data is actually connected to one another.
Postgres also allows the use of functions on a database-level & more to tailor the saved data to my exact expectations - so even if unwanted data were to come through the frontend and APIs, the database would check it a third time before saving.
Cache-Sync-Service
A small microservice written in Rust. It subscribes to the Message Queue and looks for cache invalidation events. Each message holds a list of values that indicate which cache keys should be invalidated.
It connects to the Cache API whenever it receives a message and sends an invalidation request for all given keys.
Those messages are usually sent by the Database API after modifying the database.
Message Queue
I chose RabbitMQ for the Message Queue to handle events like cache invalidation. It includes a dead letter exchange out of the box, which improves data consistency and helps with error-handling.
Everytime a message is sent, there has to happen an acknowledgement before the message is finalized or else it is sent to the dead letter exchange. This way messages that couldn't process successfully aren't just dropped and can be retried instead.
Protobuf
I decided to use Protobuf as part my data serialization for DTOs (Data Transfer Objects).
Reason being that multiple services are using the same DTOs.
Let's take this Project Post for example:
- It is received in the frontend from the Cache API,
- saved in the cache as JSON object,
- returned by the Database API in case of a Cache-Miss.
So this means that the Database API, Cache API and frontend all have to know about the same DTOs. Thankfully I can automatically generate Rust structs and TypeScript interfaces for this, so that they are all in-sync!
Static File Server
NGINX is often used as a reverse proxy, but it can be configured as a file server too. So files like Images and PDF documents that are saved on my server can be safely accessed through the Static File Server from their specified directory.
Services
As I already mentioned in the Introduction, this is not a simple static website or WordPress page. Instead, I am using multiple microservices and event-driven mechanisms.
This post and all the other Services posts are rather technical, so if you were more interested in the design of
individual pages, have a look at the Design posts.
If you want to learn about all the moving parts that make this website run however - grab yourself a coffee, get comfy &
enjoy reading! :)
Let's start with a simple list of all the services that work together in harmony! Many of which I built myself and others that run in docker containers alongside them.
This section gives a brief description, and you can find more details on each of the services in their designated posts.
Frontend
The actual website that you are currently interacting with to read this post. It is built with the Svelte Kit framework, Svelte 5, Tailwind CSS and written in TypeScript.
Most of the data like texts, images, files & more are saved in the database, cache and Static File Server. All requests go through the API Gateway.
API Gateway
I am using Kong's API Gateway to orchestrate requests from the frontend to the backend
services.
It can manage traffic with rate limiting, handle authentication, and also supports a bunch of plugins from
Kong and the community! It is built on top of NGINX, utilizing its reverse proxy capabilities.
Cache API
This is the main API for when data is requested from the frontend! It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route GET-Requests to the Cache API first because the cache stores entire data objects for faster lookups compared to the database!
It can also request data from the database via the Database API in case of a Cache-Miss (more details in Application Flow).
Cache
I am using a Redis cache to store already built data like JSON objects, Strings, Sets and more. It is a "dumb" storage that doesn't handle any calculations or joins.
Database API
This is the main API for when data is created, updated and deleted. It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route POST-, PUT-, PATCH-, and DELETE-Requests to the Database API first because the database is the single source of truth - so any data modification should happen here first!
A message is published via the Message Queue in most cases after data has been modified to let the Cache-Sync-Service invalidate connected cache keys (more details in Application Flow).
It can also fetch from the database to build objects that are saved in the cache - this usually happens in case of a Cache-Miss.
Database
I am using a Postgres database which is the single source of truth for all the data. It made most sense to use a relational database for the website because a lot of the data is actually connected to one another.
Postgres also allows the use of functions on a database-level & more to tailor the saved data to my exact expectations - so even if unwanted data were to come through the frontend and APIs, the database would check it a third time before saving.
Cache-Sync-Service
A small microservice written in Rust. It subscribes to the Message Queue and looks for cache invalidation events. Each message holds a list of values that indicate which cache keys should be invalidated.
It connects to the Cache API whenever it receives a message and sends an invalidation request for all given keys.
Those messages are usually sent by the Database API after modifying the database.
Message Queue
I chose RabbitMQ for the Message Queue to handle events like cache invalidation. It includes a dead letter exchange out of the box, which improves data consistency and helps with error-handling.
Everytime a message is sent, there has to happen an acknowledgement before the message is finalized or else it is sent to the dead letter exchange. This way messages that couldn't process successfully aren't just dropped and can be retried instead.
Protobuf
I decided to use Protobuf as part my data serialization for DTOs (Data Transfer Objects).
Reason being that multiple services are using the same DTOs.
Let's take this Project Post for example:
- It is received in the frontend from the Cache API,
- saved in the cache as JSON object,
- returned by the Database API in case of a Cache-Miss.
So this means that the Database API, Cache API and frontend all have to know about the same DTOs. Thankfully I can automatically generate Rust structs and TypeScript interfaces for this, so that they are all in-sync!
Static File Server
NGINX is often used as a reverse proxy, but it can be configured as a file server too. So files like Images and PDF documents that are saved on my server can be safely accessed through the Static File Server from their specified directory.
As I already mentioned in the Introduction, this is not a simple static website or WordPress page. Instead, I am using multiple microservices and event-driven mechanisms.
This post and all the other Services posts are rather technical, so if you were more interested in the design of
individual pages, have a look at the Design posts.
If you want to learn about all the moving parts that make this website run however - grab yourself a coffee, get comfy &
enjoy reading! :)
Let's start with a simple list of all the services that work together in harmony! Many of which I built myself and others that run in docker containers alongside them.
This section gives a brief description, and you can find more details on each of the services in their designated posts.
Frontend
The actual website that you are currently interacting with to read this post. It is built with the Svelte Kit framework, Svelte 5, Tailwind CSS and written in TypeScript.
Most of the data like texts, images, files & more are saved in the database, cache and Static File Server. All requests go through the API Gateway.
API Gateway
I am using Kong's API Gateway to orchestrate requests from the frontend to the backend
services.
It can manage traffic with rate limiting, handle authentication, and also supports a bunch of plugins from
Kong and the community! It is built on top of NGINX, utilizing its reverse proxy capabilities.
Cache API
This is the main API for when data is requested from the frontend! It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route GET-Requests to the Cache API first because the cache stores entire data objects for faster lookups compared to the database!
It can also request data from the database via the Database API in case of a Cache-Miss (more details in Application Flow).
Cache
I am using a Redis cache to store already built data like JSON objects, Strings, Sets and more. It is a "dumb" storage that doesn't handle any calculations or joins.
Database API
This is the main API for when data is created, updated and deleted. It is written in Rust and uses the asynchronous Actix Web framework.
The API Gateway will always route POST-, PUT-, PATCH-, and DELETE-Requests to the Database API first because the database is the single source of truth - so any data modification should happen here first!
A message is published via the Message Queue in most cases after data has been modified to let the Cache-Sync-Service invalidate connected cache keys (more details in Application Flow).
It can also fetch from the database to build objects that are saved in the cache - this usually happens in case of a Cache-Miss.
Database
I am using a Postgres database which is the single source of truth for all the data. It made most sense to use a relational database for the website because a lot of the data is actually connected to one another.
Postgres also allows the use of functions on a database-level & more to tailor the saved data to my exact expectations - so even if unwanted data were to come through the frontend and APIs, the database would check it a third time before saving.
Cache-Sync-Service
A small microservice written in Rust. It subscribes to the Message Queue and looks for cache invalidation events. Each message holds a list of values that indicate which cache keys should be invalidated.
It connects to the Cache API whenever it receives a message and sends an invalidation request for all given keys.
Those messages are usually sent by the Database API after modifying the database.
Message Queue
I chose RabbitMQ for the Message Queue to handle events like cache invalidation. It includes a dead letter exchange out of the box, which improves data consistency and helps with error-handling.
Everytime a message is sent, there has to happen an acknowledgement before the message is finalized or else it is sent to the dead letter exchange. This way messages that couldn't process successfully aren't just dropped and can be retried instead.
Protobuf
I decided to use Protobuf as part my data serialization for DTOs (Data Transfer Objects).
Reason being that multiple services are using the same DTOs.
Let's take this Project Post for example:
- It is received in the frontend from the Cache API,
- saved in the cache as JSON object,
- returned by the Database API in case of a Cache-Miss.
So this means that the Database API, Cache API and frontend all have to know about the same DTOs. Thankfully I can automatically generate Rust structs and TypeScript interfaces for this, so that they are all in-sync!
Static File Server
NGINX is often used as a reverse proxy, but it can be configured as a file server too. So files like Images and PDF documents that are saved on my server can be safely accessed through the Static File Server from their specified directory.