This content originally appeared on Bits and Pieces - Medium and was authored by Davit Vardanyan
Notifications System Design: How We Integrated It into Our Infrastructure
How we designed and integrated a notifications system into our infrastructure.
What are the notifications for businesses?
It’s obvious that no digital business can live without notifications. By sending timely notifications through various channels such as email, SMS, or push notifications, businesses can improve customer engagement and retention, as well as encourage repeat purchases. Notifications also provide an opportunity for businesses to gather customer feedback and insights, which can be used to improve their products or services.
Ultimately, a well-designed notification service can help businesses establish a strong and lasting relationship with their customers, leading to long-term success. So, at some point in time, all products provide requirements to tech for notifications integration.
Planning the integration
As the functional/non-functional requirements for notifications are mostly similar between companies/businesses, and everyone understands what is a notification feature, we can quickly note some business requirements and immediately start talking about tech.
- Anything that happened in the system from the business point, can trigger notification(s).
- Notifications to send can be targeted to one or more recipients or a group of recipients.
- We need to support different channels of notifications: Email, SMS, Push for Android/iOS devices, in-app.
- Notifications should be localized, with means to support different languages and time zones at least.
- Users should have access to their previous/old notifications (AKA notifications or notification center).
- Some push or email notifications shouldn’t be recorded/visible in the user’s notification center (such as a marketing alert about a discount, or another short-living alert), and vice versa, some notifications that are visible in the user’s notification center, shouldn’t have produced an alert such as a push notification (or an email have been sent).
- Users should be able to customize the notifications consumption (frequency, categories, channels ..), for example, set a preferred channel for some type of notifications, or even turn that off.
So, the above requirements are basic business needs on notifications in most companies. And we need to design a service/infrastructure that meets these requirements perfectly and also encourages a seamless development process by meeting some key tech needs that have a direct impact on overall development. Some of them are:
- The setup code should be easy to understand/maintain.
- Integration of new notifications is simple, fast and straightforward, without modifying any existing code, but only by extending it by a pattern/convention that is used for the previous notifications.
- Easy to change existing notifications without affecting other parts (e.g. remove the push channel for a specific type without touching any setup code, without affecting other notifications).
- Easy to add new channels/providers (e.g. browser notifications, WhatsApp messages).
- A very straightforward way of adding/editing new translations (localization) for specific notifications. And quick or even immediate reflection of new translations to the working system.
- Adding new settings for users to customize notifications is not a headache.
- Overall service is highly testable, so covering with integration/backend e2e tests is not time-consuming
- Easy to debug specific cases.
Now let’s move on to tech design grooming
As our infrastructure consists of little services each having its own clear responsibilities in the overall system, our notifications system will also be one or a few little services communicating with others via synchronous and asynchronous methods. At least we will need a backend service that handles the triggers and does the core operations with regards to notifications.
We will also need another service for building the content for notifications and another service for managing the translations. Triggering notifications can happen via the actual triggers (something happened in some service and a message/event is published to the event bus) or the triggers will be managed by another new service which will be the event service.
The event service, the translations and the templating services are out of scope of this article so we will mostly focus on notifications core service.
There are many 3rd party services that can be integrated into different parts of our future notification system to make the development faster and easier, but it will add additional cost as each 3rd party service charges for the service. Therefore a decision was made to use 3rd party services (wrapper services as we call them) but build our custom implementations. For example, a popular push notification provider — OneSignal, makes development a bit easier, especially when it comes to device management and targeting, but it continuously charges companies for that what can be built one time. Or for emails — a templating engine with a couple of simple APIs will save us a lot of costs in comparison to using something like Mailjet.
What we will need to have
- Trigger. We will need to have triggers that will produce notifications. E.g. in an e-commerce business “order-created” event where we need to send notifications to the order owners, means buyer and seller, and maybe another notification to finance or whatever company department. Let’s consider we already have triggers from other services, in the form of events mentioned above.
- Templating service(s) for building email messages based on templates, similar but maybe less complicated generators for other channels (Push, SMS ..). There are lots of open-source solutions for that in almost all the popular languages.
- Localization service for translations. For example, we don’t want to persist a bunch of text in DB for showing notification history in the client app, but we want the app to build the text depending on the language and other parameters after the history is fetched to the client app. Same for the backend, for sending localized push we need to build localized push notifications using our translations. As we want to have localized notifications I assume our infrastructure already supports localization, and we already should be using some custom build or 3rd party service. In my case, our company already had localization service.
- Service that handles the trigger, does the core logic and is capable of sending and/or persisting notifications.
- Provider integrations. Email, APNS for ios, FCM for Android, etc. depending on what providers we need.
System design
Mainly, any notification will start from an event whether published from a business service or a special event service. The event itself will be published to an event bus. Later it will be picked up by the notifications service (the core service). The notification service will prepare the data required for a specific notification using the data provided by the event. After, it will use the prepared/mapped data for building a notification case which is the declarative mechanism for processing a notification.
Before a notification is sent, it should be “converted” to a type of content/payload that a specific provider/channel expects. For this purpose, we use the templating service for getting the fulfilled email content based on a specific template using the data (parameters) and the user language.
For push or SMS notifications it is easier, as it’s just a text. So we use the user language for simply getting the translations by replacing the parameters/attributes.
For translating, it makes no sense to call the localization service each time, so, a solution might be to temporarily persist the notification-related translations locally in each notification service replica, and maintain the updates by simply polling for new changes in localization service. Or we can use an event-based approach for the last. Or a maintained distributed cache mechanism can also easily solve this problem.
After we have the content/payload, the notification service just makes an HTTP call to the provider(s) by sending it, and ring-ring-ring, we got a push notification.
Additionally, the notifications service will expose endpoints for users to fetch their notifications, for marking these as read/unread, etc. It will also expose another set of endpoints for administrative applications to manage notifications, related configurations etc. Or even for creating marketing notifications, e.g. alerting about a storewide discount.
The chunk of notifications in the infrastructure will look as shown in the diagram below:
After we decided on the high-level architecture, it’s time to dive deeper into specific components and make these clearer by taking more specific design decisions. We can decide the language that this service will write in. Also, the DB is where we store the data. For taking advantage of the strict schema (here we don’t have any arbitrary data, everything is pre-decided), relations, constraints, and ACID we choose a relational DB such as MySQL or PostgreSQL.
Service Architecture
Now let’s design the entities and their relationship.
The ERD above can be slightly changed to customize it for company needs. But, it’s simple, normalized to 5NF, and good enough not only as a starting point but as a final schema as long as we don’t have specific requirements that might impact the ERD. Sure, we have redundancy, but it’s not at the service/db level but at an infrastructure level. For example, here we have a tiny copy of the user's table, which is totally normal in microservices. Another note is that we don’t want to generate PKs for users' records, but we use the same UUID that we have in our principal user’s table in the user's service (because we know no user will be recorded here if he is not registered, means he doesn’t have a record in users service). In our case, this approach has more pros than cons.
Now let’s dive into a more detailed description of the service.
Consumer
The notification service has domain-specific consumers, e.g. order-consumer, which is had consumer handlers to order related events. And when it’s triggered it calls domain-specific service, and order-service accordingly.
Service
Order service constructs an entity called Case (notification case). Order service can additionally do some synchronous communication with other microservices if need more data for constructing the specific notification case.
Case
A Case is an entity that means a specific notification case, and it has a type which is a concrete type of a concrete notification. Each case extends a class called BaseCase, it’s an abstract class containing some reusable logic, and some abstract properties and methods. So, each case has to implement some base requirements before it can be considered a complete notification case. Each notification case is constructed by accepting some data, such as events, attributes, recipient ids and some options. On each triggered notification a new instance of Case is created, which represents a specific notification to one or more recipients, and some instructions on how to process this exact notification. And it’s ready to be processed by the notification engine.
Notification Engine
It’s a little engine that knows how to process any notification case. We can say that the notification engine knows how to do, but doesn’t know what to do. But a notification case doesn’t have any idea/instruction on how to do it but knows what should be done. So the collaboration of a Case with the engine does the whole job. A notification engine is a little processor that does all the notification-related routines, such as calling notification service for creating notification and notification recipient records in the database; calling templating and translation services for preparing notification payloads; calling notification transport module (channel providers) for sending the actual notifications if needed; or even knows how to handle common exceptions that might happen during the flow.
By having this architecture, we write as less code as possible for creating new notifications. It is as straightforward as just adding the consumer on a specific event and creating a new notification Case class, that’s it for adding one more notification.
Templating module
It’s basically an HTTP provider to the templating service. It uses HTTP requests for building email content based on notification type, attributes and language based on a pre-created email template. On each email, it does build new content based on the template.
For push notifications, it accepts translated text and attributes and returns the payload that is ready to send.
Localization module
This module has 2 key features:
- Request full translations from localization microservice and maintain it. There can be a few ways of solving the maintenance of translations, as mentioned earlier, polling for new translation updates, event-based updates, or distributed cache. All methods have their pros and cons.
- Translate notification type. Translations for notification cases are made easy because each notification case has a notification type, and notification translation keys can be made by following a convention using the notification type. For example,
app.notifications.cases.<notificationType>.title
app.notifications.cases.<notificationType>.body. So now we can get the body translation of any notification case by using this key.
Users module
It is a small copy of the company’s users. Consists of the same id for ease of maintenance, email for sending emails, and the type of user. It can also have a few more fields that are important for the notifications service.
For push notifications, each user can have one or more devices. Devices are fetched by recipient ids (user ids) when needed to construct a notification for recipients and are also used to send notifications via device token.
Channel providers
This is the last step of the processing engine so it just calls provider.send(payload, device). We create our custom clients, for example, for APNS calls we use http2 and p12 certificate for creating an APNS client, that knows how to send and handle responses from APNS while sending push notifications. Similarly for email or Android notifications. This way it’s cheap and reliable. Exactly what we want.
Exposed APIs
We expose 2 types of APIs:
- For customers (regular users) at least to
• Send device tokens and other device-related information
• List notifications
• Mark one or more notifications as read/unread
• Change notification-related preferences - For “admin” users to
• Manage notifications
• Manage various configurations
• Send/Manipulate new/existing notifications
• Monitor
A brief diagram of a notification core service described above can be found right below: 😉
💡 Building a notifications system can be complex — but by adopting a microservices architecture and making the right decisions on identifying and isolating each microservice, you can go a long way into making your system more reliable.
Treating the microservices architecture like composable building blocks is key, and it’s made easier with a tool like Bit which allows your teams to independently publish, version, document, test, and share individual components such as functions, UI elements, or data models, that can be reused across multiple microservices. This can greatly reduce the duplication of code and increase the modularity and scalability of your system.
Learn more here:
Component-Driven Microservices with NodeJS and Bit
Conclusion
Designing a notifications system requires careful consideration of various factors such as user preferences, messaging protocols, and delivery channels. But when it’s a part of an existing infrastructure then it needs us to consider even more factors. By taking a user-centric approach and leveraging the right tools and technologies, it is possible to create a system that delivers timely, relevant, and personalized notifications while minimizing the risk of spamming or overwhelming users. A well-designed notification service can cost nothing to maintain and is very easy to extend, also will be cheap and reliable.
Thank you for reading.
If you want to see more similar content, clap a few times so I get motivated 😉
From monolithic to composable software with Bit
Bit’s open-source tool help 250,000+ devs to build apps with components.
Turn any UI, feature, or page into a reusable component — and share it across your applications. It’s easier to collaborate and build faster.
Split apps into components to make app development easier, and enjoy the best experience for the workflows you want:
→ Micro-Frontends
→ Design System
→ Code-Sharing and reuse
→ Monorepo
Learn more
- From Monolith to Microservices Using Tactical Forking
- Building Microservices with Confidence: Key Strategies and Techniques
- Microservices Aren’t Magic, but You Might Need Them
- How We Build Micro Frontends
- How we Build a Component Design System
- How to reuse React components across your projects
- 5 Ways to Build a React Monorepo
- How to Create a Composable React App with Bit
- Different Protocols for Microservice Communication
Notifications System Design: How we integrated it into our infrastructure was originally published in Bits and Pieces on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Bits and Pieces - Medium and was authored by Davit Vardanyan
Davit Vardanyan | Sciencx (2023-05-03T05:09:04+00:00) Notifications System Design: How we integrated it into our infrastructure. Retrieved from https://www.scien.cx/2023/05/03/notifications-system-design-how-we-integrated-it-into-our-infrastructure/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.