This content originally appeared on Bits and Pieces - Medium and was authored by GaurangMittal
In the case of Database per service pattern, how to handle ACID properties while Implementing Distributed Transactions. What are the different patterns to handle these?
What are Transactions?
A transaction refers to a unit of work performed on a table in a DBMS in a coherent and reliable way , such that it is independent of other transactions.
A transaction generally represents any change in a database.
What are Distributed Transactions?
Transactions that span over multiple physical systems or computers over the network, are simply termed Distributed Transactions.
In the world of microservices a transaction is now distributed to multiple services that are called in a sequence to complete the entire transaction.
Example Use Case:
10 Min Food Delivery Platform
A food delivery partner is trying to launch a 10 min food delivery service , while insuring two major components/parties in the backend, they are:
- Food item must be available with the nearby store/selected store.
2. A delivery agent should be available to deliver the item in 10 mins.
When we decompose this system, we created the microservices Store Service and Delivery Service, which have separate databases.
When a Place Order request comes from the user, both these microservices will be invoked to apply changes into their own database. Because the transaction is now across multiple databases via multiple systems, it is now considered a Distributed Transaction.
Problems Associated With Distributed Transactions
The shift towards microservice architecture has resulted in a loss of the ACID properties of databases. Transactions that involve multiple microservices and databases can pose significant challenges, including:
How do we keep the transaction atomic?
Atomicity means that in a transaction either all steps are completed or no step is completed. In the example above, if the ‘assign agent’ in the Delivery Service method fails, how do we roll back the ‘Update store’ changes that were applied by the Store Service?
How do we handle concurrent requests?
If an object from any one of the microservices is being persisted to the database and at the same time, another request reads the same object.
Should the service return the old data or new ?
In the example above, once Store Service is complete and the Delivery Service is now performing its update, should requests for number of orders placed by the customer include the current order?
Ways to Tackle Above Problem
The issues described above are critical in the creation and development of microservice applications.
The following strategies can be employed to resolve them:
- Two-Phase Commit.
- Eventual Consistency and Compensation or SAGA.
1. Two-Phase Commit(2PC)
The 2PC as the name suggests, it handles the transaction in two stages, a prepare phase and a commit phase.
It executes set of distributed transactions as a single, atomic unit.
They have a Transaction Coordinator which maintains the lifecycle of the transaction.
Working
In the prepare phase, each service involved in the transaction, votes either to commit or abort the transaction.
If all the services votes to commit the transaction, then the transaction enters the commit phase, where all services involved in the transaction are notified to execute the transaction.
The transaction coordinator might also issue a rollback command to all the microservices.
Lets take the 10 mins food delivery system as an example:
In the above example, the TransactionCoordinator will first start a global transaction with all the relevant context data when a user sends a Place Order request.
To reserve the food, it will first send a prepare command to the Store Service. In order to reserve the agent, it will then send a prepare command to the Delivery Service. When both services agree that the change can go through, they lock down the objects to prevent any additional changes and alert the TransactionCoordinator.
The TransactionCoordinator will then ask each microservice to persist its changes by requesting a commit with the transaction after it has determined that each microservice is prepared to apply its changes.
Problems of a 2-phase commit Implementation Wise
After a system confirmed the prepare command, it needs to make sure that it will be able to commit the transaction when it receives the commit command. That means nothing is allowed to change until that system gets the commit or abort command.
The only way to ensure that is to lock all the information that you changed in the transaction. As long as this lock is active, no other transaction can use this information. These locks can become a bottleneck that slows down your system and should obviously be avoided.
This problem also existed in a distributed, monolithic application. But the small scope of a microservice and the huge number of services that are often deployed make it worse.
A 2-phase commit between a transaction coordinator and 2 external systems is already bad enough. But the complexity and the performance impact of the required locks increase with each additional external system that takes part in the transaction.
Benefits
- 2PC guarantees atomicity of the transaction. The transaction will end with either all microservices being successful or all microservices have nothing changed.
- It insures Isolation of objects,the changes on objects are not visible until the transaction coordinator commits the changes.
- The approach is a Synchronous Call, where the client would be notified of success or failure.
Drawbacks
- High Latency Task -2PC transactions are quite slow as compared to transaction on a single service. They are highly dependent on the transaction coordinator, which can really slow down the system during high load.
- The lock could become a performance bottleneck and it is possible to have a Deadlock, where two transactions mutually lock each other.
2. Eventual Consistency and Compensation / SAGA
According to microservices.io , eventual consistency means:
“Each service publishes an event whenever it updates its data. Other service subscribe to events. When an event is received, a service updates its data.”
Working
In this approach, each business transaction is implemented as a saga.
A saga is a sequence of local transactions.
In each local transaction, the related microservices updates its database and publishes a message or event to trigger the next local transaction in the saga. If a local transaction fails because it violates a business rule then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.
There are two ways of coordinating sagas:
- Choreography — each local transaction publishes domain events that trigger local transactions in other services.
- Orchestration — an orchestrator (object) tells the participants what local transactions to execute.
Lets take the 10 mins food delivery system as an example:
In the above image, the client requests the system to Place the Order.
On this request the Choreographer emits an event Update Store, marking the start of the transaction.
The Store Service listens to this event and updates the food item inventory, if it was successful it emits an Food Item Updated event.
The Choreographer listens for this event and proceeds to reserve the agent, by emitting the Reserve Agent event.
The Delivery Service listens for this event and reserve’s the agent, if it was successful it emits an Delivery Agent Reserved event. Which in this example means the end of the transaction.
All the event based communication between microservices happens via the Event Bus and is Choreographed by another system to address the complexity issue.
If for any reason the Delivery Service failed to reserve the agent, it could be due to multiple reasons such as no agents available, all agents busy, the order placed location is not serviceable right now, all depends upn the business logic.
It emits a Failed to Reserve Agent event. The Choreographer listens for this event and starts a Compensating Transaction, by emitting an Undo UpdateStore event. The Store Service listens to this event and deletes the food item-related changes made.
Benefits:
- It enables an application to maintain data consistency across multiple services without using distributed transactions.
- No deadlock.
- Highly Scalable.
Drawbacks:
- The programming model is more complex. For example, a developer must design compensating transactions that explicitly undo changes made earlier in a SAGA.
Fortunately, an open-source tool like Bit provides a solution to this complexity by enabling developers to create and manage reusable components that encapsulate the logic of the compensating transactions. This means that once the components are created, they can be reused across different SAGA implementations, saving time and effort in development.
Additionally, Bit provides a centralized platform for managing and sharing these components. This can significantly simplify the programming model for SAGA, reducing the complexity and effort required to design compensating transactions.. Learn more here.
- No read Isolation to end user. Though there could be ways to handle this scenario depending on the business requirements in that architecture.
Conclusion
While building a monolithic architecture, try to avoid having distributed transactions, as much as possible.
If we need to make changes in two places or usage of transaction spanning across services then as opposed to the two-phase commit, the Eventual Consistency/SAGA technique is a better method of handling distributed transactions when it becomes necessary to change data in two places as a result of a single event.
The Eventual Consistency approach also introduces a new set of problems, such as how to atomically update the database and emit an event.
References
- https://thorben-janssen.com/distributed-transactions-microservices/#:~:text=In%20a%20microservice%20architecture%2C%20a,choreographed%20SAGAs%20avoid%20these%20problems.
- www.microservices.io
Please follow me & give this article your love & your valuable feedback.
Check out my profile to see more such content.
You can connect with me on LinkedIn.
From monolithic to composable software with Bit
Bit’s open-source tool help 250,000+ devs to build apps with components.
Turn any UI, feature, or page into a reusable component — and share it across your applications. It’s easier to collaborate and build faster.
Split apps into components to make app development easier, and enjoy the best experience for the workflows you want:
→ Micro-Frontends
→ Design System
→ Code-Sharing and reuse
→ Monorepo
Learn more
- How We Build Micro Frontends
- How we Build a Component Design System
- How to reuse React components across your projects
- 5 Ways to Build a React Monorepo
- How to Create a Composable React App with Bit
Handling Distributed Transactions in Microservices was originally published in Bits and Pieces on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Bits and Pieces - Medium and was authored by GaurangMittal
GaurangMittal | Sciencx (2023-02-28T02:59:27+00:00) Handling Distributed Transactions in Microservices. Retrieved from https://www.scien.cx/2023/02/28/handling-distributed-transactions-in-microservices/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.