This content originally appeared on Bits and Pieces - Medium and was authored by Piumi Liyana Gunawardhana
How to Scale your NodeJS Application Execution on Multiple Processor Cores
When it comes to business applications, we often striving to improve performance. But, sometimes, we go ahead with potential trade-offs.
One such case is with NodeJS. Since it uses JavaScript, we often choose it without much hesitation. But, there are inherited limitations with NodeJS that affect the performance of some workloads.
So, in this article, let’s look at one such use case optimizing a NodeJS application to handle heavy workloads.
What is NodeJS Clustering?
Not many realize that NodeJS runs in a single thread by default. Besides, it utilizes only one CPU core for this thread. So, for example, if you’re using a 4 core machine, Node will only be using a single core.
To take advantage of multi-core systems, we need to launch a cluster of NodeJS processes to handle the load.
So, we can use the NodeJS Cluster module to leverage all the processors in the system. It will spin up copies of your program, so if you’re in a 4 core machine, you’ll be running 4 copies of your program. That means you can handle 4 times the traffic capacity. This inevitably gives a performance boost to your NodeJS application.
How Does Cluster Module Work?
The NodeJS Cluster module creates several child processes (workers) that operate parallelly sharing the same server port.
Each generated child has its own V8 instance, event loop, and memory. Master process and the child processes communicate with each other via IPC(Inter-Process Communication).
With multiple processes, if one is occupied with a CPU-intensive task, the other processes can handle the other requests, employing the other CPUs/cores available. The effectiveness of the cluster module lies in the fact that workers balance the load and the application does not come to a halt because of heavy loads.
Incoming load is divided across child processes in either way listed below.
- The parent process listens for the incoming load on a port and shares them among the workers in the round-robin method. Other than in Windows, this is the default mode on all systems.
- The parent process yields a listen socket and distributes it to interested workers, who can then instantly handle traffic coming.
Advantages:
- The use of all the available cores for application execution improves scalability.
- If a generated process dies unexpectedly or on purpose, a new process can be created immediately as a substitute for the deceased one without waiting or any manual intervention.
- It’s simple to set up because the NodeJS module handles everything and there’s no necessity to add extra dependencies.
- The number of resources wasted was drastically reduced by utilizing the processor’s full capacity.
Disadvantages:
- Session management isn’t possible; instead, a developer must handle the alternatives, which adds to the complexity.
- IPC is a time-consuming task for managing an application, and it is not recommended for some applications.
How to Use Clustering in Your Node Application?
To understand the benefits of clustering, we’ll look at a sample NodeJS app with clustering comparing it with one without it.
Let’s start creating a simple NodeJS server without clustering, performing a CPU-intensive task blocking the event loop on purpose.
Setting up a simple NodeJS Express Server
Set up a new project by executing the following commands on the terminal.
mkdir nodejs-clustering
cd nodejs-clustering
npm init -y
npm install --save express
Then create a file called “no-clustering.js” and after that, your project folder structure should look like this.
Add this code to no-clustering.js file.
Let’s have a check at what the code does. We’ll begin with a basic Express server running on port 3000. It has two (/) URIs that display a message “Hi There! This application does not use clustering…” and /api/nocluster is an alternative route.
The lengthy loop in the nocluster API GET method loops 8⁷ (2,097,152) times. It performs a math.pow(), which returns the base to the exponent power and add it to a number, in each loop and is assigned to the result variable. Then, it console logs that number and returns it as the response.
It was designed to be blocking and CPU-intensive in order to observe how it would affect a cluster afterwards.
Run the server with node no-clustering.js, and you will be able to see the following output after going to http://localhost:3000/api/nocluster.
The terminal where the Node server is running will appear like this.
Adding Clustering to the NodeJS Express Server
Add a with-clustering.js file with the same content as in the no-clustering.js file but here we will be using the Cluster module.
If we dissect the code from the beginning, you can see that we have required the Cluster module, and require(‘os’).cpus().length returns number of CPU cores available, it was 8 in my machine. First, we check whether the cluster is a master; then we fork a similar number of child processes(workers) to the cores available. Only if it is not a master process a worker will call the start function.
The program works similarly as earlier, but this time we’re creating many child processes that will share port 3001 and be capable of handling requests made to that port. The fork() technique is used to start the worker processes. It returns a ChildProcess object with an integrated data transmission connection for passing messages across the child and its parent.
When you run the above code in with-clustering.js file, it will give you the following output on http://localhost:3001/api/withcluster.
The terminal will appear like shown below. You can see that all the available 8 cores are utilized here.
However, one of the best practices is that you should not create more workers than the number of available logical cores, resulting in scheduling overhead.
This occurs because the system must arrange all of the newly formed processes, so each takes a turn on the few cores available.
Build & share independent JS components with Bit
Bit is an extensible tool that lets you create truly modular applications with independently authored, versioned, and maintained components.
Use it to build modular apps & design systems, author and deliver micro frontends & microservices, or simply share components between applications.
Bit: The platform for the modular web
Performance Comparison
To illustrate the effect of clustering on NodeJS application performance, let’s test how our two apps manage a huge number of incoming connections by running a load test on them. This will be conducted with the loadtest package.
The loadtest package makes it easy for you to create test the performance of your API by simulating a significant number of parallel connections.
First, you need to install the loadtest package globally with the command below.
npm install -g loadtest
Then run the application you need to perform a load test on. Open and another terminal while it is running and execute the below command to run a load test.
I will be testing the application “without clustering” first.
loadtest http://localhost:3000/api/nocluster -n 1000 -c 100
This will set the max number of requests to 1000, and the level of concurrency to 100 for the application URL. You will get the following result at the end of the test.
The server was capable of handling 7 requests per second with a mean latency of 13168 milliseconds (the average time it took to complete a single request).
Then stop running the application “without clustering” and test the application that we developed “with clustering” for the same load test.
loadtest http://localhost:3001/api/withcluster -n 1000 -c 100
Below are the test results for the application with clustering.
When comparing to the 7 requests per second handled by the application without clusters, the application with clusters was able to manage 35 requests per second.
The clustered application has a mean latency of 2745.6 milliseconds, contrasted to 13168 milliseconds for the app without clusters.
It is obvious that clustering has done a significant enhancement to the app. It is a performance boost by 5 times than before.
Conclusion
The results above speak for themselves. The time per request in a single-threaded environment increases drastically as the number of concurrent requests increases. When the application was upgraded to leverage clusters, we witnessed a significant enhancement in throughput.
Another important fact that you need to keep in mind is “clustering excels at CPU intensive tasks.” Therefore, when your application is expected to perform such operations, clustering provides a benefit concerning the number of operations it can perform at once. But, if your software doesn’t perform many CPU-intensive operations, it is not usually advisable to launch a high number of child NodeJS processes due to the excessive resource allocations.
So, I hope this article gave you insights into using the Cluster module to scale your NodeJS application. Then, try this out with your next Node project to fully utilize all the available CPU cores. It will allow you to improve the performance of your NodeJS application by making better use of system resources.
Thank you for reading…….!!!
Learn More
- Is Deno Secure, Fast, Lightweight, and Better than Node?
- Using HTTPS for Local Development for React, Angular and Node
- Using the Performance Web API with Chrome DevTools
NodeJS Performance Optimization with Clustering was originally published in Bits and Pieces on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Bits and Pieces - Medium and was authored by Piumi Liyana Gunawardhana
Piumi Liyana Gunawardhana | Sciencx (2021-08-12T17:51:07+00:00) NodeJS Performance Optimization with Clustering. Retrieved from https://www.scien.cx/2021/08/12/nodejs-performance-optimization-with-clustering/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.