This content originally appeared on DEV Community and was authored by Hilton Fernandes
Financial markets have discontinuities: prices jump up or down in a time so short that it can be considered a real discontinuity if the time measured by our clocks were really continuous, real line continuous.
Those discontinuities create problems for many forms of mathematical modelling, since their modelling are based upon continuous functions. For instance, many price oscillations look like periodic functions, but when a discontinuity is found, any harmonic analysis becomes troublesome.
Actually, a trend can also be troublesome to the fitting of periodic functions to financial data. But in this case, fitting a polynomial of low grade to the data can filter the trend and then a periodic function series can be fitted to the residuals, the difference between the fitted polynomial and the original data.
The purpose of this suite of articles is to present a simple method to eliminate jumps from the observed data. Of course, when reconstructing the fitted data, the discontinuity will be added back.
Roadmap
This one, the 1st of 4 small articles will present the general concepts for the solution, the 2nd one will present the implementation of the solution in Python, the 3rd article will add a sinoidal decomposition of the data after the filtering of the solution, and the 4th and last one will use all the elements to attack a real problem in cryptocoins.
Cluster analysis as a means to group similar data
Cluster analysis is a means to group data elements that are similar between themselves. In a metric space, similarity means closeness. There are several ways to devise the groups or clusters of data, and one of the simplest is called k-means. In very few words, it creates clusters by assigning a mean average of the coordinates to a point, that's a centroid.
In two dimensions, that's a typical representation of two groups in two dimensions.
The points are in blue, and the centroids are in red.
Cluster analysis in the line
Since the k-means clustering is based upon distance of the points, an interesting effect will happen when the points are in a line, therefore much closer than in the a cloud, like in the previous image.
Consider the following image that shows a time series with a discontinuity.
When a cluster analysis is applied to it, the centroids of the clusters are shown in red.
It's easy to see that the two groups are in different levels, as shown in the following image:
Then to eliminate the discontinuity, it's enough to lower the Group 1 to the level of the Group 2. That is: to subtract from the points y coordinate the difference between the level of the two groups.
That can be shown in the following image:
Now the two groups are indifferenced.
This content originally appeared on DEV Community and was authored by Hilton Fernandes
Hilton Fernandes | Sciencx (2025-10-30T18:52:11+00:00) Leveling with cluster analysis in Python. Retrieved from https://www.scien.cx/2025/10/30/leveling-with-cluster-analysis-in-python/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.