Designing to Patterns: A Pythonic Example

Photo by Jack Carter on UnsplashIntroductionThe principle goal of software design is not only that the software is functional, but also that the software is easy to maintain and extend. Thus, new features can be readily added, and incoming bugs can be …

Photo by Jack Carter on Unsplash

Introduction

The principle goal of software design is not only that the software is functional, but also that the software is easy to maintain and extend. Thus, new features can be readily added, and incoming bugs can be quickly fixed. To this end, the SOLID design principles (introduced below) can be used to identify areas in your code to refactor. The problem though is that the principles dictate “what” not “how”. To gain deeper insight to the “how” of achieving extensible, modular design, design patterns can be leveraged. In this article, we discuss the design thinking that goes into moving from design principles to design patterns to refactored code. This will be illuminated with an example in Python.

Example Problem

Consider the following case. Given an object stub (incompletely built object), that contains the keys to units of data stored in a persistent key-value store, a service layer function is needed that will fetch all of the data, and re-construct the full object. The motivation for this is that, rather than storing only the constructed object, by storing the original units the object could be rebuilt in case the algorithm turning units into the final object were to change. So whatever function does the rebuilding is expected to change. There are 7 different object types in total, and they all have in common that the data to re-build them is split across multiple units: multiple database queries are needed to rebuild the full object. However, each object has a unique construction signature so one interface is insufficient. Nevertheless, there are two patterns that are followed by construction objects. The first:

We see the object can be created by iterative calls to an object-specific build() function. The second pattern is different:

Here the object needs all the final data together in one place and an aggregate method (sometimes in several stages) is called to construct the object in one go from all the units.

Given these two patterns, and the fact that the actual signature is unique for each object, how can we build a single service layer method that will re-construct the objects for us? The design goal is not only a service layer to reconstruct the objects, but one that is easy to extend in case new object types are added, and safe to modify in case the construction algorithms have changed.

Naïve solution

A first pass solution is worth noting because all too often this is really what gets written. I have sometimes seen such examples in production code.

This should not be done. This example violates most of the SOLID principles, and doesn’t meet our design goals. Let’s learn some design principles, and then look at some design patterns to achieve extensible and maintainable code.

Design Principles

Design principles help identify code that needs refactoring and give guidance in where that refactoring needs to go. They serve as guiding light rather than strict rules that have to be followed, and often are insufficient to tell a developer what needs to be done, rather indicating that something needs to be done, leaving the developer to figure out the how by himself. The why you should follow design principles are several. First, by coding to a set of principles, regardless of what they are, your code will become easier to reason about because the same patterns appear repeatedly. You basically come to know what to expect in your own code, rather than finding scattered chaos. Second, following design principles serve to make code easier to understand, easier to reason about, and easier to reuse because the principles themselves enforce common, basic rules. It’s not just that the pattern has been followed repeatably, but that the pattern itself is conducive to better, more cohesive design.

The SOLID principles are one such set of guiding principles, and we’ll cover all them here, but only use three of them, as they are pertinent the example above, along with their companion the DRY principle (don’t repeat yourself). As we cover them, we’ll point out how they will help improve the design of your code. For reference, the SOLID principles are:

  • Single-responsibility principle
  • Open-close principle
  • Liskov substitution principle
  • Interface segregation principle
  • Dependency-inversion principle

We’ll cover the DRY principle after covering the SOLID principles.

Single-responsibility principle

The SRP dictates that a function, or a class, or module should be confined to a single responsibility. This can also be restated as it should only have a single reason to change. As the scope increases from function to class, etc., the reason becomes principle looser, but should still be adhered to in principle. The responsibility of the function, or class, can often be, and in principle, should be inferred from the name. An additional helpful exercise is inspecting the code and asking all of the reasons it might change. If possible changes stray from its sole responsibility, then refactoring is likely needed. In general, less responsibility tends to be better, but only up to a point. Why you should follow the single-responsibility principle is that code becomes easier to reason about. If you know the code only does one thing, and one thing only, its name alone will tell you that reason (or at least it should). Multiple purposes either yields very long names, or general confusion that can only be resolved by careful inspection of the code.

In the example case above, the responsibility of our service layer function is to rebuild an object from data stored in the database. It needs to know about the database, and object construction. Referring back to our naïve solution, there are many reasons for it to change. If the database access patterns change, then the code must be changed in several places. The function also needs to change if the construction patterns of any of the objects changes. While these possible changes do align with the responsibility of the service layer, we’ll see below how we can do better, and simplify the responsibility even more.

Open-Close Principle

This principle states that a class should be open to extension but closed to modification. That is its reason to change should be configurable by leveraging design patterns. Rather than changing the behavior of the class by changing the code, runtime configuration or static extension via inheritance should yield the desired behavior. Following the principle is particularly important for robust code as every time the code of a function or class has to change, there is a possibility of introducing new bugs. The hard work put into ensuring sound, secure code is lost with every modification.

The example above violates this principle by requiring the if-then-else block to be manually updated every time a new object type is added. Additionally, if the build definition changes for an object, similarly the function needs to be modified to reflect the changes. Most of the design patterns in fact deal with methods to make classes follow the open-close principle, as we’ll see below.

Liskov substitution principle

Though not directly used in the current example, this principle is helpful to keep in mind when designing classes and inheritance schemes. The Liskov substitution principle is that the base class should be substitutable for the child class. In other words, strict polymorphism should be adhered to. In statically typed languages, variables that are typed to the base class should behave in an expected manner even if a child class is assigned to that variable. There should be no unexpected surprises in behavior as child classes are swapped in and out for the base class.

Interface Segregation Principle

Simply put, a class that implements an interface should not have to implement any extraneous features, namely functions. In this case, the dissimilar functions should be put into two separate interfaces, and the class should only implement one of the interfaces. This not only reduces boilerplate in the application, but also introduces conceptual clarity. By keeping that conceptual clarity, you prevent confusion that might arise from an expectation that an interface should be satisfied, when in fact it is not. The interface segregation principle is related to the Liskov substitution principle as adhering to it ensures that every object that satisfies an interface can be reasoned about identically.

This principle we will actually defy in our solution below since we have to accommodate two unique construction patterns, highlighting the need for pragmatism when solving a design problem. In most cases, strict adherence to all known software principles will be impossible, leaving the developer to carefully weigh the pros and cons of solutions that favor some principles over other principles. In this case, our original design goal is to have a single interface that can accommodate all possible ways objects can be rebuilt. So in essence, the interface segregation principle must necessarily be skirted to serve the broader goals.

Dependency-Inversion Principle

The dependency-inversion principle leads to designs favoring the use of interface and abstract classes as the direct dependencies of high-level code, and allow factories and other construction methods to supply the concrete classes for use. The term “dependency-inversion” refers to the fact that the arrow of dependency on the concrete classes has inverted from the perspective of the high-level class: the concrete class now depends on the details of the interface or abstract class, rather than the high-level class depending explicitly on the concrete class. This has a number of benefits, primarily polymorphism. It also de-couples design decisions from the details of low level classes, and enforces strict interfaces. All of this serves to make code loosely coupled, and easy to extend and maintain.

The naïve example above violated this principle by depending on all the concrete details of the construction functions. Not only did it depend on construction functions themselves, but it also had explicit knowledge of how the construction was done. Such explicit knowledge violates several of the SOLID principles, and should be scrupulously avoided. As we’ll see, implementing the dependency inversion principle will help isolate details, and will also serve to reinforce the Single-Responsibility Principle and the Open-Close Principle.

Don’t Repeat Yourself Principle

Though not a part of the SOLID principles, this one bares mentioning because it’s vital to basic software development. Code that is repeated in general should be generalized and abstracted away into functions, classes, and modules for re-use. This provides an ever growing arsenal of tools that gradually simplifies your life as a developer, and prevents copy-pasta: errors that creep into a code base due to mis-use and mis-renaming of code that has been copied over from another part. A good rule of thumb is that if you find yourself repeating a code pattern at least 3 times, it’s time to refactor and move the code to another place.

Sometimes though you should not follow this principle. This is particularly true if in generalizing a function to handle every single use case you intend to replace with it you need to tie yourself in knots to accomplish your purpose. Code that is abstracted away into common functions and classes should itself follow good design principles. Basically, if you need to start violating other design principles to accomplish not repeating yourself, then the refactoring should be carefully considered before undertaking.

In the above naïve example, however, we have two patterns we need to handle, and they are repeated multiple times each, so this is ripe for abstraction into a something more general. But it should be noted that this is not the primary reason to change. Here the single-responsibility principle and the open-close principle weigh much more heavily, so less repetition in the final solution is simply an added bonus that comes with satisfying these two other principles.

Design Patterns

The above 6 principles together are helpful to motivate a search for better design solutions in a growing code base that will make the code easier to maintain and extend, but they don’t necessarily in themselves tell you how to change the code. For that, one needs to turn to design patterns. Weighing pattern options pros and cons can be fruitful to laying out a refactoring plan. Here we’ll consider three patterns that we’ll use together to yield a more supple design. First, there are 7 distinct construction signatures we’d like to fit to a common interface. This suggests the adapter pattern. Second, even though we are rebuilding an object, it’s still construction, so the factory pattern should be considered, albeit with a slight twist. Next, the construction takes place in stages, not a single function call, so the builder pattern should also be considered. We’ll examine each of these in turn.

It should be noted that for each of these patterns, there is a more formal, rigorous explanation available in “Design Patterns: Elements of Reusable Object-Oriented Software” by Gamma et al. (also know as the Gang-of-Four book given it’s four authors and prominence it holds in the field), and the explanations given here are a subjective interpretation. While understanding the full complexity of the patterns as they were originally laid out is helpful, most often in your code you will implement an aspect or part of the pattern, not always in its original sense. This situation-specific interpretation is vital to hone as a software developer. But I refer the reader to the above book for deeper understanding.

Adapter Pattern

The adapter pattern fits one interface to another interface without changing functionality. As a simple example, many older Python libraries don’t natively implement context managers but rather provide an object with a close method. To take advantage of this, and allow such libraries to be used in modern Python syntax, the contextlib standard library provides the closing function which adapts the close method to the needs of the context manager. Using the closing function in a with context ensures that the object will be properly closed upon exit of the context. In this way we adapt one pattern (the close pattern here) to another pattern (the context manager pattern in this case) allowing the two to be used in concert without having to re-write either interface (this is also incidentally an example of the decorator pattern, we’re adapting and decorating at the same time).

Factory Pattern

The factory pattern produces an object built to the correct specification on demand. It abstracts the details of construction away from the caller, allowing factories to be swapped in and out as needed, and lowers coupling in the code. A common use case is building gui components. In cross-platform API’s the underlying code to build the gui can be different depending on the platform. Thus rather than calling the native API’s directly in application code, the construction of the objects can be delegated to a factory, with a common interface, and different factories depending on the platform. This way the application is agnostic to the details of the gui, and relies on the correct factory to produce platform-native components on demand.

Builder Pattern

Not all objects can be built in a single go. Sometimes there are optional arguments and optional steps, and in languages that require a static number of parameters to be passed to each function call, the only way to provide the necessary flexibility is to be allowed to call methods on the builder one after another, until the object is fully built. The builder pattern is particularly prevalent in the rust language given the language constraints. Consider writing a RESTful API. There are many things that need to be configured with it. End points, timeouts, caches, authentication methods, etc. Leveraging the builder pattern, you could rather provide a set of functions that add configuration options step-by-step until the full object is built, and the server can be started. Leveraged appropriately, the builder pattern can be more readable and easier to maintain than a massive init function. And, if you are in rust or a similar language, possibly the only way to go.

The Solution

Now that we have our three patterns laid out, let’s weave them together into a solution that will satisfy our design principles. First, we need to design an interface that will accommodate our two unique construction patterns. After that, we’ll leverage the adapter pattern to fit the constructors to that interface.

Interface

In designing the interface, let’s start with the builder pattern first. Our two construction patterns have a similar pattern. Here’s the pseudo code that encompasses both:

  1. Given builder start construction
  2. Loop over all unit keys in object
  3. Given unit key, fetch data from database
  4. Either (1) save data for later or (2) use data now in construction
  5. End loop
  6. Either (1) finalize the build or (2) do nothing, object is already built
  7. Return re-constructed object

Even though there are different options for what gets done in the loop and at the end, and one of those options is to do nothing, we have the makings of a common interface for all 7 objects. Let’s make three improvements here. First, we’ll use a factory to create the builder for a given object. Second, all steps in between when the builder is created and the object is returned should be handled by the builder for logical completeness. That means not looping over the keys in the object, but delegating that behavior to the builder. We’ll make the builder an iterator for this purpose. Third, because there is behavior that must occur at the end (finalize the build), this suggests the Python construct of a context manager. It provides a guarantee that no matter how the build is done in the middle of the context manager, the end will always run. This removes ambiguity and the possibility for errors in the coding process.

Here’s our final Python implementation of the interface:

Implementing the interface this way means that because this function does not repeat itself in ‘if’ statements, it is open to extension because the factory is expected to be extended and the interface is flexible enough that any construction pattern can be handled by it, and it is closed to modification because there’s no need to modify it to extend it. Furthermore, it has a single responsibility of connecting the database to the object re-construction, delegating details to the factory and the builder, and it inverts dependency because it depend only on abstract interfaces of the factory and builder, and not on implementation-specific details of low-level objects.

Factory

Turning our attention to the factory, how can we also construct a factory that is open to extension but closed to modification? Luckily looking back at our anti-example from the beginning, we see that in each stage, branching is chosen based on the object type. Another way to pattern this is to use a Python dictionary which allows for arbitrary association between keys and values. The factory has to return an initialized builder, so the internal dictionary should store the mapping between object type and builder class. The factory can be made extensible by providing a register method that will update the dictionary. Here it is:

We’ve chosen here to use the dunder method __call__ so that it has an interface like a factory function to keep the service layer code simple.

Builder

So we have an interface, and a factory, now we need to design the builder. Using an abstract base class aligns well with our need for an interface. In short, abstract base classes are Python’s way of stating that objects that inherit from it must implement certain details. Those details are marked with the “abstractmethod” decorator. Not implementing them will throw an error on class creation. Using an interface like this is where dependency inversion comes in. Rather than the service layer directly depending on implementation details, the concrete class depends on the abstract base class, thus the arrow of dependency gets inverted. Let’s implement the abstract base class for the above interface:

Here we made a design decision to prevent re-construction of an object that is already built. We further copy the object so that changes to the object doesn’t inadvertently influence things that might have references to the object already. Also, the code does a check on the number of times that the build step is called, and on whether or not the object is built. This is a type of contract enforcement to ensure that the builder is used properly, and that units are inadvertently skipped or missed, and was lacking in the naive solution above. This code also expects the object to have an interface. This motivates writing an abstract base class for the object as well:

Here we’ve leveraged properties in an abstract class to specify a contract for attributes on the class since its not possible to specify normally that a child class of an abstract base class should have certain attributes. Note that the “abstractmethod” decorator must go beneath, or inside, the “property” decorator. Read only properties are also a good way to enforce data protection in your classes.

Let’s look at what the concrete builders would look like for each of our two construction patterns in consideration. Remember, each class has to implement all of the abstract methods on the base class. First the pattern that leverages partial building on each unit of data:

Here we see that we just need to call the objects unit constructor in each pass of the build method. Next the concrete implementation of the pattern that leverages a finalize step:

In this case the build step collects the units internally to the builder, and stores them for use in the finalize step. As this process is common to all builders of this pattern, it would be helpful to move this up to a parent class (sub-class of the builder abstract base class) that does this to maximize code re-use.

Summary

In this example we have seen how to use design principles to guide the selection and leverage of design patterns to achieve extensible, modular design that will help applications scale quickly, and with fewer defects. In the final solution we uncovered an added bonus that the builder could be abstracted further to be re-used to allow construction not just from our database, but from filesystems and network io, at least with some creativity.

Now, the attentive reader will notice something: in the final solution there is much more code than in the naive solution. This will always be true. The quick, dirty way of programming will give you the least amount of code. Creating an enforcing interfaces produces more code. But the path of least resistance will always end up costing much more in the long run as an application becomes brittle and near-impossible to maintain. By separating out an application out into modular, loosely-coupled components, it is easier to maintain and extend. In our first solution, all of the code was in one place, but the construction code was likely duplicated elsewhere for other purposes, and the database access pattern would have been duplicated 7 times. Change in either would necessitate more refactoring. Additionally, it would have been harder to reason about what the common pattern was among elements, and impossible to enforce a standard as developers endlessly added their own twists to both database access patterns and object construction patterns, further complicating the process of maintaining and extending code. By enforcing an interface, it becomes easier to reason about what the application is doing and any given point in time, making debugging easier, and refactoring simpler.

On a final note, when solving design problems, it is helpful to keep in mind the final goal, and have a single overall objective. Here it was to keep the database access code in the service layer, and the explicit object construction details out of it. Next, work through the design principles and ask pointed questions about design ideas, and see if a more workable design can’t be had which is more generable and abstract. While the single responsibility principle drove our example, it was the open-close principle and the dependency-inversion principle helped guide to the final solution.

Finally, with an objective and principles in mind, consider design patterns that can be used to achieve your design goal. Design is an iterative process, so you will find yourself working backwards and forwards in the design process considering patterns, how the principles illuminate the proposed patterns, and hone away any excess from your solution. With a final solution in hand, turning to language specific constructs to further simplify an interface, and enforce design decisions can turn your code to something fluent and beautiful.


Designing to Patterns: A Pythonic Example was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.


Print Share Comment Cite Upload Translate
APA
Christopher Angell | Sciencx (2024-03-29T00:38:40+00:00) » Designing to Patterns: A Pythonic Example. Retrieved from https://www.scien.cx/2022/04/06/designing-to-patterns-a-pythonic-example/.
MLA
" » Designing to Patterns: A Pythonic Example." Christopher Angell | Sciencx - Wednesday April 6, 2022, https://www.scien.cx/2022/04/06/designing-to-patterns-a-pythonic-example/
HARVARD
Christopher Angell | Sciencx Wednesday April 6, 2022 » Designing to Patterns: A Pythonic Example., viewed 2024-03-29T00:38:40+00:00,<https://www.scien.cx/2022/04/06/designing-to-patterns-a-pythonic-example/>
VANCOUVER
Christopher Angell | Sciencx - » Designing to Patterns: A Pythonic Example. [Internet]. [Accessed 2024-03-29T00:38:40+00:00]. Available from: https://www.scien.cx/2022/04/06/designing-to-patterns-a-pythonic-example/
CHICAGO
" » Designing to Patterns: A Pythonic Example." Christopher Angell | Sciencx - Accessed 2024-03-29T00:38:40+00:00. https://www.scien.cx/2022/04/06/designing-to-patterns-a-pythonic-example/
IEEE
" » Designing to Patterns: A Pythonic Example." Christopher Angell | Sciencx [Online]. Available: https://www.scien.cx/2022/04/06/designing-to-patterns-a-pythonic-example/. [Accessed: 2024-03-29T00:38:40+00:00]
rf:citation
» Designing to Patterns: A Pythonic Example | Christopher Angell | Sciencx | https://www.scien.cx/2022/04/06/designing-to-patterns-a-pythonic-example/ | 2024-03-29T00:38:40+00:00
https://github.com/addpipe/simple-recorderjs-demo