This content originally appeared on Level Up Coding - Medium and was authored by Thomaz Moura
Cargo Cult Driven Development has become the norm, it’s only the rituals that change.
It’s 2023. People have free and immediate access to a wealth of information that would make any encyclopedia jealous at their fingertips. And still, tech people are still trying to find the magical silver bullet that will solve all our problems just by existing. And there are just as many people announcing new and fancy silver bullets as there are people looking for them.
Be it Agile, Microservices, DevOps, Blockchains, or AI we are currently looking for that magic thing that when we start using will make all our problems go away. Like we are constantly just missing one tiny missing piece that when we find it will just make the whole puzzle look easy and we will look behind and think “Ha, looking back, it’s a wonder we took so long to figure it out, huh? All this time we just had to do this”
Now to be fair I do think that hyped techniques and technologies often have something to offer and are sometimes a really welcome and needed improvement. The problem I’ve seen repeated over and over again is more related to how and why the hype is being followed — people just jump onto the bandwagon because “this is the future” without ever considering what gotchas this future is holding for them.
The reasoning behind this article
I’m writing this because I’m tired of seeing news of the tech world showing how they “are actually moving back to the old ways because the new ones don’t work”. A recent one I will be talking about here is the article about how a Prime Video team managed to reduce 90% of its costs by moving from microservices to a monolith application. I will also explore how we can see this same behavior in the TDD culture. And to conclude I will give my opinion on how to avoid this.
But first, let me explain the subtitle.
What is the Cargo Cult and its relation with tech?
In case you are not familiar with the term Cargo Cult (or its origin) it’s based on the story of a small population of indigenous people that during World War II observed many military shipments being delivered to the troops by airdropping. The soldiers would sometimes share some of these goods with them in exchange for them working as guides for the region. The problem is that after the war the military abandoned those airbases and stopped dropping cargo.
Since those people have gotten used to the outsiders and their technology they missed it soon some leaders arose promising to bring back the good times. And to do it they started mimicking the soldiers, imitating their dress styles and day-to-day habits, and some even built life-size replicas of airplanes out of straw. They believed that the outsiders had some connection to the ancestors because only they would be able to provide such riches and hoped that by doing the same “rituals” that the soldiers did they too could receive those gifts.
The key point here is that those indigenous people did not understand what was behind those cargo shipments that they were seeing and tried to replicate the same results by mimicking what they saw and believed was the cause of the result they were expecting. They did not realize that the cargo was being dropped to the soldiers and because the soldiers were there (and it was being delivered by the same people who took them there to begin with) so they had no reason to keep dropping it after the soldiers were gone.
This kind of mentality is unfortunately pretty common in tech too. Often people discover some new tech that is currently hyped and start assuming that the results people that have adopted them had are a direct result of the tech being used in complete disregard of the context in which it has been used. This leads to beliefs like “Agile makes your team deliver faster” “Cloud is cheaper than on-premises” “Serverless functions are cheaper than other cloud approaches” and “Unit Tests make your code safer and easier to maintain” or even “NeoVim is better than VS Code”. There is some truth to these beliefs, but only in certain contexts and often with tradeoffs.
Let’s start demystifying some of these beliefs starting with the Prime Video case.
The Prime Video shocking case and the Distributed Fever Syndrome
After reading the article (link at the end) I have to admit that it surprised me. Greatly even. Not because of the part where they decided to take all those serverless components and consolidate them on a single application but because they actually decided to make it completely distributed to start with. One sentence in particular that left me confused was this:
Some decisions we’ve taken are not obvious but they resulted in significant improvements. For example, we replicated a computationally expensive media conversion process and placed it closer to the detectors.
I understand that replicating computationally expensive operations should always be a conscious decision after carefully considering the other costs, but the way it’s said here (and the initial approach that was taken with the serverless components) seems to imply that the only “obvious” expense we should consider is the computational cost. We know CPU is expensive, but shouldn’t it be obvious that memory and networking are expensive too?
I’m not judging the team here, they did realize the initial approach didn’t work and adapted to it — and shared the gained wisdom with everyone which I think is awesome. We all make mistakes from time to time and identifying, fixing, and learning with them is what makes us good engineers and that’s what they did.
I’m just surprised that apparently, the “default” approach was to make it as distributed as possible from the beginning. And it’s not like this kind of thinking is specific to this team — in fact I’ve seen many different stories of people taking the idea of distributed systems too far. One particular case I found amusing was someone defending the use of monorepos because it “made it easier to deal with microservices, especially on cases where the only thing a microservice does is persist the data of another microservice”.
I just laugh inside when I remember that. It’s not that the idea of keeping two systems like that together on the same repository is funny, but the very idea of having a separate system whose only job is to persist data of the other system. I mean, can you even consider it a microservice when it doesn’t even make sense to exist when in isolation?
I believe both these situations are symptoms of what I call Distributed Fever Syndrome, which is a carefully chosen and crafted name I’ve just come up with on the spot and which can be clearly demonstrated by this image:
The truth is that making a system distributed will usually add overhead. At the very least both systems will have to serialize messages, send them over the network and deserialize them on the other system. And there are many other questions that have to be considered as well, such as the minimum size of an application — depending on the technology used a module inside an application might add less than 10MB of memory to the existing application while starting another process just to run the new feature could take dozens or hundreds of MB.
But depending on the application size and the challenges of maintaining it as a single unit (especially when there are different teams working on different parts of the system) the cost of distributing the system easily pays off. The discussion here should not be if it’s better to have a monolith or a microservice, but what approach fits better this specific system maintenance needs?
The pattern behind all this
There are many other situations where I’ve seen this kind of problem arise and they often seem to have the same root cause: a valid concept or idea that starts being sold as mantra-like truth that has to be followed with zeal.
Take testing for example. I’ve seen many people fiercely defend that you should always make automated tests for every code you write and that’s the only way to be able to trust the code. I’ve also seen people fiercely fight back against the idea of dedicating time to tests because they think that all that tests do is slow down the developers. Often both these people have experiences they will happily share to reinforce their opinion, so it’s hard to say any of them are wrong.
But in my opinion, the problem that should be discussed is not whether or not to test, but when and how to test. Good tests can make wonders for the maintenance of a system, especially when it comes to regression. Bad tests on the other side can make maintenance really challenging and even make the system less stable by giving a false sense of stability. And sometimes the difference between a good test and a bad test is just what is being tested.
To me, the best example of testing being taken too far is the culture of defending that we should be striving for the biggest test coverage possible and that we should always prefer unit testing over other types, and that unit tests should mock absolutely anything that is not the SUT (System Under Test, which usually is a class that is being tested). The ironic part of that is this is a common approach for many TDD developers, while Kent Beck (the creator of TDD) himself seems to have a much more sane and wise approach (From his answer on Stack Overflow of How deep are your unit tests):
I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.
Kent Beck and Martin Fowler have also already said publicly that often take a “classic TDD” approach instead of “mockist TDD” where they usually only use a mock on a dependency when it really feels awkward or unpractical to use the real class (such as classes that depend on the network or external dependencies run).
So in the end, even the pioneers of test-driven development realize that tests have to be meaningful to bring value. And in my opinion, by focusing on things such as code coverage more than the quality of the tests you’re writing you end up much less likely to get the benefits of testing at all — it might slow you down more than actually help you.
How to break the cycle
So, by now you’ve probably remembered other situations where you’ve seen Cargo Cult Driven Development in action — heck, you might even realize you’ve done it in the past or are doing it right now. Don’t worry, we’ve all been there. But what can we do it change it? Should we just fight back on any new hype that appears to avoid the temptation of jumping on the bandwagon without knowing what we’re doing?
I don’t think we have to go that far. The truth is that usually there’s a reason why things go popular. We should embrace those new ideas and try to understand how they work and what makes them special before we decide between investing in them like the next big thing or taking them down like a pointless silly idea. The important thing to always keep in mind is this:
When we’re talking about technology, it’s always a matter of tradeoffs.
Rarely something will be so bad that there’s no use case for it or so good that you should actually use it for everything. That’s why whenever I’m learning new things I strive to learn not only its pros and advantages but also its cons and gotchas. I find it incredibly useful to learn about some technology or practices shortcomings before adopting them heavily because it does wonders to avoid the common pitfalls around them. I find it’s much easier to use something effectively when you know its limitations.
It’s much easier to reduce the costs of moving to the cloud when you know the many pitfalls that could leave you with huge unexpected bills and how to avoid them. It’s much easier (and faster) to develop on a codebase that has well-thought tests that focus on the relevant behaviors of the code, instead of trying to test everything for the sake of having tests. It’s much faster and more productive to use NeoVim for development than any other editor and so on.
Ok, I’ll admit that the last one might have been a little bit biased, but you get the point.
Let’s embrace hypes as interesting new ideas, but let’s stop taking them out of context.
Links
Waiting for John Frum: Cargo Cult of the South Pacific | When God is An American Soldier Documentary
Kent Beck answer on Stack Overflow of How deep are your unit tests
As Software Engineers, We Should Know Better By Now was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Thomaz Moura
Thomaz Moura | Sciencx (2023-06-07T01:28:34+00:00) As Software Engineers, We Should Know Better By Now. Retrieved from https://www.scien.cx/2023/06/07/as-software-engineers-we-should-know-better-by-now/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.