Why we care about tech debt
The term “tech debt” gets thrown around in software projects a lot. Engineers blame it for the large number of bugs and issues. Managers mention it as a reason for delays. But for those that do not work on the product, this can sometimes be difficult to empathize with. Because from the outside, an app that’s internally a cobbled-together mess of spaghetti code can still look and feel quite acceptable to a user.
However, tech debt will be primarily felt when new features need to be added or when bugs need fixing. Then a poor software state (e.g. outdated dependencies, poor architectural design, lack of documentation, absence of test suite) can make introducing changes and testing them very painful.
That is why tech debt needs to be taken care of regularly. Just like your house requires cleaning over and over again, until the end of time.
But how does tech debt accumulate in the first place? If everyone knows that tech debt is bad, why does it invade every software project?
The most common answer you will hear is that a certain feature was rushed because “business needed it”. The deadline meant that there was not enough time to complete all tasks that are typically needed for high-quality software. Yes, this is one possible cause. But in my experience, this is not the most frequent reason.
Thing is, it is quite difficult to completely avoid building up tech debt when you add new features. Because every added feature increases software complexity. And unless that complexity is immediately and sufficiently handled (through adapted architecture, testing, and documentation, among others) it compounds into tech debt.
Only in the most ideal case, where you have senior engineers with experience in the domain and with plenty of time, then you might have the chance to prevent tech debt to be produced at all. But this is a fantasy for most. Because in a much more realistic scenario, your team is learning on the go, and deliverables are expected early by the business.
So your team gives their best attempt at making something good enough in the given time. If it’s a fire-and-forget kind of project then you will probably not touch it again unless there are bugs. But if the project needs to be extended, then suboptimal code that was written before could be impeding progress. And then it’s time to refactor.
Because that is often the goal of refactoring: simplifying or streamlining the codebase so that feature changes are easier to make.
It is a fundamental law of software engineering. New features increase complexity (also called “software entropy”). Complexity eventually turns into tech debt. Tech debt slows down feature development. But then refactoring reduces complexity. This makes adding new features easier again, and the cycle continues…
So if you think about it, increasing software entropy is just a natural consequence of evolving software.
Tech debt should not be regarded just negatively. First of all, it simply means that maintenance has not kept up with the passing of time. In a world where everything digital moves lightning-fast your dependencies keep getting updates, new frameworks are conceived almost daily, and best practices are not set in stone yet. This means that without consistent effort to keep it up-to-date, all software turns into tech debt eventually.
But secondly, it also shows that you and your team have learned from your mistakes. And you now should know how it can be done better. Issues have revealed themselves in production and you know which parts of the code are most flaky. This helps to focus on the parts of the code that can benefit most from refactoring.
For example, if you know that a certain component has suffered from many bugs lately due to its complexity, then it is time for a refactor. Or if an old package version is causing poor performance, then you could consider updating or replacing it.
I also find myself refactoring after I understand the problem better. Especially in cases where my team does not have so much experience in the domain yet, the first version of whatever we are building is not so good. And once you’ve launched your first version, you might already have ideas for how to improve the design due to your advanced understanding of the problem.
Sometimes this instant happens later. After you have thought about the problem more or learned about other solutions, you identify better ways to do it.
Even a partial or complete rewrite can be worth the effort if it makes the code simpler and thus less error-prone. This makes knowledge sharing easier and aids in future development.
It is no coincidence that all successful software is at least on version 2 and usually much ahead. For example C17, Java 17, Python 3, Chrome 98, Firefox 96, Microsoft Office 2016, Microsoft Windows 11, and Apple macOS 12.1 (just to name a few well-known ones, I’m not saying these are good software products). The first version of a program is generally not very good.
It takes many iterations to get a good piece of software. Countless hours of feature development, together with regular grooming of the codebase to keep the code quality in check.
So don’t fret about tech debt. Acknowledge that it will crop up, even with the best proficiency and intentions.
Your job is to keep track of it, assess its impact and determine if it is worth tackling. Because in some cases it is okay to let software rot, if that allows more time for higher-impact tasks. You can always come back to it. Just make sure everyone is aligned on this decision as well. What is a sufficient level of software quality is a subjective matter. So be prepared for some friction.
But remember these discussions are worth it. Great software is the product of many minds, each with its own idea of how it needs to be. Only by talking and collaborating frequently with each other, can programs be improved over time and reach excellence.
