Posted on March 5, 2016
by Doug Klugh
The manner in which technical debt is managed can give a company a competitive advantage when it's most needed or destroy a well conceived product. When incurred properly, technical debt can greatly decrease time to market, providing a short-term advantage over the competition. But if that debt is not paid off quickly, it will slowly, yet surely, erode the quality and the value of your software. If it is allowed to linger within your code, that short-term advantage will quickly turn into a liability.
What is Technical Debt?
Technical debt is a deficiency in the expected level of quality for individual artifacts or artifacts integrated as a whole. These artifacts may include business or domain models, requirements, designs, code, tests, or infrastructure. Or as noted below, it may also include a lack of knowledge or understanding.While there are many ways to incur technical debt, one of the more common ways is by simply writing bad code (also known as cruft), either intentionally or not. A developer may deliberately write code quickly to meet a tight deadline, but in the process, may, for example, create code that is tightly coupled, making it difficult to maintain, extend, and test. There is usually a price to be paid for writing code quickly, and that "price" is technical debt.
Tech Debt can be incurred through cruft, a poorly conceived architecture, or low test coverage. Each of these needs to be refactored (or remediated) to provide a good, high-quality product. And in this case, high quality refers to more than just the lack of bugs (see Quantifying Software Quality). Although, not all technical debt has a large impact. If it was incurred within code that is now dead, it may not have any impact at all (although dead code is usually considered to be tech debt itself). One of the big challenges of managing technical debt is in understanding the impact. When new functionality or a critical bug fix needs to be released quickly, technical debt can significantly delay that release. This could lead to missed market opportunities or the inability to simply keep customers happy.
An Analogy to Financial Debt
The price of developing cruft is usually explained best as an analogy to financial debt; which often helps in communicating with non-technical people. Just as someone may take out a loan to buy something quicker than they may otherwise be able, developers may take out a loan against the technical collateral built up within their code. They may quickly code a solution and get it to their customers very fast, but in doing so, they establish (or perhaps extend) the debt incurred in their code. Eventually, they will need to refactor that code. And until they do, that code will impact the development of new features. For example, if the Open/Closed Principle (OCP) was ignored and the code ended up being difficult to extend, it may slow down future development of features or enhancements, or may promote writing more bad code in order to keep pace with the team’s regular velocity. This is the interest that accumulates on that debt. And until the debt is removed (or fixed), the interest will continue to accumulate and interest payments (workarounds) will be required on a regular basis. Principal payments (fixing the bad code) is the only thing that will eliminate the debt. As with financial debt, the best thing is to remove that debt as quickly as possible. Otherwise, buying (developing) a new feature or bug fix will end up costing a lot more than it should, due to compounding interest. Taking shortcuts and ignoring basic engineering principles may (sometimes) provide a quick, short-term gain, but it will cost more in the long run. Even if that debt is eliminated soon after incurring it, it will take longer to fix it than it would to do it right the first time. And if that debt is allowed to remain, the cost will rise very quickly. Always consider the long-term impact when incurring technical debt. Make sure the short-term gain is worth all those interest payments (workarounds) that will need to be made in the future.Types of Technical Debt
Technical Debt is work that can be done to raise the quality of a product. To help explain this, I have split different types of technical debt into the following five categories. These categories apply to all types of software development, regardless of platform, language, development approach, or process.As Bob Martin often says, a good architecture delays decisions for as long as possible. For example, in the early stages of architectural design, there should be no decisions on what platform the software will run, or even on what type of platform it will run. A good architecture allows for flexibility and changing requirements. On the other hand, an architecture that has missing layers, does not easily facilitate cross-cutting concerns, or one that simply lacks extension points, adds to architectural debt. If it is difficult to add new features to the existing architecture, or extend it properly, then technical debt is most likely being added with every change.
Code Debt includes numerous aspect of how code is written. Code that is difficult to understand, hard to maintain, inextensible, insecure, unstable, or difficult to test contributes to code debt. Following well-established engineering practices, such as applying DRY and SOLID design principles, will go a long way in ensuring high quality, well written code.
To defend code from poor quality, tests need to be written quickly and easily. Because the longer it takes to create and run tests, the longer it will take to discover bugs, and the greater the potential impact. And tests must be though. Ideally, every solution would have 100% code coverage, robust test data, automated unit, functional, and integration tests, and test environments that resemble real-world user environments. But all too often, there is nowhere near 100% code coverage, test data does not cover less common use cases (or even edge cases), too many tests have to be run manually, and test environments don’t represent the users’ configurations. All of these deficiencies constitute test debt.
Test debt will slow down development, either by taking too long to write and execute tests, or by having to spend too much time managing and fixing bugs discovered by customers. It will also inhibit refactoring. Developers often avoid refactoring fragile or complex code for fear of breaking it, especially when there is a lack of automated tests guarding that code. If manual tests are used to validate the refactored code, it may take too long to refactor; or worse, get refactored without testing the changes, further contributing to code debt.
Knowledge Debt occurs as a result of software artifacts that are not well understood. Team members may have difficulty figuring out what a segment of code does, how it works, or perhaps who is even using it. This occurs more frequently in organizations with high turnover. Once the authors leave the company, there is often no one with first-hand knowledge who can answer questions and provide explanations of the artifact(s).
Sometimes it is not apparent that dead code remains in the code base. Developers can waste a significant amount of time trying to understanding code that will never run. And depending on changes to dependent code, dead code could end up breaking the build or perhaps failing automated tests.
Risks associated with knowledge debt can be mitigated largely through documentation that is actively updated on a regular basis. Documentation can quickly become obsolete and older documents are not always trusted to reflect the current state of a system.
Emerging Debt occurs naturally over time as advancements in tools and frameworks replace older technologies. This becomes a problem as new solutions are added to older platforms, or even as old solutions are added to newer platforms. For example, this becomes a problem when features that depend largely on Flash or Silverlight components greatly inhibit extending those applications for use on mobile devices. Telling customers to hold off on upgrading their operating systems or web browsers for fear of compatibility issues is a sure sign of emerging debt. Continuous refactoring will help to reduce, or even eliminate, emerging debt.
Reasons For Technical Debt
In 2009, Martin Fowler presented the Technical Debt Quadrant to further explain why technical debt is incurred. This ranged from Reckless to Prudent debt on the horizontal axis, and from Inadvertent to Deliberate debt on the vertical axis. For example, reckless/inadvertent debt may result from a team that is ignorant of software engineering principles and therefore makes no decision on their use. Reckless/Deliberate debt would result from a team that knows and practices good engineering principles, but decides not to follow them because they think they can’t afford the time. Prudent/Deliberate debt may result from a team making a conscious decision to take short cuts in the code in order to ship features faster to deliver ahead of the competition or to meet short-term customer demand. And prudent/inadvertent debt would result from a team that knows how to write good code and intends to do so, but realizes a better solution near completion of the project. The latter scenario occurs often, even with mature teams.
«Prudent debt is essentially clean code resulting from professional behavior. It’s relatively easy to pay back and can be a distinct advantage to the organization. Reckless debt is poorly implemented, or poorly architected code, that’s the result of unprofessional behavior. It is a long-term detriment to the organization and is very difficult to pay back.»
Quantifying Technical Debt
To maintain high quality and high value of software products, the amount of technical debt must be well understood. Development teams must actively record debt as it is discovered through normal development activities and especially as it is incurred. From my experience, this is best maintained in the team’s ALM tool, either as Technical Debt cards or noted in a document or list of technical debt. If the team follows Agile practices, the team should consider sizing these tech debt cards and having the Product Owner prioritize them in the backlog. The team should regularly make its case as to why the debt should be eliminated within a particular schedule and the value to be gained by removing that debt (just like a user story). Of course it helps to have a Product Owner who truly understands the impact of technical debt. Different metrics can also help to quantify technical debt. Most metrics are used to define trends that serve as warning flags to potential problems that either define tech debt or indicate the likelihood of debt within given artifacts. These metrics can be attained through static code analysis integrated with IDE and CI tools, source control metrics, test metrics, and ALM metrics. My article on Quantifying Software Quality covers these metrics in detail.Conclusion
Incurring technical debt can, at times, help to meet short-term goals. But more times than not, it’s something to be avoided. The longer it festers within your software, the more it’s going to cost to enhance, extend, and validate that software. Mature development teams avoid technical debt, unless they can show a clear benefit to incurring that debt that outweighs the “interest payments” they will have to make on that debt. And they do not incur debt simply because they “don’t have time to do it the right way.” There is always a tradeoff, and the compromise must be well understood and communicated to project stakeholders (if you’re following any type of empirical process framework, such as Agile). To successfully manage technical debt, you must track all debt, have a good understanding of the extent of debt in your software artifacts, and understand the short-term and long-term impact for each debt item.Dirty code is to technical debt as the pawn broker is to financial debt. Don't think you are ever going to get your code back.
— Ward Cunningham (@WardCunningham) September 3, 2009
Doug Klugh
Doug is an experienced software development leader, engineer, and craftsman having delivered consumer and enterprise firmware/software solutions servicing more than one billion users through 20+ years of leadership.
