Warning this blog post is a bit of a rant, if you want something more technical there's plenty of other articles on this blog and many others!
Technical debt is a concept used in software engineering to express the additional complexity that is added to a project due to technical decisions that result in inferior solutions being chosen because they can be delivered quicker. It is analogous to financial debt. Borrow money for some gain, incur debt but -and there's always abut - you have to pay the debt off. Otherwise, with interest, the debt will grow and you will eventually be bankrupt. Similarly, it is ok to incur some technical debt. If you spend too long looking for the perfect solution your customers will have moved over to someone else. At the same time, if you incur too much technical debt and you are not meeting your repayments, just as the interest on the financial debt will compound repayments, if you don't meet your technical debt repyaments the complexity and entropy increases with time and eventually it will stagnate your product. Something that should really only take one month to deliver, suddenly takes 3 months, then 6 months, then there's only a few people in the entire company who can do it in 6 months, then a few of them leave, then...
Where the analogy breaks down is that all too often technical debt is also used to express that there has just been some bad engineering. Bad engineering decisions are in a different category to ones that were tactically made with full knowledge that the short-term priority was worth it. When it's clear that such a decision was, in fact, a tactical decision, it is much easier to convince people that refactoring needs to happen and the debt has to be paid off. Unfortunately, when the term is used as a polite way of saying bad engineering, it's unlikely there is any repayment strategy in place and it is even harder to create one because first, you need to convince people there is some bad engineering, then you need to convince people it is causing problems, then you have to have the ability to think of a better approach, cost it and then convince the various stakeholders the investment is worth it. It is like trying to win 5 matches in a row away from home when the odds are against you.
So, what are the signs that your technical debt (irrespective of whether it is intentional or accidental) is just too high and you could be filing for bankruptcy soon?
Well, think about this. In the Agile world, we want to develop, build, release in quick cycles and get rapid feedback from the customer and go again. This is only possible if there is an abundance of high quality and well engineered automated tests that run fast and provide confidence that a change no matter what it is has not broken anything. It doesn't matter if the breakdown of the tests are: 68% unit tests and 32% are integration tests or 91% are unit tests and 9% integration tests, the tests have to run fast and they have to provide confidence to all stakeholders. Otherwise releasing will be a pain and it will not be possible to release regularly. That means being Agile and getting all the benefits of it will be very difficult — no matter how good your backlog grooming sessions are.
What usually makes it difficult for developers to write good tests? Well, it's usually technical debt. It doesn't matter if it is intentional or accidental.
Now, there are all sorts of tools that will measure technical debt and put it up on a nice looking SonarQube board but usually these tools only pick up the trivial stuff — removing an unused import etc. Who cares? Such trivial stuff isn't going to slow anyone down. The real technical debt problems are the ones that slow people down, they make it harder to make changes, fix bugs, add functionality and do it all quickly with confidence - why because they have made things much harder to test. Sadly, these are technical problems are usually not just something an IDE, a PMD or Checkstyle will prompt you to do. They are generally much deeper in nature - towards the architectural end of specturm. Some examples:
- Lack of data encapsulation and immutability leading to huge cyclomatic complexity, unpredictable code paths and difficulty in predicting impacts.
- Impedence mismatches
- Lack of modularity and too much coupling
- Lack of or bad application of patterns
- A proprietary language introduced with no IDE support, no mechanism to easily unit test or debug code. Zero support from Stackoverflow.
- Spawning of threads and asynchronous call paths when are better approaches which would be much easier to test
This is where the analogy of technical debt and financial debt breaks down. When you have architectural debt you have big problems. When you have a HashMap used when an ArrayList would have made more sense, you don't. Financial debt doesn't have such a critical distinction. Unless we say, the debt is due to a friendly and sympathetic bank you have a good relationship with or its debt due to some lunatic load shark who will call around to your house with a baseball bat.
So, if you realize you are approaching your credit limit, what do you do? Firstly, you need to get confidence in your tests. They need to test key functionality and they need to be maintainable. That is more important than speed. If you can't get confidence, you can't ship. There is not much use with tests that run in 5 minutes if no-one has any confidence the functionality you deliver will actually work. Secondly, once you have confidence in the tests (even if they are ugly end-to-end tests), you need to get them to run fast. This is where you can start refactoring — the end to end tests should facilitate refactoring of parts of the call paths; there may be obvious sprouts for example and this should help to move towards a classical test pyramid.
Thirdly, you now need to understand why the code is so difficult to achieve high-quality tests. Too much coupling, bad exception handling, bad decompositions it's probably a long list. Understanding why your code is difficult to test is a key architectural and engineering skill as it requires the ability to not just understand the complexity but the ability to be able to know how to reduce it. Reducing the debt should then provide a pathway to make the code easier to test and thus make good tests run fast. Achieving this means you are winning the battle against eventual project stagnation.
So, the last part of the rant. There is a growing problem with the application of Agile which means we end very easily end up with faux Agile. The various Agile books, courses, blogs will detail things like story points, burn downs, stand ups. That stuff is all good but there is not enough focus on technical excellence. Without the technical excellence you get inevitable architectural problems and code that is difficult to test irrespective of your best efforts at story pointing, backlog grooming, sticking yellow post-its up on walls, having arguments over the definition of done, doing your show and tells. All that stuff is nice and beneficial but in comparison to technical excellence it is almost superficial. It is something people can easily see but on its own it never captures the complexity of technical debt. It is the ratio of technical debt to technical excellence that determines whether you can write testable code easily and thus be able to deliver in regular short iteration which is the goal of Agile. Isn't it?
Lastly, something we all say to our kids when they try any sport: "If you are losing kido, never give up, play to the end of the match".
Technical Debt is natural, it happens in every project in the world. Your job is to keep it in check and when it gets too high to be able innovate your way work around it and eventually lesson it. That's such an essential characteristic of a good Architect, it is also a handy interview question.
"Mr. Candidate, describe some Technical Debt you have experienced, describe the effects of it and what strategies you put in place to either work around it or deal with it?"
Or even,
"Mr. Candidate here are some examples of Technical Debt, how would you prioritise which ones to deal with? Could you describe some better solutions and how would you convince people they are worth it?"
The quality of detail in the answer you get back is very likely to indicate the level of pragmatism, experience and the innovation skills of the Architect / Engineer.
Until the next take care of yourselves.
No comments:
Post a Comment