Insight | Web Development

How to identify and manage technical debt

The term ‘technical debt’ was coined by Ward Cunningham in 1992. Technical debt is the result of a design or construction approach which will save you resources in the short term, but will result in larger costs over time.

The story goes, he was working on some financial software, and needed to get features out, but he was aware that he and his colleagues weren't 100% certain of how to structure the software. He wanted to release features in a working state, gain knowledge of the domain, then take additional time later to refactor once their mistakes became apparent. He made this analogy of debt as a way of justifying this slightly counterintuitive approach to his boss.

Short vs. long-term thinking

Developers want to do a good job, and may remember past projects which were debt burdened. They may know that if they cut corners now and get the job done quickly they will be stuck maintaining something buggy in the future. They may want to spend money rewriting things just so they can use the latest, exciting developments.

Business people need to balance long and short term goals, as they tend to want more with fewer resources. They may unknowingly reject cost saving measures because they can see the cost but not the benefit. They may also encourage constant corner cutting and short term thinking, causing development to slow down as the number of quick hacks and amount of unmanageable code increases. It’s easy to convince a non-technical project manager that something doesn't need testing, but much harder to convince them that something needs additional testing.

Intentional technical debt

Suppose you need a new vehicle because your current one is a write-off. The car that meets your needs is £6,000.

You need that vehicle ASAP, but you don't have that capital available. This is the kind of situation where one would intentionally take on a debt. You will end up paying more than £6,000 in total because of interest, but you are making a conscious decision to use your cash flow rather than capital to solve the problem. This is comparable to implementing a piece of software to meet a deadline by a certain date. Perhaps there’s a promotion coming up and you need to be ready to go the day it starts. If you cut corners with testing, you'll find yourself spending money debugging and fixing things later, like the interest on the loan. A bug in production is a lot more expensive to fix than a bug in development.

Or you could buy a cheap stop-gap measure, a vehicle which doesn't meet all your requirements but is good enough to survive. This is analogous to releasing your software with the bare minimum requirements, and leaving some features until later. This can result in increased costs if the original developers move to a new project and you need to bring in a new person, or opportunity cost if the work of implementing those features gets in the way of the next release.

Accidental technical debt

You need new tyres – you buy them second hand from your mate Dave and save money. The "new" tyres wear down quicker. There is also the chance that you get a blow-out and have to write off your car. Now you’re repaying that debt.

This can apply to software in cases where you leave defects in your product, either through corner cutting, rushing, or naivety. Eventually, maybe months later, those defects will result in a client facing error - and cost you in development time or lost profits.

Emergency fixes as intentional technical debt

Picture the scenario: it's 5pm and the client suddenly calls. They need to change one of their templates, and quickly. You could follow the usual process - change the template on your development version, test it, then release it to live, but this would take 30 mins to complete and require the attention of somebody who is not immediately available. So, you decide to cut a corner - change the template yourself, directly on the live site. What you've done here is to create a debt burden - your developers are going to find their copy is out of date and this might break the quality control. If you use the debt analogy, you can identify what is required to pay off the debt, probably this just means doing the 30 mins process you would have done anyway the next day, in the morning.

The surprises of unintentional technical debt

Imagine a greetings card site where third party suppliers use an API to add new images and card formats. This API includes a field for the date from which this design should become available. Imagine that the original developer saved themselves some time, by skipping on validation on date formats - they've saved money in the short term. The field is only used internally, so if it contains dodgy dates, it’s no problem - the client manually adds designs after vetting them.

Now skip 6 months ahead - the client asks you to add a new feature, where designs from trusted suppliers are automatically made available, based on the date field, but you have a problem. You have mixed up US and UK date formats. Nobody knows what day “3/7/16” refers to. One supplier puts things like “Christmas Eve” and “my cat’s birthday”. Now, not only do you have to do what you should have done from the start (validate the date field), but you have to fix bad data - that extra time represents a debt burden you weren't expecting. It might translate into opportunity cost, or it might just mean somebody has to manually unpick a few thousand dodgy values.

The moral hazards of technical debt

Imagine you are fitting locks into a building. You bill the client for a very secure lock, but most people don't know what is inside a lock... so after billing for top-end, durable deadlock, you fit something not as secure. This could be actively immoral to increase a profit margin, or you may have told the customer that you'll fit it today when you don't actually have the lock they wanted in stock. You can probably get away with it - the client doesn't know what the difference, right?

However, the value of the client's items that get stolen two years later amount to more than the mere difference between two locks. In terms of software, this can translate into badly implemented or fake security measures - for example, your client wants a login screen with username, password, and a PIN code - but the pin code you implement is just for show. Perhaps it’s used for something else, and everybody has the same pin.

Non-coding causes of unintentional technical debt

Imagine that the main developer on a project is going on holiday - somebody else will be filling in for the short term. You could have a brief handover meeting, but meetings are boring. Now imagine that only the main developer knows how new features are released to the live site. This is a ticking time bomb. Eventually, the fill-in developer tries to release something - and breaks it. On Friday afternoon. When everybody else is at the pub. And they have to stay back and unbreak the site for 1.5 hours.

In environments where developers are rewarded for their turnaround time instead of the quality of their work, as personally experienced in previous settings, a culture of corner cutting can develop. Many developers, when they find their performance being compared unfavourably to a developer who is cutting corners and accruing debts, decide to cut corners themselves to compete. This demonstrates the effect that company culture can have on the quality of work that employees produce. Instead of cutting corners, invest the appropriate amount of time and do things properly; this is why it is so important for agencies to be committed to a 'quality first' philosophy. Any performance metric must include some way of taking qualitative measures; some things are subjective and can't be displayed in an end of month spreadsheet. Examples of this are outlined below.

Some ways to avoid debt

  • Spending money to save money - code analysis tools (jshint, Resharper, etc.) cost time and/or money up front to set up and encourage people to use, but it can save you money in the long-term and reduce your debt burden.
  • Spending time developing a quality control process - this will pay dividends, but only once you've spent the money up front to train people and iron out the wrinkles where it doesn't quite work for older products that you maintain.
  • Code reviews - some developers get defensive about code reviews, or think they are so awesome they don’t need one. If you’re that good, the review will be over quickly and not yield any further tasks.
  • Beware of what I refer to as "Boss Code" - code written by somebody who does not answer to any other technical person. Boss code can stick around for a long time, with nobody daring to change it for fear of upsetting The Boss.

Management which comes under constant pressure from targets often penalises mid-term and long-term thinking, or cost saving measures which must be paid for up front - especially if nobody can justify it to them in words they can understand. Use the debt analogy and present it to them as a choice - choose between: 1) cheap now, expensive later, higher total costs 2) expensive now, cheap later, lower total costs.

Our industry has lots of intangibles; some elements of software quality or developer performance are subjective. Developers may want to spend extra time implementing something to ensure they don’t get bogged down later, but might go too far. If you need to strike a balance or rationalise a shortcut, use the debt analogy in your negotiations.