5 Steps to Avoid a Technical Debt Black Hole

Several years ago, I found myself owning one of those classic unpopular business software systems. The (fairly substantial) team delivered one or two production releases each year, with a backlog of ageing change requests several releases deep(technical debt), and the business users were infuriated by the minimum 14-month lead time on changes.

When I dug into the underlying reasons, there were some typical software development symptoms: experienced analysts described the system as a ‘big ball of mud’, new developers complained about spaghetti code, every change broke something unexpected, and each release was coaxed through testing with last minute duct-tape-and-bailing-wire fixes. The scared development team had adopted a terrified ‘don’t change anything unless we absolutely have to’ approach.

In other words: considerable energy was put into the software, but sucked into a black hole of tech debt.

Technical debt is a phrase that’s chucked about a lot. Simply put, it is the ongoing cost of expedient development decisions made when implementing code. It’s all those shortcuts or workarounds in technical decisions that give short-term benefit in earlier software release and faster time-to-market.

Used carefully, a little technical debt is not a bad thing – it speeds software development towards a short-term goal, and sometimes that deadline is the most important thing. The key, as with most forms of debt, is to pay it back promptly, before managing the interest brings you to bankruptcy. If we are to look at this in the context of Salesforce, technical debt will typically mean any of these things:

• Incoherent configurations: different styles of approvals for similar business processes, or multiple email templates for essentially the same communication
• Excessive customisations: writing a trigger when a Process Builder would achieve the result, or creating a new custom object rather than tweaking the requirement to use standard objects.
• Poor code quality: poor unit tests written to meet a coverage requirement rather than actually assert a valid result, or writing the same Apex code multiple times rather than consolidating in a shared helper, or hard-coded variables rather than custom metadata.
• Redundant cruft: all of the fluff that can accumulate if you let it, such as unused packages, inactive workflows, half-built validation rules, never-executed code, and remnants of proofs-of-concepts and prototypes

With all this in mind, how do you go about avoiding excessive tech debt? I focus on five key steps.

1. Have a tech debt strategy

Martin Fowler usefully categorised technical debt into four quadrants, tracking along spectrums from reckless to prudent and deliberate to inadvertent:

• Deliberately reckless: “We don’t have time for design.”
• Inadvertently reckless: “What’s layering?”
• Deliberately prudent: “We must ship now and deal with the consequences.”
• Inadvertently prudent: “Now we know how we should have done it.”

Software design choices at the ‘prudent’ end of the scale are examples of the ‘not all debt is bad’ discussion – this is the zone associated with hitting milestones, shipping code, getting early learning, and proving value.

Prudent debt is the corollary of the agile software development principle that ‘perfect is the enemy of done’. Prudent debt can be deliberate: we know we’re going to have to fix this later. It can be inadvertent: the world has changed since we made that choice. Either way, it’s unavoidable – the objective is to minimise reckless debt and properly manage prudent debt.

This starts with having the discipline to be aware of the design choices you’re making, and to track the debt as you become aware of it. The steps in your approach need to include:

• Track architectural decisions. Record the rationale for choosing to build a certain way, so that you can check whether inputs have changed if you need to revisit a decision.
• Add known debt to your backlog. For prudent decisions, this debt can be tracked from the moment the decision is taken. For inadvertent debt, start tracking the needed fix once it becomes known.
• Assign and track cost. The cost is the effort required to remove the debt, and will typically increase with every incremental change made on top of the imperfect configuration; that is, every release that passes without repaying the debt increases it.
• Assign and track value. Value is the gain to be made from fixing the debt. This may be a business impact of increased usability or better adoption, or it may be technical value in increased velocity, improved agility or reduced support costs.

Make your product owner responsible. It’s critical that the debt is owned by somebody with long term goals – beyond the lifespan of a single project or release – and is given the same weight as enhancements in future releases.

2. Find, develop, support the right people

All of the worst business software you have used was built by a contract developer working from a 160-page design document that was written by a solution architect from 90 pages of business requirements that were passed to her by a business analyst who was hired by the business for this project and spent three afternoons with the one guy in accounting who happened to be available that week. 

Put simply: the single largest cause of inadvertent technical debt is the delivery team not truly understanding the business need.

There are two high-impact roles absolutely central to avoiding this problem and successfully delivering a high quality solution that meets the business need with minimal debt. You may have different labels for these roles, but you’ll recognise the function:

• Product Owner: functionally responsible for delivery of business processes enabled by the platform, and must know the business. If you want to get the best solution, then pull the best people out of the business to drive it.
• Platform Owner: has technical ownership of the platform and must know the platform. Support them, invest in their enablement, and get them on the Salesforce Architect certification track.

Give both of these roles long-term objectives outside of the immediate project deliverable and they can drive the right behaviours in their teams. Salesforce Admins and Developers should be up to date on the platform and default to no- or low-code solutions. Where code is written it should follow patterns, and be reusable, documented and well tested.

3. Use strong development patterns

For all of the variation across different businesses, a large proportion of business software solutions can be built from a much smaller number of common components. Strong adoption of good patterns avoids the unhealthy divergence that happens when the same thing is built multiple times in different ways by people who don’t talk regularly.

The Salesforce platform is itself a collection of reusable patterns: Process Builder to deliver automation, Custom Objects to extend the data model, Lightning Component Framework to create a consistent user experience. All of these are strong functional patterns.

But a powerful platform also means there are different ways to do similar things: process builders, workflow rules, visual flows, Apex and AppExchange tools all provide different approaches to implement custom business logic.

Your Salesforce solutions need to adopt consistent patterns for your business. What sequence will your approval processes follow? What is your common approach for authenticating and authorising different integration flows?

Identify those patterns that appear frequently across your business and have a high impact on velocity, then invest in making these patterns strong. Once you have identified these ‘best buy’ patterns, ensure they are discoverable by the delivery team and used consistently.

4. Invest in improvements alongside new delivery

How often have you seen this story play out in an ‘agile’ enterprise program? The business identifies a need that’s going to take some effort to deliver. A project is stood up and does some discovery. The team identifies 100 user stories and estimates a rough velocity of 20 stories per sprint. So five ‘sprints’ are planned and a delivery date is committed all the way up to the big scary boss. Two sprints in, the project team is already scrambling and talking about cutting scope. A familiar tale?

A major objective in moving to an iterative, more agile way of working is to learn from the build as it progresses, constantly checking in to ensure you’re building the highest-value solution. Yet this common waterfall/agile hybrid approach, with minimal possible delivery effort and fixed delivery milestones, removes any space for learning, adapting, refactoring and reflection. It is a recipe for technical debt.

In my experience, the most effective delivery teams spend 30% of their time on long-term improvements to the solution. That is, 30% of time not directly delivering new functionality, but instead refining and refactoring the already built solution – including paying off those tech debt stories captured earlier – plus improving ways of working and strengthening delivery capabilities.

In planning mode at the start of a project, allocating resources away from new features feels like a huge investment. But the payback over time is improved velocity and reduced entropy.

Teams that don’t invest in software improvements quickly burn more than 30% of their time dealing with technical debt. By allocating time upfront and tracking debt, you can consciously decide on the right level of refactoring and keep on effectively delivering a high-value solution, instead of building compromised features on shaky foundations.

5. Know the software platform

Most inadvertent, reckless technical debt is caused by not knowing there is a better option.

The Salesforce platform is big and it moves fast, with 56 major seasonal releases and counting. To get full value from the out-of-the-box capabilities of the platform, you need people who know what can be done, what should be done, and what is coming next.

The Customer Success Group at Salesforce deliberately embeds continuous learning into the culture. Every role has a defined learning path with required Certifications, Trailhead badges, internal courses and peer-learning. With every Salesforce release, this learning path evolves, and staying up to date is a key metric.

Most of these learning resources are available to the whole Salesforce Ohana. Our most successful customers install Trailhead Tracker and evangelise the self-directed learning on Trailhead; they encourage and reward Salesforce Certifications so that they know their team are staying up-to-date with maintenance exams; and they use the whole Success Cloud for specialist services to develop their delivery skills.

Final thoughts

On my inherited disaster, we didn’t have the advantage of three ongoing software releases per year delivering innovation, but we did manage to drag it out of the worst depths of the black hole by identifying and tracking the technical debt, getting the right people to prioritise value, and allocating time to paying back the debt. It was a hugely painful exercise and the opportunity cost from the years of missed productivity was monstrous.

Don’t fall into the trap.

1. Make sure you have a technical strategy that prudently manages debt while minimising reckless debt.
2. Ensure you have the right people in high-impact roles focused on long-term success.
3. Make good use of discoverable, strong patterns.
4. Ensure you are giving the software development team time to manage the debt and refine the solution.
5. Empower your team to know the platform.

This article was originally published in 2019 and has recently been updated.