THE BRIEFING ROOM

Your agency is using AI to build your project. What are you actually asking about it?

I was on a call a few months back with the CTO of a mid-sized SaaS company. They'd just finished a platform rebuild with an external agency - came in under budget, ahead of schedule. Everyone was delighted. Champagne emoji in the Slack channel, probably.

Then their internal team started working on the codebase.

The code did what the spec said. But it was inconsistent in a way that was hard to put your finger on at first. Different modules had clearly different authorship styles. Some sections were over-engineered with patterns that didn't match the rest of the architecture. The documentation described behaviours that didn't quite match what the code actually did. The internal team spent six weeks just getting comfortable enough with it to make changes confidently. Six weeks. On a project that had supposedly been delivered.

He wasn't angry about it, which I found almost more unsettling. Just resigned. "We got what we asked for," he said. "We just didn't ask for the right things."

Was AI involved in the build? Almost certainly. Was that the problem? Not exactly. The problem was that nobody had asked how AI-generated code would be reviewed, integrated, or quality-assured. The agency delivered what was in the spec. They just delivered it in a way that optimised for their timeline, not for the client's long-term ability to maintain the thing.

That conversation has stuck with me. Because it's happening everywhere, and most clients have no idea.

The bit your delivery partner probably hasn't told you

Your development partner - the agency, the consultancy, the freelance team, whoever's building your platform - is using AI in their delivery process. Right now. Whether they've told you or not.

That's not a controversial statement. GitHub's 2024 developer survey found that 92% of developers are using AI coding tools in some capacity. Copilot, Claude, ChatGPT, Cursor - the specifics vary, but the direction doesn't. If you're commissioning digital work in 2025 and you assume your delivery team is writing every line of code by hand, you're operating on an outdated mental model. It's a bit like assuming your accountant still uses a ledger book.

And honestly? That's fine. AI-assisted development can be brilliant. Faster prototyping, better test coverage, cleaner documentation. Code that would have taken a senior developer three days knocked out in an afternoon. I've seen it work beautifully.

But I've also seen it go wrong in ways that don't show up until months after launch. And the person who pays for that is you.

AI-generated code is fine. It speeds things up and reduces cost. What's the problem?

The problem isn't AI. The problem is that you're buying the output without understanding the process - and that gap creates risk you probably aren't pricing in.

What's actually changed in how things get built

Lee Conlin, our Head of Engineering at Distinction, has been tracking this shift closely. His observation is worth repeating: AI hasn't changed what gets built, but it's fundamentally changed how it gets built. And that matters more than most clients realise.

Here's roughly where AI shows up in a modern development process. Not every team uses all of these, but most are using several.

Code generation is the big one. Developers describe what they need, and AI generates code - sometimes whole functions, sometimes boilerplate, sometimes complex logic. GitHub Copilot alone is now used by over 1.8 million developers. The productivity gains are real. But "generated" is not the same as "reviewed," and "fast" is not the same as "correct."

Automated testing is, when done well, one of the most valuable applications - it produces more robust software, catches edge cases a human might miss. When done carelessly, it produces tests that pass without actually testing anything meaningful. I've seen both. The difference is usually whether someone senior is paying attention.

Documentation used to be the thing that got skipped when timelines got tight. AI can now generate it from code comments and function signatures. The upside: you actually get documentation. The downside: documentation generated from poorly commented code is just a more formal version of confusion.

Prototyping is where things get interesting. Need a working prototype in two days instead of two weeks? AI makes that possible. For discovery phases and proof-of-concept work, this is fantastic. But prototypes built fast have a habit of becoming the thing that ships. And prototype-quality code in production is a ticking clock.

So far, so positive. Most experienced engineers are using these tools thoughtfully - as an accelerator, not a replacement for thinking. The problem is that "most" and "all" are different words, and when you're the client, you're exposed to the specific team working on your project, not the industry average.

Who actually benefits from the efficiency gains?

This is where it gets commercially interesting.

AI-assisted development makes your delivery partner more efficient. Some estimates put productivity gains at 30-50% for routine coding tasks. Which raises an obvious question: who benefits from that efficiency?

If your partner quoted you 800 hours for a project, and AI means they can deliver the same scope in 550 hours, what happens to the other 250? Do they pass the saving to you? Reinvest it in better quality assurance? Quietly pocket the margin? Or, and this is the scenario worth losing sleep over, do they deliver in 550 hours and skip the quality steps that would have filled the other 250?

I'm not suggesting anyone's being deliberately dishonest. But incentives matter. And if you're not asking the question, you're trusting that your partner's commercial interests are perfectly aligned with your quality requirements. That alignment doesn't happen by accident. It happens because someone - usually the client - makes it a condition of the engagement.

The governance questions nobody's asking

When you commission a piece of digital work, you're buying an asset. That asset needs to work today, but it also needs to be maintainable, extendable, and - increasingly - ready for whatever AI-adjacent changes come next. The governance around how that asset is produced has real commercial consequences.

Who reviews AI-generated code? The most important question and the one least often asked. In a well-run team, AI-generated code goes through the same peer review process as human-written code. In a less disciplined team, it gets committed with a quick glance because "the AI wrote it and it passes the tests." Those two approaches produce very different outcomes over time.

What are the IP implications? This one's murky. AI coding tools are trained on vast repositories of open-source code. There are active legal debates - and actual lawsuits - about whether AI-generated code can inadvertently reproduce copyrighted or licensed material. If your delivery partner is generating significant portions of your codebase with AI, you need to understand the licensing implications. Probably fine is not a legal position I'd want to defend to a board.

How is quality assured? AI-generated code can be syntactically perfect and logically flawed. It can pass automated tests while embedding assumptions that don't match your business rules. The testing question isn't "do you test?" - everyone says yes. The question is "how do you test AI-generated code differently from human-written code?" Because the failure modes are different. AI tends to produce code that looks confident and plausible even when it's wrong. Experienced developers have learned to treat AI output with healthy scepticism. Junior developers sometimes haven't.

What happens when things need changing? Code that was generated quickly can be surprisingly difficult to modify. If the developer who prompted the AI to write a function doesn't fully understand what that function does - and I've seen this - then changing it later becomes an archaeology project. You're not maintaining code; you're reverse-engineering someone else's conversation with a machine.

Is the AI use documented? Sounds bureaucratic, but it matters. If your delivery partner leaves and you need another team to pick up the work, knowing which parts of the codebase were AI-generated is useful context. It's the difference between handing over a map and handing over a mystery.

What to actually ask

If you're commissioning digital work - a platform build, a product development engagement, a replatforming project - here's what I'd want answered before signing anything. Not aggressively. Just as two adults having an honest conversation about how the work gets done.

Start with the basics: what AI tools does your team use, and for what? You're not looking for a specific answer. You're looking for transparency. A partner who says "we use Copilot for boilerplate and test generation, Claude for documentation, and everything goes through peer review" has clearly thought about it. A partner who gets cagey or says "we don't really use AI" is either behind the curve or not being straight with you. Neither is great.

Ask how they review and quality-assure AI-generated code. The answer should describe a process, not a vibe. Code review, automated linting, integration testing, manual testing of business logic. If the answer is "our developers check it," press harder. Check it how? Against what standard?

Ask about IP and licensing policies for AI-generated code. No policy isn't necessarily a disaster, but it means they haven't considered the risk - and you're inheriting it.

Ask how they ensure long-term maintainability of AI-assisted codebases. You want to hear about coding standards, architectural consistency, documentation practices, handover processes. If the answer focuses entirely on speed and cost, they're optimising for delivery, not for your ownership experience.

And ask whether they'll tell you if the balance of AI-to-human contribution changes significantly during the project. If they start with senior developers and gradually shift to junior developers leaning heavily on AI, you should know. That's a different risk profile than what you agreed to.

The right answer to most of these questions isn't "we don't use AI." That ship has sailed. The right answer is "we use it deliberately, we govern it properly, and we're transparent about it."

The thing that actually bites

The most expensive AI-related risk isn't bad code on day one. It's unmaintainable code on day 365.

A codebase that works but can't be easily understood, modified, or extended by the next team who touches it is a liability dressed up as an asset. And AI-generated code has a specific tendency towards this problem, because it can produce working solutions that no single person fully understands. The developer prompted the AI. The AI generated the code. The tests passed. But nobody sat down and truly comprehended every line - because that would have taken longer than writing it manually, which defeats the point.

I've written separately about the fragility of digital foundations, and this connects directly. Your CMS, your platform, your custom-built tools - they're only as valuable as your ability to evolve them. If AI-assisted development produces something that works today but creates a maintenance nightmare in eighteen months, you haven't saved money. You've deferred cost and added interest.

We use a framework at Distinction called WHNN - the What and the How, for the Now and the Next. The "Next" part is what most people skip. What does your codebase need to look like in 18 months? Who's going to be working on it? Will they be able to understand what was built and why? If your delivery partner can't answer those questions, the speed and cost savings of AI-assisted development are a false economy.

A word in defence of AI (yes, really)

We use AI tools in our own development process at Distinction. Lee and the engineering team have been integrating them for a while now, and the results - when governed properly - are impressive. Faster delivery, better test coverage, documentation that actually exists. Real improvements.

But we also review every line. We maintain architectural consistency. We document what's AI-assisted and what isn't. And we have honest conversations with clients about how we work - because that transparency is part of the service.

The challenge for you, as the person commissioning the work, is that you can't see inside the development process. You see the demo. You see the staging environment. You see the launch. You don't see whether the code was written by a senior engineer, generated by Copilot and rubber-stamped by a junior, or some combination that nobody's tracking. The difference between those scenarios might not be visible for months.

We've been quoted three different prices for essentially the same project. The cheapest one is half the cost of the most expensive. They must just be more efficient.

Maybe. Or maybe they're planning to have a junior developer and Claude do 80% of the work with minimal review. You won't know unless you ask. And by the time you find out the hard way, the codebase is your problem, not theirs.

Where this is heading

AI-assisted development is only going to become more prevalent. The tools are improving fast. Within a couple of years, the proportion of code that's AI-generated versus human-written will probably flip. That's not a crisis - it's an evolution. But it means the governance questions here aren't a one-off checklist. They're a permanent part of how you should evaluate and manage delivery partners.

If you're a CTO, a head of digital, or a technology leader commissioning work, this is your responsibility now. Not because you need to understand the technical details of how Copilot works, but because you need to understand the commercial implications of a development process you can't directly observe.

Our Fragility of Digital Foundations scorecard is a quick self-assessment that helps you benchmark the health of your platform foundations - worth ten minutes if you're not sure whether the systems you're building on are as solid as you think.

And if you're reading this thinking "I should probably have a conversation with our current agency about this" - yes. You probably should. The partners worth keeping will welcome it. The ones who get defensive are telling you something too.