Every mature CMS scores roughly the same on features, so that's the wrong test. Here's how to weight what actually predicts fit in three years.
Every mature CMS on the market can do personalisation. Every one of them can handle multi-channel content delivery. Workflow approvals, A/B testing, marketing automation integration - yes, yes, yes. So when you sit down with a feature comparison spreadsheet and score six platforms against forty capabilities, you end up with six platforms that all score roughly the same. And then you're stuck.
That's not a failure of analysis. It's a failure of method. You've been incredibly thorough at measuring something that doesn't actually predict whether this platform will work for your organisation in three years.
I've been involved in roughly thirty platform evaluations over the last decade. The regret I hear in post-implementation reviews is almost never "we picked a platform that couldn't do the thing." It's "we picked a platform that could do everything, but our team can't use it without filing a support ticket" or "the hosting costs turned out to be three times the licence fee and nobody modelled that." The failure mode isn't capability. It's fit.
We've looked at the options. We know the market. We just need to make a decision.
Fair enough. And you probably do know the market better than you think. The problem isn't knowledge - it's that you're using that knowledge to evaluate the wrong things. What follows is a framework for evaluating CMS platforms against the criteria that actually predict success, weighted for your specific situation, with a process that moves from longlist to decision without the whole thing collapsing into analysis paralysis.
There's a companion piece on how to manage the stakeholder and governance side of platform selection - because the politics of choosing a platform can be just as paralysing as the technical evaluation. I'd recommend reading both together. This one covers what to evaluate. That one covers who decides and how.
I don't mean that maliciously. Vendors aren't being dishonest. But a demo environment is the best possible version of any platform. Clean data, ideal content structures, expert configuration, and a presenter who knows exactly where to click. It's like viewing a show home - beautifully staged, everything works, not a single cable out of place. Then you move in with your actual furniture, your three kids, and a dog, and suddenly the layout doesn't work at all.
A few years ago, I was in a room watching a mid-market professional services firm choose a CMS almost entirely on the strength of the demo. The vendor's solution architect built a personalised homepage in real time during the presentation. The marketing director was sold. The CTO was impressed. I was quietly thinking: nobody's asked who's going to maintain this. Six months into implementation, the marketing team couldn't replicate any of it without developer support. The personalisation engine worked beautifully in theory. In practice, it required someone who understood JavaScript and the platform's API to configure every rule. The marketing director who'd been so enthusiastic in that demo room was now raising tickets to change a banner.
I didn't say anything in that meeting. I should have. We weren't engaged until after the contract was signed, by which point the damage was done.
Feature comparison grids reward breadth of capability. A platform that ticks forty out of forty boxes looks objectively better than one that ticks thirty-five. But if five of those extra ticks require specialist development skills your team doesn't have, or integration middleware you haven't budgeted for, they're not advantages. They're liabilities you haven't priced yet.
These aren't theoretical. They come from patterns we've seen across platform evaluations and implementations - professional services firms, financial services, legal, SaaS businesses. I've written them with enough specificity that you should be able to apply each one directly to your own situation.
Before I run through them, I'll tell you which one matters most: editorial usability. If I had to pick a single criterion that predicts long-term success more reliably than any other, it's that one, and it's not even close. Everything else can be worked around. A platform your team can't use without developer support is a slow-motion disaster, and it almost never gets fixed after go-live. It just becomes the new normal.
Right. The eight:
Total cost of ownership over five years. Not the annual licence. The full cost: implementation, hosting, ongoing maintenance, customisation, training, content migration, and - this is the one everyone forgets - eventual migration off the platform when it reaches end of life. The platform with the lowest Year 1 cost frequently has the highest five-year total cost. Licence fees are the visible bit of the iceberg.
Editorial usability for your actual team. Not for a developer. Not for the vendor's demo specialist. For the marketing manager or content editor who will be creating and updating content every day without technical support. This needs testing against their real workflow - their content types, their approval process, their level of technical confidence. And the editors themselves need to be in the room for that test. I've seen evaluations where the editorial usability score was based entirely on a vendor walkthrough video. The actual editors weren't involved until after the contract was signed. Predictably, they hated it.
Developer flexibility within your team's actual capability. A composable or headless architecture that requires deep JavaScript expertise is not "flexible" if your team has one part-time developer who's more comfortable with PHP. Flexibility is relative to who's in the room. Be brutally honest about this one.
Integration capability with your specific systems. Not generic integration capability. Tested against your CRM, your marketing automation platform, your analytics stack, your client portal - whatever the platform needs to connect to in your environment. "We have an API" is not the same as "we have a documented, maintained integration with HubSpot that handles bi-directional contact sync." Ask for the specific integration, not the general promise.
Vendor stability over a five-year horizon. Financial health, product investment trajectory, market position, and the likelihood that the platform will still be a serious option at your next migration point. Some platforms that feel safe today have uncertain roadmaps - we've covered the CMS end-of-life landscape in our Replatform Reckoning guide if you want the detail.
Hosting model fit with your IT governance. Cloud-managed SaaS, self-hosted, or hybrid - each has real implications for data sovereignty, security compliance, and how much your IT team needs to be involved on an ongoing basis. For regulated firms in financial services or legal, this criterion often ends up being the decisive one. A platform that's perfect in every other respect but can't satisfy your compliance team's hosting requirements is not a viable option.
Migration ease when you eventually need to move again. Every CMS has a shelf life. The cost and complexity of the next migration - getting your content out, preserving your URL structures, maintaining your integrations - is a real and estimable factor in today's decision. Platforms that store content in proprietary formats or tightly couple content to presentation make future migration significantly harder and more expensive. This is the criterion that nobody evaluates and everybody regrets ignoring.
Implementation partner ecosystem. The quality and availability of certified partners who have delivered implementations for firms of comparable size and sector to yours. A brilliant platform with two implementation partners in your market is a riskier bet than a good platform with twenty. Partner quality often matters more than platform choice - it's genuinely one of the most underappreciated factors in the whole process.
The eight criteria above are useful to know. But knowing how to weight them for your specific context is what turns a generic evaluation into a decision you can actually defend.
The relative importance of each criterion shifts significantly depending on who you are. A 50-person firm with a non-technical marketing team and a modest development budget should weight editorial usability and TCO most heavily. If your editors can't use the thing without raising a ticket every time, it doesn't matter how elegant the architecture is. A 500-person firm with a dedicated digital team and complex integration requirements across multiple business units should weight developer flexibility and integration capability more heavily, because that's where their implementation will succeed or fail.
And here's the bit that matters most: do the weighting exercise before you talk to any vendors. The weights should reflect your business priorities, your team's capabilities, your technical environment, and your investment horizon. They should not be adjusted retrospectively to justify the platform that gave the best demo.
I've seen that happen more times than I'd like to admit. A team unconsciously shifts the weightings after the fact to make the numbers support the decision they've already emotionally made. It's human nature. But it defeats the entire purpose of having a structured evaluation. We're being objective about this - yes, I know, but are you really?
If you want the eight-criteria decision matrix as a formatted evaluation template - with weighted scoring, a longlist/shortlist format, and a proof of concept success criteria section - download it here. It's designed to be completable without specialist knowledge and it's vendor-neutral throughout.
Once you've got your weighted criteria, the evaluation moves through four stages. Each one narrows the field and increases the quality of evidence.
Stage one: Longlist. Which platforms can potentially satisfy your must-haves, based on published information, analyst reports, and your own team's existing knowledge? This should produce four to six platforms. Don't overthink it. The point is to exclude the obviously wrong options, not to make the final choice.
Stage two: Shortlist. Score the remaining platforms against your weighted criteria using the best evidence available - vendor documentation, independent reviews, analyst assessments, published case studies. This should get you down to two or three platforms. If you can't separate them at this stage, that's fine. That's what stage three is for.
Stage three: Proof of concept. This is the most reliable evidence in any CMS evaluation, and I'd go as far as saying it's non-negotiable. A structured technical evaluation in your own environment, with your own content types, your own integrations, and your own team doing the work. Not the vendor's environment. Not the vendor's content. Yours. PoCs have a tendency to expand into mini-implementations if you're not careful about scope - I've written a companion piece on how to run one without wasting three months.
Stage four: Reference checks. Conversations with firms of comparable size and complexity who have implemented the platform within the last two years. And I mean the references you find independently - through the implementation partner network, through your own contacts, through industry forums - not the three hand-picked references the vendor provides. Vendor-supplied references are always the happiest clients. You want to talk to the ones who hit problems and find out how those problems got resolved.
I could list ten, but three come up so consistently that they deserve specific attention.
Over-weighting the demo. I've already banged on about this, but it bears repeating. The platform with the best presenter wins demos. This tells you almost nothing about implementation quality, day-to-day usability, or long-term fitness. Treat demos as a starting point for questions, not as evidence for decisions.
Under-weighting implementation partner quality. A well-matched partner on a good-fit platform will reliably produce a better outcome than a poorly-matched partner on a theoretically superior platform. The partner evaluation should run in parallel with the platform evaluation, not after it. By the time you've chosen the platform and then start looking for a partner, you've already locked yourself into a pool that may not include anyone with relevant experience in your sector or at your scale.
Ignoring TCO in favour of Year 1 cost. A £15,000 annual licence that requires £200,000 of implementation and £40,000 a year in specialist hosting is not cheaper than a £30,000 licence that's implementable in half the time on standard infrastructure. The maths isn't complicated. It just requires someone to actually do it before the contract is signed, rather than after.
Once the platform decision is made, you've still got to build the board case for the investment and govern the implementation itself. That's different territory - I'd point you towards the phased proposal framework as a structure for presenting the investment in a way that actually gets approved.
But the evaluation framework itself? It's designed to give you something most CMS selections lack: a defensible rationale. When a partner or board member asks "why this platform?", you should be able to point to weighted criteria that were agreed before any vendor contact, evidence gathered through a structured process, and a proof of concept that tested real-world fit rather than demo-quality fiction.
Nobody's going to thank you for a beautiful evaluation spreadsheet if the platform's a nightmare to use eighteen months in. But if you've done this properly, you'll at least know you made the right call with the information you had - and you'll have the paper trail to prove it.
If you'd prefer to run the evaluation with structured facilitation that keeps the process honest and on schedule, book a platform evaluation workshop. Sometimes having someone in the room whose only job is to stop the evaluation from drifting is worth more than any scorecard.
