Often, established businesses have inefficient processes that result from their history and culture, particularly when those processes extend globally. In helping the world’s largest certified membership organisation to improve its examination and marking processes, we helped transform the business, make it fit for the digital age and reduced overheads.
Modernising the qualification process
It’s all too easy to think the stress of exams resides solely at the desk of the students, nervously turning over their papers. But for the Institution of Structural Engineers, the stresses of the whole examination process went much deeper.
As the world’s largest certified membership organisation – numbering almost 30,000 – the Institute has for over 100 years offered a range of qualifications, all of which require an exam to be sat in hundreds of locations across the world. The qualifications are a valuable source of income, one which they were looking to maximise, yet one which was costing the Institute far too much to run.
Confronting a lengthy process
The problem was, the Institute was wrestling with a paper marking process which had remained unchanged for decades. They’d always done it that way, but cracks were beginning to show.
An entirely manual and physical event, this process involved collecting completed A3 exam papers by hand, placing them into boxes of 500, shipping to the UK, anonymising, splitting out into tranches of 20 and posting to individual examiners who, making life more difficult, are not always UK-based. Also requiring the marks of a second examiner, the whole process then had to be repeated. So far, so lengthy.
Recognising the need for change
Faced with such a problematic process – papers went missing, handwriting wasn’t always legible and disparate marks required the extra involvement of a third-party adjudicator. The Institute knew it was time for a change.
The financial benefits of doing so were compelling. There was the desire to avoid the spiralling costs of global postage. Then there was the need to shorten the process from its current nine months to fit in more revenue-earning exams (students were having to wait a full 12 months before being able to re-sit). Owning such an outdated process also reflected poorly on the Institute’s brand.
This was a landmark project for IStructE; affecting almost all of our members. Distinction understood that from the beginning, and went to great lengths to ensure a smooth process.
Exam marking online saved time and money
Diagnosing the problems
Conducting a thorough diagnosis of the current situation, including conversations with every level of stakeholder together with a little judicious fact finding, we recognised that a major obstacle was in a stereotypical view of the examiners. These were often in their 60s and 70s, mainly male and recently retired. Consequently, it had always been assumed they would be resistant to change and fearful of something new.
Removing uncertainty
During our workshops, we listened to the views of the examiners, and involved them in all our explorations and possible better ways forward. We removed the fear factor and found the prejudice was unfounded. Instead of finding opposition, our transparent and non-judgmental approach helped the examiners became healthily invested in change for the better.
And, on the first run through of our prototype, their excitement clearly demonstrated they had more in common with the digitally-savvy exam sitters than previously thought possible.
So what were we suggesting that produced such an about face? Simply that by taking the whole marking process online, by automating each stage, would make the whole journey quicker and pain-free for everyone.
The new digital interface of IStructE
Managing large amounts of data
We realised that to manage the large amount of data quickly and efficiently, the system would need to be developed in four parts:
Candidate distribution and management: enabling exam managers to distribute multiple candidates to examiners and then track an individual candidate’s progress.
PDF script viewer: giving examiners a high-resolution PDF view of a script and a simple system for marking and providing feedback on each question.
Adjudication app: automating the distribution and management of by the exam manager.
Feedback app: automatically loading examiners’ comments and marks, before saving, previewing and sending a feedback email directly.
Each of these elements made life simpler and easier for examiners and enabled them to spend much less time on marking and managing each paper.
Engaging stakeholders early
All combined into one app, the seamless process brought immediate results. With six months cut from the entire process, exams were completed quicker, leading to a threefold increase in fees. Examiners were delighted to move from the cost and hassle of a physical, postage-based system.
In the end, our early involvement proved decisive in that we could involve the audience from the very start and keep the various stakeholders engaged at every stage. Being invested in the vision, the journey to lasting change became a truly satisfying and rewarding project for everyone.
