Beyond Completion Rates

Beyond Completion Rates: How to Measure Whether Training Actually Worked

Short answer: Completion rates tell you that people finished a course. They do not tell you that anything changed. To measure whether training worked, track four things instead: what people can do that they could not do before, what they do differently at work, what happened to the business metric the training was meant to move, and how much of that you can reasonably attribute to the training. You do not need a research team to do this. You need to decide what success looks like before you build the course, not after.

Why completion rates became the default metric

Completion rates are easy. Your system counts them automatically. They produce a clean number you can put in a slide. And for compliance training, they genuinely matter, because in an audit you need to prove people were trained.

The problem is that everyone knows this number is soft. A completion rate tells you someone reached the last screen. It cannot tell you whether they read anything, whether they remember it a week later, or whether they behave differently because of it. It is an operational metric dressed up as an effectiveness metric.

This matters more than it used to. L&D budgets are under real scrutiny, and leadership teams increasingly want evidence of impact rather than evidence of activity. Research from the Association for Talent Development has consistently found that organisations which measure training properly perform better than those that report participation alone, not because they train more, but because measurement tells them what to fix.

What completion rates can and cannot tell you

It helps to be precise about this, because completion data is not worthless. It is just answering a narrower question than most reports imply.

Completion rates can tell you:

Whether your training reached the people it was meant to reach

Whether the course is the right length, sharp drop-off at the same point usually means the content is too dense there, not that people are lazy

Whether you have an evidence trail for a regulator

Completion rates cannot tell you:

Whether anyone learned anything

Whether the learning survived the week

Whether behaviour at work changed

Whether the problem the training was built to solve got smaller

There is a design dimension to this too. A course that people abandon halfway is not generating any data worth having, so format is the first lever. We looked at this in more detail in interactive courses versus slide decks, and the pattern is consistent: courses that ask learners to do something produce both better retention and richer signals to measure. Courses built in Saga see 3.4× higher completion rates than slide-based equivalents, which means the underlying data set is simply bigger and less skewed.

The four levels, in plain language

The most widely used framework for this is the Kirkpatrick model, developed in the 1950s and still the standard reference. It has four levels. Most teams measure the first two and stop, which is exactly where the useful information starts.

Level 1 — Reaction: did people find it useful?

A short survey immediately after the course. Keep it to three or four questions and send it at the end of each module rather than the end of the whole programme, because recall drops fast and so do response rates. Ask whether the content was relevant to their actual job, not whether they enjoyed it.

Level 2 — Learning: can they do something they could not do before?

This needs a before and after. Build a short pre-assessment before the course launches and run the same questions afterwards. Without the baseline you have a score, not a change. Scenario-based questions work far better than recall questions here: asking someone what they would do in a specific situation tells you more than asking them to define a term.

Level 3 — Behaviour: are they doing anything differently?

This is where most measurement programmes stall, and it is the level that actually matters. Behaviour change shows up in manager observations, quality checks, audit findings, support ticket categories, or system logs. It rarely shows up inside the learning platform, which is why it gets skipped.

The single most effective thing you can do at this level costs almost nothing: give managers a one-page conversation guide when the course goes out. Training that no manager ever reinforces almost never changes behaviour, no matter how good the course was.

Level 4 — Results: did the business metric move?

Pick a number the business already tracks and that the training was genuinely meant to influence. Time to productivity for new hires. Reported phishing attempts. Safety incidents. Error rates. Customer satisfaction on a specific issue type. If you cannot name that number before you build the course, you will not be able to claim it afterwards.

Why teams stop at Level 2

Not laziness. Three practical reasons.

The data lives somewhere else. Levels 1 and 2 sit inside the learning platform. Levels 3 and 4 sit in the CRM, the helpdesk, the incident log, or a manager's head. Getting them requires asking another team for something.

Nobody agreed what success meant. If the request was "we need GDPR training," there is no target to measure against. If the request was "we are getting too many data handling mistakes in the support team," there is.

Attribution feels intimidating. People assume they need a control group and a statistician. You mostly do not. Being honest about what else changed in that period is usually enough for a credible internal conversation. If you do need something more rigorous, the Phillips ROI methodology adds isolation techniques on top of Kirkpatrick's fourth level.

Five metrics worth tracking instead

If you want a practical starting set, these five are realistic for a small team and far more informative than completion alone.

1. Knowledge gain, not knowledge score. The difference between the pre-test and the post-test. A post-test average of 85% means nothing if people scored 82% before they started.

2. Delayed retention. Re-ask three or four key questions thirty days later. This is the single most revealing number in most training programmes, and almost nobody collects it. Forgetting is the default state, and spacing the material out is the fix — something we covered in microlearning versus traditional courses.

3. Application rate. Ask managers a single question sixty days on: has this person changed how they do X? One question, asked consistently, beats an elaborate survey nobody fills in.

4. The business metric you named up front. Tracked from before the training started, so you have a baseline.

5. Where people drop off. Not the completion rate itself, but the point in the course where completion breaks. That tells you what to rewrite.

Course design decides what you are able to measure

This is the part that tends to get missed. Measurement is not something you bolt on after the course exists. The format of the course determines which signals it is capable of producing.

A linear slide deck with a quiz at the end can only produce two data points: did they finish, and did they pass. There is nothing else to read. A course with embedded knowledge checks, decision points and practice scenarios produce a running record of where understanding breaks down, and it does so per question, per learner, per topic.

That is why simulation-style formats outperform lectures so consistently in areas like security awareness, as we set out in why simulations beat lectures. A learner clicking a simulated phishing link generates a genuine behavioural signal. A learner watching a video about phishing generates a timestamp.

Saga is built around this: chat-based roleplay scenarios, swipe quizzes and knowledge checks are generated into the course automatically rather than added by hand, so the measurable moments are there by default. Courses export to standard formats, so the results land in whatever system you already use for reporting rather than in a separate silo.

A simple 90-day plan

Days 1–7. Pick one course. Name the business metric it should influence and write down where that number sits today. Write four pre-assessment questions.

Days 8–30. Run the pre-assessment, deliver the course, run the post-assessment. Send managers a one-page guide on what to reinforce.

Days 31–60. Re-ask three questions from the assessment. Ask managers the single application question.

Days 61–90. Pull the business metric again. Write up what changed, what did not, and what else was going on in that period that might explain it. Being honest about the last part is what makes the rest believable.

One course, one quarter. That is enough to have a completely different conversation with your leadership team than a completion percentage allows.

A note on compliance training

Compliance is the one area where completion and documentation genuinely are the deliverable, because you need to demonstrate to a regulator that training happened. But even here, the record is stronger when it includes some evidence of understanding rather than attendance alone. This is explicit in the AI literacy duty under the EU AI Act, where the expectation is that training is calibrated to the role rather than delivered as one generic module — we go into it in our guide to Article 4. A knowledge check attached to the completion record is a materially better audit document than a tick.

Frequently asked questions

What is a good training completion rate?

For mandatory compliance training, most organisations target 95% or higher, because anything less is an audit gap. For optional or developmental training, anything above 60% is respectable and the figure matters far less than what the finishers actually did afterwards. Chasing the completion number on optional training usually produces courses that are shorter and emptier rather than better.

How do you measure training effectiveness without a big budget?

Pre-test, post-test, a thirty-day retention check and one question to managers. That combination costs almost nothing and covers three of the four Kirkpatrick levels. The expensive part of measurement is usually the reporting infrastructure, not the measurement itself.

How long after training should you measure impact?

Reaction immediately. Learning immediately. Retention at around thirty days. Behaviour at sixty to ninety days, because people need time to encounter a situation where the training applies. Business results usually need a full quarter, and sometimes two.

Can you calculate ROI on training?

Yes, but only if you converted the business outcome into a monetary value and captured the full cost of the programme, including participant time. For most internal training the more useful exercise is demonstrating a clear directional link between the training and a metric leadership already cares about, rather than producing a precise percentage that invites argument about the assumptions.

Do learners find heavily measured training annoying?

Only when the measurement is bolted on. Assessments that feel like surveillance get resented; knowledge checks that are part of the learning experience do not, because retrieval practice is itself one of the most effective ways to make something stick. The measurement and the learning can be the same activity.

Start with courses worth measuring

Measurement exposes weak training. That is the point of it, and it is also why it makes people nervous. If a course was never going to change anything, better data will simply show that faster.

So, the two problems are worth solving together. Saga generates complete, interactive courses in about five minutes from material you already have, a policy document, a presentation, or a briefing note, with knowledge checks and practice scenarios built in rather than added afterwards. That means the course is more likely to work, and it produces something to measure when it does.