Thinking Differently About Training Evaluation

An advantage of Kirkpatrick’s four levels of training evaluation is their wide adoption.

They are a tenet of the training profession.

But one disadvantage is that Kirkpatrick’s four levels give trainers a different mindset than the rest of business people.

Business people typically think in terms of effectiveness (results) and efficiency (costs). What does something cost, and does it work?

But Kirkpatrick’s four levels focus on four levels of effectiveness (with costs being addressed in Phillips’ fifth level of ROI).

A second disadvantage of Kirkpatrick’s four levels is that trainers often do not have access to the data needed for its upper levels of evaluation.

Trainers are service providers, and service providers simply aren’t given easy access to performance and business data. (Studies show time and again that most training evaluation is Kirkpatrick Level 1 reaction data.)

What if trainers had had a different model of training evaluation (see graphic above) for the last 50 years?

What if trainers had focused on both efficiency and effectiveness, just like other business people?

And what if trainers had accepted that they need to focus on evaluation data that they can collect (or estimate) themselves?

And then — if available — data that requires client cooperation to collect?

What if, for the last 50 years, trainers had been used to calculating — as Level 1 evaluation — cost data that they can collect independently: total costs, time, reach, scale, and so forth?

And what if Level 2 evaluation had been effectiveness data that they themselves can collect: reactions, usage, observations, performance on exercises, and so forth?

What if Level 3 evaluation had been efficiency data that required client cooperation (and thus was tougher to get): cost/time of follow-up coaching and OJT, cost of errors and accidents, and so forth?

And what if Level 4 evaluation had been effectiveness data requiring client cooperation: test results, supervisor feedback, job performance, business results, ROI, ROE, and so forth?

I think that if trainers — for the last 50 years — (a) had been routinely focusing on costs as readily as smile sheets and (b) had come out of the closet about the difficulty of obtaining performance and business data — the training profession would be very different today.

More business-like, more in sync with senior management, more skeptical of fads but at the same time more open to efficiencies.

Tagged as: Evaluation, Kirkpatrick, Measurement