Measuring HR impact

Training Evaluation That Survives The Budget Review

Training Evaluation That Survives The Budget Review

Training Evaluation That Survives The Budget Review

Agree the indicator, baseline, comparison, timing and owner before design. The frameworks, the return on investment arithmetic and the conversation with finance.

Agree the indicator, baseline, comparison, timing and owner before design. The frameworks, the return on investment arithmetic and the conversation with finance.

·

8 min read

var(--variable-s32cKztID)

Key findings

  • In 2025, 93 per cent of organisations collected participant satisfaction data and 30 per cent were good at using learning data for decisions.

  • Kirkpatrick, Phillips and Kaufman each work backward from the business result to the programme.

  • A benefit-cost ratio of 3 to 1 equals a return on investment of 200 per cent.

  • In the Phillips example, isolating the effect cut a claim of 10 fewer complaints a month to 7.

  • Relationships with transfer looked stronger when one source supplied both measures, so the owner of the figure should sit outside the learning function.

A benefit-cost ratio of 3 to 1 equals a return on investment of 200 per cent.

To measure training effectiveness, agree five things with the sponsor before design. They are the business indicator, a baseline taken before delivery, a named comparison, the timing of the check and an owner outside the learning function. Without them, attendance and satisfaction are the only evidence left, and a finance partner discounts both.

Kirkpatrick, Phillips and Kaufman all describe the same order of work, which runs from the business result back to the training.

Most organisations measure satisfaction first and most often, as the survey figures below show.

Where Practice Stands

The Association for Talent Development (ATD) surveyed 277 talent development professionals in 2025.

It found that 93 per cent of their organisations collect participant satisfaction data.

Only 30 per cent are good at using learning data to make business decisions, and 43 per cent say their business and learning goals are aligned.

A further 29 per cent do not measure and evaluate their programmes at all, and they name isolating the impact of training on results as the greatest obstacle.

The pattern is older than the survey.

A 2016 ATD study, cited in the ATD Handbook, found that 88 per cent of organisations evaluated reaction, 60 per cent evaluated behaviour and 35 per cent evaluated results.

In 2009 the ROI Institute and ATD asked chief executives which data they valued.

Impact and return on investment ranked first and second, yet only 8 per cent of the executives received impact data and 4 per cent received return on investment.

The Four Levels

Donald Kirkpatrick created the four levels in the mid-1950s, in a doctoral dissertation on supervisory training for industrial foremen.

The levels are reaction, learning, behaviour and results.

Jim and Wendy Kirkpatrick revised the model in 2010 as the New World Kirkpatrick Model, which adds critical behaviours, required drivers and leading indicators.

Critical behaviours are the few specific actions that, performed consistently, have the biggest effect on the result.

Required drivers are the processes and systems that reinforce, monitor and reward those behaviours.

Leading indicators are short-term measures suggesting that the behaviours are on track.

The model works as a design sequence when it is used in reverse.

Start from the result the organisation exists to deliver, name the behaviour that produces it, then name the knowledge and skill the behaviour needs, and last design the experience.

The Kirkpatricks call this principle "the end is the beginning".

They treat the sponsor's expectations as the standard of value and recommend asking the sponsor what success will look like to them.

Alzate (2026) describes the aim as showing a contribution rather than proving that training was the sole cause.

Phillips And Kaufman

Phillips evaluates programmes at six levels: input, reaction, learning, application, impact and return on investment.

Kaufman places a programme within three levels of results, which are the individual, the organisation and the society it serves.

Kaufman and Guerra-Lopez separate two roles that share data and differ in purpose.

Needs assessment defines the future the organisation wants, and evaluation determines whether it was reached.

That link is why the gap found at the start supplies the measure used at the end.

The gap analysis itself is covered in training needs analysis when the solution is already chosen.

Return On Investment Worked Through

Two ratios are in common use, and they are often confused.

The benefit-cost ratio compares gross benefit with cost.

Return on investment divides net benefit by cost and states it as a percentage.

A benefit-cost ratio of 3 to 1 therefore equals a return of 200 per cent.

A fully loaded cost counts design, delivery, participant time, venue, technology and administration.

Phillips and Phillips give a worked example for a leadership programme.

  • Fully loaded programme cost was $425,000

  • Complaints above a set severity fell by 10 a month six months after the programme

  • Analysis isolated 7 of the 10 as an effect of the programme

  • Each complaint cost $6,500, so 7 a month for 12 months gave $546,000

The return was 28 per cent, which is $121,000 of net benefit divided by $425,000 of cost.

The isolation step cut the claim by three tenths, and Phillips and Phillips state that omitting it produces a baseless claim of business results.

Both the monetary value of a complaint and the isolation estimate are judgements, and a finance partner will test both.

The target return is agreed in advance, at the level of other investments such as 18 per cent, slightly above it, at break-even or at the client's expectation.

Five Parts Of A Measure

A measure that survives scrutiny has five parts, all agreed at contracting.

  • The indicator. An operational measure such as rework rate or time to competence

  • The baseline. Taken before the intervention, and the part that cannot be recovered later

  • The comparison. A named group or period that a sceptic accepts

  • The timing. Set by how often the behaviour occurs

  • The owner. Someone outside the learning function who reports the figure

Blume and colleagues (2010) found that transfer relationships looked stronger whenever one source supplied both measures in the same sitting.

The same independence applies to the owner, because a figure reported by the function that ran the programme carries less weight than one reported by the operating manager.

A check at four weeks catches first attempts and a check at six months catches retention, so the date follows how often the behaviour occurs in the job.

Baselines In Existing Systems

Most baselines need no new collection, because the systems already hold them.

  • Quality records, for error and rework rates

  • Service desk data, for escalation and repeat contact

  • Safety reporting, for incident and near miss counts

  • Sales systems, for stage conversion and cycle time

  • The HR information system, for internal movement and regretted attrition

  • Time recording, for cycle time on a defined task

A conversation with the owner of one of those systems usually produces twelve months of history.

Because the history was recorded before the programme was planned, it can serve as a baseline.

Ask for the extract before design begins.

People analytics without a data team covers working with the data that systems already hold.

Impact and return on investment ranked first and second among chief executives, yet only 8 per cent received impact data and 4 per cent received return on investment.

When Comparison Is Impossible

Most organisations cannot run a controlled comparison, and a claim that pretends otherwise fails under questioning.

Three weaker alternatives are honest.

  • Staged rollout. Units that receive the programme later act as the comparison

  • Before and after with a stated confound. Report the change and name what else changed in the period

  • Chain of evidence. Show the links from behaviour to output to result, and present it as a chain

Alzate notes that results depend on managerial reinforcement, organisational systems and external conditions as well as training.

Stating the limits of attribution costs less credibility than an overstated claim that a finance partner takes apart.

The Finance Conversation

Finance partners usually accept learning investment and reject claims they cannot verify.

Three requests prepare the conversation, and the HR adviser makes them at contracting.

  • Ask the operating manager which result they expect to move, and write it in their words

  • Ask the finance partner which convention they use, such as cost per unit of output, cost of vacancy or cost of rework

  • Ask the sponsor to confirm the review date and who will report the figure

The adviser then converts the learning measure into the finance convention and presents the cost of the gap beside the cost of the intervention.

The sponsor and the finance partner both see a comparison between two numbers.

The review date goes into the proposal, and the report follows on that date whichever way the figure moves.

The business case for a people programme sets out the case structure, and contracting with an internal client covers the meeting.

A Worked Plan

The plan below illustrates an onboarding programme for operations coordinators.

It has four roles, which are the operations director as sponsor, the team manager as owner, the finance partner and the HR business partner.

The gap is time to competence, which is the number of weeks a new coordinator takes to reach the standard.

  1. Contracting. The HR business partner asks the team manager how long a new coordinator currently takes to reach the standard and records the answer as the baseline

  2. Comparison. The previous cohort is the comparison, and the finance partner confirms the cost of a coordinator below standard per week

  3. Design. The programme is built backward from the standard, with the critical behaviours named by the team manager

  4. Delivery. The next cohort receives the redesigned programme

  5. Check. The team manager reports the weeks to standard for each person, at a date fixed in the proposal

  6. Report. The HR business partner writes one page with a recommendation and sends it to the sponsor and the finance partner

The timing follows the role and not the programme, and the owner is the operating manager.

The same pattern is covered for the first months in the first ninety days after hiring.

The One Page Report

Evaluation reports grow long as the evidence shrinks, and one page carries more weight.

  • The gap the programme was commissioned to close

  • What was delivered, in one line

  • What changed in behaviour, and who observed it

  • What changed in the result, with the confounds named

  • What to do next, as a recommendation

The recommendation chooses among four decisions.

They are continue as designed, change a named element, stop and redirect the budget, or extend to a further population.

Kaufman and Guerra-Lopez advise using evaluation for fixing and improving and never for blaming.

A report that always recommends continuation reads as advocacy.

Other Learning Formats

Portfolios now include coaching, mentoring, communities and self-directed content, and each needs its own unit of measurement.

For coaching, the unit is the goal, agreed at the start between coach, client and sponsor and reviewed at the end.

Alzate suggests the coach ask what the current situation looks like and what evidence would show progress.

For mentoring, the useful measures are whether pairs met at the agreed frequency and whether the mentee moved role.

For self-directed content, usage data shows consumption only, so check whether the people consuming it were the people with the gap.

Compliance training has a fixed standard, so the measures are completion and retention of the required knowledge against that standard.

Claims That Damage Standing

Three claims damage a function when they are examined.

  • Return on investment as a single figure. The isolation step rests on estimates, so show the range and the method

  • Correlation presented as consequence. The link between engagement scores and performance runs in a direction that nobody has established

  • Satisfaction reported as effectiveness. High scores can accompany a programme that changed nothing

Satisfaction belongs in the report as a finding about the experience.

When it is presented as impact, executives learn to discount the other numbers the function produces.

Programmes Already Running

Practitioners often take over a portfolio that someone else built and nobody measured.

Retrospective evaluation of the whole portfolio is mostly wasted effort.

Take one programme with a defined population and an operational measure, and instrument the next cohort properly.

An hour with each sponsor resets the terms, covering what will be measured from this cohort onward and what baseline can still be captured.

Behaviour change depends on transfer conditions, which leadership development and the transfer problem sets out for checking before a cohort starts.

Where The Tools Sit

In Trusted Talent Advisor, the evidence standard, the measure and the business case sit in component 06, Evidence and Impact.

The contracting conversation that sets them is in component 02, The Stakeholder Conversation Toolkit.

The Talent Template Pack includes the impact measure canvas and the cost of inaction estimate.

Related reading: skills inventory.

Sources

1. Association for Talent Development. The Future of Evaluating Learning and Measuring Impact: Improving Skills and Addressing Challenges. ATD Research, April 2025.

2. Alzate, V. M. Building a Culture of Evaluation. ATD Press, 2026.

3. Kirkpatrick, J. D. and Kirkpatrick, W. K. Implementing the Four Levels of Evaluation. In Biech, E. (ed.), ATD's Handbook for Training and Talent Development, 3rd edition. ATD Press, 2022.

4. Phillips, J. J. and Phillips, P. P. Impact and ROI: Results Executives Love. In Biech, E. (ed.), ATD's Handbook for Training and Talent Development, 3rd edition. ATD Press, 2022.

5. Kaufman, R. and Guerra-Lopez, I. Needs Assessment for Organizational Success. ASTD Press, 2013.

6. Blume, B. D., Ford, J. K., Baldwin, T. T. and Huang, J. L. Transfer of Training: A Meta-Analytic Review. Journal of Management, 36(4), 1065 to 1105, 2010.

From the system

Trusted Talent Advisor sets out the full method: seven components, 51 tools, The Talent Template Pack and The Prompt Library.

See the system

Questions this answers

How do you measure the effectiveness of training?

Agree an indicator, a baseline, a comparison, a timing and an owner with the sponsor before design. Take the baseline from existing systems, collect the figure after the behaviour has had time to occur, and ask someone outside the learning function to report it.

What are the four levels of the Kirkpatrick model?

Reaction, learning, behaviour and results. Donald Kirkpatrick created them in the mid-1950s. The updated model adds critical behaviours, required drivers and leading indicators, and it is applied in reverse, starting from the result the organisation exists to deliver.

How do you calculate training ROI?

Divide net programme benefit by fully loaded programme cost and state it as a percentage, after isolating the programme's effect. In the Phillips example, a cost of $425,000 and an isolated benefit of $546,000 gave a return of 28 per cent.

What is the difference between Kirkpatrick and Phillips?

Kirkpatrick describes four levels from reaction to results and works as a design sequence. Phillips evaluates at six levels, adding impact and return on investment, and requires the programme's effect to be isolated and converted to money before the ratio is calculated.

How do you prove training impact on business results?

Build a chain of evidence from behaviour to output to result, use a staged rollout or a previous cohort as the comparison, and name the other factors at work. Present the result as a contribution, with the limits of attribution stated.

What metrics should L&D report to leadership?

The operational indicator the sponsor owns, such as time to competence or rework rate, the behaviour change and who observed it, the cost, and a recommendation. Satisfaction can appear briefly as a finding about the experience.