Measuring HR impact
·
8 min read
Key findings
Four tests reduce a dozen candidate findings to the one or two that a business head can act on.
Pay, career paths, workload and inconsistency between units belong to executives, and sending them to managers produces the next cycle's cynicism.
An action plan needs eight fields, including the executive owner, the date, the threshold and a comparison group.
A before-and-after reading with a comparison group shows whether the action worked.
A plan with one change to one item, with a date, outperforms a plan with six aspirations.
An engagement survey action plan works when it holds one or two actions, each owned by an executive who can change a decision.
The plan also states in advance what the next survey must show.
Managers receive only the changes that are within their control.
Pay, career paths, workload and inconsistency between units go to the people who own them.
Why Engagement Action Plans Fail
Most organisations run the survey, send unit reports to managers and ask every manager for a plan.
Nine months later nobody can name a decision that changed, and the next survey opens with a lower response rate.
Three causes sit underneath.
Managers are asked to fix things outside their control, such as pay or workload driven by demand
The plan has no owner above the manager, so nothing escalates
Nobody checks, so the plan is written once and never opened
Ray and colleagues describe the weakest organisations in the same terms.
They gather employee feedback regularly and do not take the time to turn it into action plans.
They also name two common mistakes.
One is prioritising everything employees want over what the business needs.
The other is lacking business outcome data at a level that links to engagement scores.
What An Engagement Survey Measures
A survey measures belief at a point in time, as reported by the people who chose to respond.
It is good at three things.
Detecting where an experience differs sharply by group
Tracking a specific belief over time
Producing the question worth investigating
It is weak at explaining causes and at supporting a claim that engagement produced performance.
A practitioner who states those limits before presenting is in a stronger position than one who is asked about them.
Choosing One Or Two Findings
A survey report produces a dozen candidate findings.
Four tests reduce them to the one or two that a business head can act on.
Is it a specific item, and not the index?
Does it differ sharply between comparable units?
Does it connect to something the business already counts, such as attrition, absence or customer complaints?
Can a named executive change a decision within a quarter?
A finding that passes all four goes into the plan.
A finding that passes the first three and fails the fourth goes to the executive team as a decision to make.
Everything else stays in the unit reports.
Read The Items, Not The Index
A composite score hides the two or three items that moved.
Three reads produce more than any headline.
Item-level movement since the last cycle, ignoring the index
The gap between the highest and lowest scoring units on the same item
Items where the distribution is bimodal, which means two populations are having different experiences
Most reporting tools hide the third by showing averages.
Cut By Manager And Tenure
Most reporting cuts the data by function and level, because that is how the organisation chart works.
The experience that produces the answers is local.
Two other cuts usually explain more.
By manager, where the population allows it without identifying individuals
By tenure band, which separates the joining experience from the staying experience
A function cut tells you which director to talk to, and a tenure cut tells you what to talk about.
Free Text Comments
Comment fields are usually reported as word clouds.
A structured read is more useful and needs no tooling.
Take two hundred comments and read them before choosing any category
Code them into six to eight themes and count them
Choose two verbatim quotations per theme, for representativeness and not for force
Compare the themes with the survey items
The comparison shows whether the instrument asks about the right things.
The counts change an executive conversation in a way the scores do not, because the comments contain specifics.
A Worked Example
Consider a hypothetical retailer with 4,200 staff and a 71 per cent response rate.
The report yields eleven candidate findings.
HR applies the four tests and two findings survive.
The first is the item on planning life around the rota.
It scores 48 per cent favourable in stores and 71 per cent at head office, and store attrition is the highest in the business.
The retail operations director can change rota publication from one week ahead to three.
The change costs a scheduling licence.
The second is that fewer than half of store staff can see a next role.
No store manager can fix that, so it goes to the HR director as a career path design task with a date.
The other nine findings go into unit reports with one suggested change each.
The plan for the rota finding fits on one sheet, and the next section sets out its fields.
The Plan Template
An engagement action plan needs eight fields.
Finding. The item, the units and the scores
Decision to change. The rule, schedule or budget that produces the score
Owner. The executive who can change it
Date. When the change takes effect
Cost. The budget line, or a statement that there is none
Evidence of change. The item and the behavioural measure that should move
Threshold. The size of movement that counts
Comparison. The units that did not change, used as a check
A plan with one change to one item, with a date, outperforms a plan with six aspirations.
Employees notice a finding that appears in the comments and never appears in the response.
Three Things A Manager Can Change
Most local action plans should be short and concrete.
How the team meeting runs, and who speaks
How work is allocated and how that is explained
How and when feedback and recognition happen
A manager takes one change to one of these, with a date, and reports back at the next team meeting.
What Belongs Above The Manager
Four themes recur in most organisations, and none can be solved locally.
Pay and its perceived fairness, which is a reward decision
Career paths and their visibility, which is a design decision
Workload driven by demand or headcount, which is a planning decision
Inconsistency between units, which is a governance decision
Sending these down to managers produces the cynicism that shows up in the next cycle.
The practitioner's most useful act is to name which findings are not a manager's to fix and to take them to whoever owns them.
The same routing applies to exit data, as set out in exit interviews worth reading.
Who Owns The Engagement Action Plan
The business owns the plan, and HR owns the process around it.
Each action has an executive owner who can change the decision, and that name appears on the plan.
HR supplies the analysis, the template, the calendar and the check at ninety days.
A plan owned by HR loses its owner when HR moves on to the next survey cycle.
Where the finding is a rule, changing culture by changing rules sets out how to audit and change it.
Sharing Results With Employees
Results reach employees in three steps.
Brief managers first, with their own unit's results and the one change they are expected to make
Share the headline results and the two or three findings the organisation will act on, in one message from a senior leader
Say which findings will not be acted on this year, and why
A fortnight to a month after the survey closes is a workable target, because silence in that period is read as inaction.
The third step is the one most organisations skip.
Employees notice a finding that appears in the comments and never appears in the response.
Proving The Action Worked
The plan states the evidence before the action starts.
Schiemann and Dinsell describe a financial services firm in which each team that adopted a new approach to performance management was surveyed before and after.
The scores of those teams were markedly higher than the scores of teams that kept the traditional approach, and the gain lasted well after adoption.
The design has two parts a practitioner can copy.
The first is a before-and-after reading on the same item.
The second is a comparison group of units that did not change.
Add one measure from records, such as unplanned absence or attrition in the changed units.
Peters describes a similar sequence at WMATA.
An internal survey on the new performance management programme found that 47 per cent of managers thought it inaccurate at measuring job performance.
The 2014 employee opinion survey then reported on the same programme, with 67 per cent of employees saying they received clear and regular feedback.
Response Rate As Evidence
Response rate is usually treated as an administrative statistic, and it is also a finding.
A falling rate in a unit says something before the scores do.
Three readings are useful.
A rate below half means the results describe a self-selected group and should be reported as such
A rate that falls year on year usually means the previous cycle produced nothing visible
A rate close to a hundred per cent in one unit deserves a quiet question about how it was encouraged
Why Employees Stop Filling In Surveys
Employees stop responding for two reasons that the organisation controls.
The first is that they saw no change after the last survey.
The second is that they were asked too often.
Pulse surveys multiply because they are easy to run and produce a number that looks like management information.
Weiss, Molinaro and Davey suggest a way to avoid fatigue in a survey of leaders.
They sample a quarter of the population each quarter, until everyone has been asked once in the year.
Two rules protect the tolerance for being asked.
Ask only where the answer will change something within the cycle
Report what changed since the last round before opening the next one
How Many Focus Areas
Two is the working limit for an organisation and one for a unit.
The limit follows from the budget of executive attention, because each focus area needs an owner who changes a decision.
An organisation that can name two decisions it changed in a year has a better record than one that can list twenty actions.
Engagement And Performance
The link between engagement scores and business performance is asserted more confidently than the evidence supports.
The direction of causation is uncertain, because people in successful units have reasons to report positively.
Two claims are defensible.
Specific items, such as whether someone intends to be here in a year, relate to behaviour the organisation can observe
Sharp differences between comparable units indicate something local that is worth investigating
A claim that a rise in the index produced a rise in revenue gets a function's numbers discounted.
Reporting To The Executive Team
Four items carry the report.
What moved at item level, and where
What the free text says, with counts
What HR recommends, and who owns each item
What will be measured next, and when
The third item separates a function that advises the business from one that reports to it.
A report without recommendations hands the interpretation to the audience, who will bring their own assumptions.
Related Reading
Survey themes are strongest when they sit beside other evidence, as in people analytics without a data team.
Where The Tools Sit
Trusted Talent Advisor covers the measure, the threshold and the recommendation in its evidence and impact component.
The conversation that routes a finding to its real owner is in its stakeholder conversation toolkit.
The template pack includes the coding frame and the findings summary.
Related reading: how to measure training effectiveness.
Sources
1. Ray, R., Dye, D., Hyland, P., Kaplan, J. and Pressman, A. How to Build a Culture of Engagement, in Bickham, T. (ed.) ATD Talent Management Handbook. ATD Press.
2. Schiemann, W. A. and Dinsell, C. Adopting a More Dynamic Approach to Performance Management, in Bickham, T. (ed.) ATD Talent Management Handbook. ATD Press.
3. Peters, M. Engaging Leaders in Performance Management, in Bickham, T. (ed.) ATD Talent Management Handbook. ATD Press.
4. Weiss, D. S., Molinaro, V. and Davey, L. Leadership Solutions: The Pathway to Bridge the Leadership Gap. Jossey-Bass.
From the system
Trusted Talent Advisor sets out the full method: seven components, 51 tools, The Talent Template Pack and The Prompt Library.
See the system
Questions this answers
What should you do after an employee engagement survey?
Choose one or two findings a business head can act on, brief managers before results go out, share headline results and the chosen actions within a month, and route the rest to the people who own them.
How do you create an action plan from survey results?
Apply four tests to each finding: a specific item, a sharp gap between comparable units, a link to something the business counts, and an executive who can change a decision within a quarter. Then complete an eight-part plan.
How soon should you share survey results with employees?
A fortnight to a month after the survey closes is workable. Brief managers first, then send the headline results and the two or three chosen actions, and say which findings will not be acted on this year.
Why do employees stop filling in engagement surveys?
They saw no change after the last survey, or they were asked too often. A falling response rate in a unit says so before the scores do.
Who should own the engagement action plan?
The business owns it, through an executive who can change the decision behind the finding. HR owns the process, the analysis, the template and the check at ninety days.
How many focus areas should an action plan have?
Two for an organisation and one for a unit. Each focus area needs an executive who changes a decision, and executive attention is the scarce resource.
Keep reading
Measuring HR impact
Internal Mobility Metrics Without A Platform
Seven measures an HR adviser can build from existing records, each with a stated definition, a base and a way to read it.
7 min read
Workforce and future skills
Job Rotation As A Talent Gap Strategy
A job rotation programme closes a talent gap when the role, the person, the cover and the evidence are agreed with both managers before it starts.
8 min read
Talent reviews and succession
Seeing Talent Below The Top Team
Visibility of leadership potential drops below the top team. A cross-unit forum, run on one page per unit and a written evidence standard, is the method for seeing it.
8 min read