e learning age - the magazine for the learning organisation

Making an impact

Learning technologies can deliver the innovation this economic climate needs to support business priorities – we know that. But many experts in learning and development (L&D) believe real evidence of their UK impact is lacking. 

Last month e.learning age published a review of Becta’s report “Delivering Results”, which consolidated  evidence and case studies that demonstrate the impact of learning technology to support business priorities. Becta also supported the Towards Maturity Impact Indicator – a complementary online study with 200 organisations (across the private and public sector) to understand how technology is adding business value and improving efficiency.

Given the nature of the challenges associated with measuring learning impact (and the reluctance of time starved participants to allocate precious resources to independent research), we didn’t try to isolate the impact of particular learning technologies in a learning solution in this study (although there is some review of technology in the final report).

However, following discussions with prominent L&D experts, we believe the collective opinions of a significant number of participants would provide a useful indicator of the value learning technologies can deliver. Our objectives in this study were:

• To clarify the extent to which learning technologies deliver basic efficiencies, ie more learning for less cost
• To understand the extent to which learning technologies help L&D professionals deliver additional tangible, relevant business value to their organisations
• To consolidate opinion and cut through the hype that inevitably surrounds technology innovation
• To develop an independent evidence base that will support organisations looking to make appropriate business decisions over the way that learning technologies can help learning to adapt to changing business priorities and help organisations engage management teams and decision makers.

The study set out to investigate three main ‘impact indicators’ that would have potential value for an organisation.

• Efficiency This looked at the basics – impact on cost, volumes delivered and time savings
• Business agility Looked at the ability of learning to help organisations respond productively to change, considered time to competence; the ability of L&D to respond to business need; and overall business responsiveness to change
• Management perception of value This came about as a result of research by IBM/ASTD in 2006 that looked at top executives’ perceptions of the strategic value of learning and found they are less concerned about return on investment (RoI) and more concerned about the perceptions of key stakeholders on the strategic value of learning. In this study we were interested in how we manage management perception of value and considered what we measure and what we report on.

Efficiency indicators

Efficiency without effectiveness results in disengaged audiences and wasted resources. While a focus on efficiency alone may impress accountants, history has shown it is completely counterproductive for staff and managers who fail to see the value in the learning. In fact research studies from the US (including the ASTD’s ‘Value of Evaluation’ study) showed that far fewer e-learning than classroom programmes are evaluated because they are thought to be cheaper to produce and so warrant less scrutiny.

So what are the findings? Participants reported:

• Average cost savings of 11%
• Average time saving of 28% (36% including travel)
• Average volume increase of 47%

Figures are based on conservative calculations and have implications for both private and public sectors.

Efficiency drivers and the emphasis on leaner organisations are at the forefront of thinking in both private and public sectors. Cost savings speak for themselves in this environment and time savings can be equated to a bottom line value when you consider the cost of staff time, lost productivity and increasing the existing capacity of stretched L&D teams.

Increased volumes of learning can also increase the capacity of an existing L&D function and improve the proportion of staff being trained in a business. The BIS “Skills for Growth” report highlights (Nov 09) that “A one percentage point increase in the proportion of employees trained is associated with an increase in productivity of 0.6 percentage points – which in turn is worth around £6 billion a year to the UK economy.” So potentially the volume of learning delivered could really impact an organisation – as long as it is directed at appropriate needs in appropriate ways.

We also found that many organisations may be doing themselves a disservice. Only 29% conduct a simple cost benefit analysis (comparing costs of e-enabled learning versus classroom) even though the calculation is quite simple (and many suppliers will provide the tools to do the calculation).

However, the 29% that do measure reported significantly improved results over those that based their answers on estimates of the benefits they are receiving. This is interesting in itself. In a survey such as this, there is concern that participants will over- estimate the benefits but this finding highlights the opposite. Those estimating are more conservative.

It is clear that L&D departments are missing a trick. A simple cost benefit analysis is relatively easy to complete and many businesses faced with the challenge of developing a leaner and more efficient organisation may welcome the feedback, particularly when accompanied with reports of sustained or improved quality of learning outputs.

Business agility

So are learning technologies delivering more than just efficiency? Business responsiveness and agility is an increasing priority in times of rapid change. We asked if introducing learning technologies to the mix of learning delivery made a difference to the time it takes staff to reach and prove competence.

Time to competence was not measured by 20% of the sample, but 69% of the remaining 153 participants reported improved times. Most (57%) said competence was reached (and proven) more quickly than before but without quantifying, but an additional 12% were able to comment that staff took half the time to reach time to competence as a direct result of implementing learning technologies.

Businesses need to adapt quickly to survive and while getting staff up to speed faster in new jobs or skills is important, the ability of L&D to respond faster to changing business demands is also critical. In this survey 82% felt they could deliver learning interventions faster, with 26% doing it in less than half the time. Meanwhile 59% reported increased ability to implement changes in products and process and 52% said it improved their ability to roll out new IT systems faster.

These are significant statistics – combining improvements in efficiency and business agility allows us to start to create some powerful value propositions for management teams. So how do managers perceive the value we are adding?

Management perception of value

L&D staff believe their managers agree that e-enabled learning is more convenient (80%), and 55% believe their managers agree that e-learning provides additional business benefit, but beyond that most L&D staff are unclear on more specific details.

Given the desire for most L&D professionals to engage managers and improve business alignment, there is a considerable lack of awareness of what their managers think about the value their technology enabled solutions are having on business priorities:

• 38% do not know if line managers agree productivity has increased (36% are unsure)
• 28% don’t know if new solutions are now more focussed on business performance (33% are unsure)
• 16% don’t know if line managers believe new staff are up to speed more quickly (34% are unsure)

Previous indicators show that when we reflect on the value that learning technologies are adding to the bigger picture, it’s good news. But we aren’t reporting back to managers! The graph shows which successes L&D are most likely to report back to management.

L&D staff are most comfortable reporting benefits of take-up and efficiency in demonstrating compliance – which are important indicators to trainers. But we are less confident in reporting back on the areas that managers value as indicators. This is surprising, since in most situations L&D participants were convinced that the solutions are delivering benefits.

There is clearly a breakdown in communications.

Resolving to influence change

Last month, I outlined five resolutions that we need to keep if e-learning is to achieve its full potential. The last one was influence more, raise management perception of value. We have not done this very well. This has to change. At a minimum we should be:

• Measuring and reporting on what we can – cost benefit analysis comparisons would be a good start!
• Using our follow up surveys more usefully – how many people ask questions about confidence in doing the job, for example (asking for some examples of how they are doing things differently, faster, better).
• Conducting focus groups or one-to-one meetings with managers and learners (particularly those that respond well to the surveys).
• Sharing good news more: anecdotes, quotes, award wins, implications of the numbers etc.
• Using the evidence that we are building up on your behalf (via Becta, towards maturity, e.learning age’s BBB campaign) to help you build your case.

If our impact indicator is a sign of the success we may have this year, then we’ll have something to shout about – let’s makes sure our own staff get to hear about it as well!

1. Download Becta’s Delivering Results report.

2. Download the Towards Maturity Impact indicator.

Laura Overton is managing director of Towards Maturity.