For those who are more story-driven, data analysis can be a challenge, to the point where it can become difficult to even know what numbers to explore. Yet, plenty of material shows that stories can be told through data — you just have to be diligent in your exploration to find them.

Diligence means caring and being conscientious. If you care about the story, you have to care about the data and be intentional about which data you use to tell that story.

Finding the story that shows how effective training is a journey. It begins with providing new knowledge and developing new skills, continues as employees apply and retain those skills over time, and reaches a “happy ending” when learning transforms how they work and who they become.

Using a Learning Transfer Effectiveness Index (LTEI) enables us to track all three stages of this story.

Stage 1. Measuring Skill Acquisition (SA)

Our story begins in a classroom, with learning objectives, eye-opening experiences and engaging activities. To support employees on this journey, we first need to know where they are starting.

A great way to do this is through a pre-test. Pre-tests communicate course objectives in formal training and, when compared with post-tests, show the knowledge or skills gained over the course of the training.

Pre-tests can also be used for hands-on training through the use of simulated environments, or even an observed first attempt. The key is having an effective rubric. Clear scoring and expectations need to be established, including automatic fails if critical steps are missed or safety is compromised. Scoring a first attempt as if it were a final attempt gives you a true baseline for skills acquisition.

Compare post-test scores to pre-test scores relative to the maximum possible improvements to get your first metric for your LTEI score:

SA= (Post-Score – Pre-Score) / Maximum Possible Improvement

With this, we complete our first chapter. Next, we leave the classroom to see how well learners retain their newfound knowledge and skills.

Stage 2. Measuring Skill Retention (SR)

Retention measurements are incredibly useful as both markers of past training success and indicators of areas that need improvement. The key is to keep it simple. You’re not testing new knowledge, you’re checking how well existing skills are retained.

Use the same tools as in the SA phase but vary written assessments to ensure you’re testing true knowledge retention, not memorization. For example, create quizzes that randomly pull from a larger question bank or vary the question order.

For hands-on assessments, the same rubric used for SA can be applied. The most important factor for measuring SR is observing retention in the regular flow of work. This is also a great way to recognize decision-making and task competence, both identified by Thalheimer’s LTEM tool as effective measurements of training.

Compare current evaluation scores to those at initial acquisition:

SR = Current Score / Score at Acquisition

This data enriches our story because it builds directly from the first chapter.  Employees may initially be reluctant about retention evaluations, so communicate the value clearly and, when possible, incentivize participation.

Stage 3. Measuring Skill Engagement (SE)

In this chapter of the learning transfer story, we look for the answers to two questions: What is the impact on productivity and quality, and how useful are these skills to the business? This is where we discover the true impact of our efforts.

While the data to answer these questions may not come from the same source, you can find meaningful connections through correlations with SA and SR data. SE data can be a bit more complex, but the good news is that much of it is already being collected through customer satisfaction surveys, production output, defect rates or other performance metrics.

The challenge lies in how you examine this data. Start by examining productivity and quality separately. An employee may be highly productive but make frequent errors, or the opposite. Measuring both provides a more accurate view of performance

Once aligned with expectations, combine these two data points for your SE score:

SE = (Skill Usage Instances / Expected Usage Instances) x (Actual Quality Rate / Expected Quality Rate)

Individually, these metrics offer valuable information to help us plan for immediate and future training needs. But the real magic happens when we view them together.

The Whole Picture

Have you ever read a story and when you got to the end you thought, “I didn’t see that coming,” but then you read it again, and you noticed all the signs? That is the goal of the LTEI.

If an employee’s productivity is low or is struggling to meet quality expectations, we can look back through the data to find the cause. Perhaps a gap in skill retention or limited growth from the starting point. This allows us to better support that employee.

Even better, if we catch issues early, at the point of acquisition, we can prevent future problems and track each employee’s growth with confidence. That’s the power of the LTEI.

The only thing left to decide is how to weigh each component in your overall score. The weights should equal 1.0 or 100%, but it will depend on your training organization’s goals and areas of influence.

For example, if you have the strongest direct influence over SA, but only slightly less on SR and significantly less on SE, you could break it down as follows:

SA=0.45 (w1), SR=0.4 (w2), SE=0.15 (w3)

The final formula is as follows:

LTEI= w1 x SA + w2 x SR + w3 x SE

Conclusion

Formulas and spreadsheets can still feel overwhelming, and measuring training impact can still feel like sifting through a multilayered story with interweaving plot lines and complex character interactions. The Learning Transfer Effectiveness Index doesn’t eliminate that complexity, but with diligence, it helps us focus on the parts of the story that matter most.

With care and conscientious use of data, you can reveal the real story behind learning, turning bystanders into heroes and insight into meaningful impact.