Training evaluation is commonly referred to as a nice to have. However, when we have a look at the amount of money, manpower and time that is invested over a fiscal year to training, we may want to have a clue about the return on investment. Earlier on we introduced four levels of training evaluation – widely applauded for their clarity and straightforwardness. Unfortunately there is another - bigger – obstacle before we can get this ball rolling: process thinking!

Flashback

In an earlier article I have introduced the 4 levels of training evaluation as described by Donald Kirkpatrick. In short, each of these evaluation levels tells you something different.

They are:
1. reaction of student – what they thought and felt about the training
2. learning – the resulting increase in knowledge or capability
3. behavior – extent of behavior and capability improvement and application
4. results – the effects on the business or environment resulting from the trainee’s performance

Process Thinking

Unfortunately, a lot of training managers fail to manage multiple levels and I think I know the reason. These training managers have not yet understood that training evaluation is a process. Just like any other business process there are basic characteristics which determine its effectiveness. The most important characteristics of a process are:

- it describes a repetitive chain of actions that lead to a certain result
- all of these actions are owned by actors who drive the process (in this case: the training manager, the learner and his boss)
- it is spread over time
- it is documented, paced and controlled by a process owner (in this case: the training manager)
- it is vizualised in a swimlane model, which vizualises the responsibility of each actor. This visual documentation is very important to create a shared understanding of the overall process and the extent of everybody’s contribution for all actors involved.

As an example I have drafted a generic training evaluation process on some cardboard last night. Drawing up a swimlane model like the one above is very easy to do. It helps you to get clarity on training evaluations and it is a first step in standardizing and automating your administration.

Why should I care?

Once you start drawing up these swimlanes you will be amazed how many actors are involved (you may end up involving administrators, HR business partners, shop floor supervisors, etc.) As I just mentioned: the cardboard is just a generic example to get you started! You may wonder why on earth a training manager should care about the training evaluation process. There are several reasons for this.

First, it is important to let all the actors involved know that successful training evaluation is a joint venture and not the task of one person one his/her own or one single department.

Second, introducing and communicating a process like this helps to foster ownership over the process by the customer (being: the learner and his/her boss). By forcing them to clarify their WIIFM (What’s In It For Me) in advance and by asking them afterwards if they reached these goals, they gradually become responsible for the learning process.

Finally, this process vizualises the training manager’s reason to exist. Always remember that training evaluation links the training initiative to the performance gap that was observed upfront. This process enables you to standardize the measurement of added value to the customer.

Why so-called Smiley-sheets are important

Donald Kirkpatrick first published his ideas on training evaluation in 1959. His four-level model is now considered an industry standard across the HR and training communities. It was later redefined and updated in his 1998 book ‘Evaluating Training Programs: The Four Levels’.

According to Kirkpatrick, each of these evaluation levels tells you something different. They are:
1. reaction of student – what they thought and felt about the training
2. learning – the resulting increase in knowledge or capability
3. behavior – extent of behavior and capability improvement and application
4. results – the effects on the business or environment resulting from the trainee’s performance

Although there is a lot of discussion going on about the measurability and the isolation of cause-and-effect relationships of level 3 and 4, I would like to point out another problem. It is a problem of arrogance or bad attitude towards the first level.

Now that the Kirkpatrick model has become a standard way of thinking, I see training departments taking their own smiley-sheets not so serious anymore. Reactions are not interesting anymore, ‘because they’re just level 1; no insights to gather there!’ This is a big mistake, for the following reasons:

1. You have created an expectation
In many organizations, feedback gathering is viewed as an isolated activity. They gather feedback but do nothing with the information they’ve obtained. This failure to take action is mostly a major step backwards in building trust because, having been asked for their feedback, participants then watch for changes to take place as a result of their input.
As Naomi Karten states: gathering feedback and taking no action based on the findings is worse than not gathering feedback to begin with. A smiley-sheet creates an expectation for a follow-up action.

2. Training is a Moment of Truth
Reactions (or: satisfaction) will give you an indication of how well the training initiative is perceived. This is more than just smiley sheets about coffee, temperature, and trainer friendliness. The undertone of the wording will give you an impression of the extent to which participants will decide to trust the program. Eventually, trust is the currency of change. in previous posts I have explained that training is most of the times the first real confrontation of participants with their new future. This is an emotional moment that can cause a lot of different reactions (from pure apathy to furious anger).

Looking at the smiley-sheet from a participant’s ‘point of view’ AND ‘point in time’ is essential here. We should take into account that about 90% of the training participants in a change program get a smiley-sheet right after they’ve been told that their wolrd is going to change. With this insight you may want to alter your current smiley-sheets or add a question about the participant’s feelings. I am convinced that spending time on smiley sheet reengineering and disciplined follow-up is a wise investment of attention: it communicates that you care about their reaction.

In short: not only are you creating an expectation for follow-up; you have the unique opening of a slot to demonstrate that you care about the participant’s needs. I’d say that’s pretty fundamental from a customer relationship management (CRM) point of view.