Trainers: Are Kirkpatrick Level 1 Surveys Necessary?

Recently at a conference I heard a presenter bring up an interesting point; are level one survey necessary? Kirkpatrick level one surveys are the surveys that are administered to students during or at the end of training. They are sometimes also referred to a 'smile sheets'. These surveys commonly ask questions question on instructor performance, facilities, and overall curriculum development.

The argument against administration of level one survey was that in the corporate world, the training department is usually the only department that assesses their performance by surveying their customers. This may cause instructors to get less than favorable performance marks based on student feedback that is biased or ill-informed. Moreover, instructors may be more conscience of the feedback from students and may tailor their performance to meet student needs instead of performing to the set standard of the corporation.

This logic is somewhat flawed. I can understand the frustration of the training director. The problem is not with the administration of the surveys or with the feedback being received, but with the administration of this feedback.

There is a reason Kirkpatrick designed the evaluation system- it's because it is a complete system and without administrating any one segment of the system- the system is incomplete. Level one surveys are just the first step in evaluating the course. Level one surveys are just the student's reaction to the course. The training director and administration has to understand that these surveys may be biased. If an instructor comes in every morning with fresh coffee and doughnuts for the students, jokes around and lets students leave early, without covering the required material, that instructor will most likely get great feedback from the students who had a great time. Another instructor who covers all the material, does not bring doughnuts and coffee and makes students work late to ensure material is covered, may get less than favorable evaluations. Who is the better instructor? If you were the training director reading the surveys, you would assume the instructor with fresh doughnuts. It wouldn't until other evaluations were completed (level two - assessments, and level three - transfer) that the picture became clearer.

This is why instructor performance evaluation marks cannot and should never be based on level one evaluations only. If your company does yearly performance marks, the instructor's delivery needs to based on trend analysis, direct observation by the training director, and student performance (through level two and three). Basing instructor performance marks solely on one source of data and unfair to the trainer and the organization.

Finally the comment that the training department is the only department that assess itself based on customer feedback displays a lack of communication in other departments, not a fault in the training department. Other service departments such as Human Resources, Employee Support (cafeteria, uniform maintenance, etc.) should all have some sort of level one feedback assessment. Other branches of the corporation have their own source of feedback based on their role in the organization. For instance operations feedback would be on output, maintenance would be on equipment downtime. That is the role of these departments, not customer satisfaction. If they do receive feedback from their customers, then that is a bonus. That being said, the role of departments may be different in different companies, therefore they may have different mean of measuring themselves.

In conclusion, level one assessments are necessary. These surveys need to ask the correct questions and students need to fully understand that their input is valuable and needs to be unbiased. Training managers need to understand that some surveys will remain biased and this should not reflect on the trainers. Trend analysis has to be conducted and properly evaluated.