Assessments Mean the Difference Between Training and Learning
There’s a quote from George Bernard Shaw that says “the single biggest problem with communication is the illusion it has taken place.” By contrast, the issue with most eLearning is when we think that learning has actually happened. We’ve provided the training so now everything should be better, right? Unfortunately no. It’s not enough to just provide training. We need to know it’s making a difference. That’s where assessments come in.
Using Assessments to Protect Your Investment
Many organizations watch their spending and have limited resources. Perhaps your organization is one of those. As a result, it is very important to make sure that every dollar we spend is getting us something. The popular term is Return on Investment or ROI. We want to know that if we invest a dollar in something that there is some kind of return for it. We want to know we are spending our money wisely.
Why would we then invest money in purchasing or developing an eLearning program and not worry about determining whether it is worth it? Assessments can be one of the ways we determine our ROI.
What Qualifies as an Assessment?
An assessment is a device we use to gather information about our eLearning effectiveness. Usually it’s a quiz or survey. Yet it could be an activity of some sort like performing in a software simulation.
We may also use an assessment to find out how the learner is doing. Do they understand the concepts? Can they remember them? Do they know the process? Have they mastered the material? It may be part of the process for certifying someone on a skill or knowledge set.
Types of Assessment
There are different types of assessments we can use. They vary based on complexity as well as the investment required to carry them off effectively. Kirkpatrick’s Four Levels of Evaluation best describes the types of assessments for evaluating the learner. The model was originally conceived by Donald Kirkpatrick in the 1950’s and was improved over time.
In essence, Kirkpatrick’s model arranges assessment types ranging from essential to analytical.
Do They Like It?
Have you ever taken a course and immediately after you were asked to complete a course evaluation? This is a Level 1 Evaluation. Checking the learner’s reaction to the content of the course and whether it was helpful. Also typically there will be questions about the method of delivery or the instructor. This type of evaluation is more a survey than anything else. Most everyone does this, however, because it is extremely low cost to implement and gather data.
Did They Learn Something?
On Level 2 we give them a quiz to determine if they retained any of the material that was covered. This also is typically done either during the course or at the end of the course. As a result, it really only checks the short-term memory of the learner. Yet that’s a good start. Retention finalizes when we move information from short-term to long-term memory. So making sure it is in short-term memory helps.
Did It Change Behavior?
A major goal of any learning program should be making change or improvement. Is the learner performing a skill better? Has their customer service improved? Are they thinking differently? This is the Level 3 evaluation.
As much as that sounds critical, ironically most organizations don’t really evaluate this. Why? Because it requires more time and expense. It usually means measuring the targeted behavior before training and then measuring it again a few months after the training. That means time and investment in gathering the data and analyzing it.
Is the Organization Better as a Result?
Ultimately, Level 4 is what we are looking for. If we provide an eLearning course on how to handle difficult customers we should be able to see positive results from that. So not only would we want to see behavior change in the employees but we would want to see impact on the company as a whole. Has customer satisfaction and retention improved? Is employee turnover lower?
As you might guess, that means gathering the numbers before training is implemented. After training, gathering data and tracking results for at least a year afterwards. Again, time and expense that very few organizations are willing to go through.
Summative and Formative
When we apply a Level 2 evaluation, they are typically for one of two purposes. They are either SUMMATIVE or FORMATIVE.
A Formative evaluation is meant to provide remediation. In other words, we insert some quiz questions in the middle of the course, for example after each topic or lesson section. The idea is to find out if they understood what we just covered. If not, it’s easy to provide some immediate correction that also reinforces the learning.
A Summative evaluation is when we administer a quiz at the end of the course. Our primary goal here is to determine when they mastered all the content of the course. This type of quiz is usually scored and the results stored on the Learning Management System. Further, they are considered pass/fail.
It’s not unusual to use both types in a course. Offer them as much help in the learning as we can and then determine later whether that learning has been achieved.
Matching Up with Objectives
For eLearning, if we are trying to determine mastery of conduct there is a very critical key. We have to make sure the questions in the assessment line up with the SCO’s and objectives we laid out when we designed our course. Every SCO and/or objective must be covered by some of the assessment questions. Without that, we cannot reliably say that mastery was achieved. That also means plans for assessments should be included in the design phase of course development.
If you are going to invest in eLearning, it pays to invest in making sure it works. Assessments are your best bet for that.