One of the traditional tasks in instructional design is the creation of “knowledge checks” — standard quiz items usually placed in-context with content. The thought behind these types of assessment items is to provide the learner an opportunity to self-assess in sequence, immediately after information acquisition. A common type of item is multiple choice. Multiple choice items can work if you have a great deal of time and understanding of the material, and you are able to construct items that probe higher levels of reasoning. However, too often the reality is that instructional designers without domain expertise write multiple choice items that are irrelevant to real learning, and in many situations, cause more harm than good. Let’s dissect a multiple choice item and probe a bit deeper on why they can be dangerous, and how to make them better.
The inherent issue with multiple choice items like this is the fact that they only test recall, and, in this case, the recall is requested seconds or minutes after the content is provided. And, to impede the process even further, the learner can’t progress until they answer. In this item, the learner is exposed to wrong responses as well as correct ones. The standard four distractors are offered with radio-style buttons: one correct, and three incorrect. The learner scans the list, and makes their choice by clicking in the radio button and then clicking a “Submit” button to receive a response, such as this one:
In this instance, feedback is displayed immediately because the instructional designer has chosen to allow only one try for the item. On incorrect, a “Sorry, that’s incorrect…” statement appears next to a large red “X”. Visual cues are strong, and in this case the type of visual reinforcement and the placement is critical to how useful the item is for learning. When the learner attempts to recall information supported by this item, an unhelpful visual may appear.
For this to be good instructional design the red “X” should be over (or beside) the wrong selected radio button, and the correct answer is should be highlighted. The learner never has a correct visual to overlay the incorrect visual. The learner leaves with a powerful, incorrect visual, instead of a bolder, corrective one. Proper feedback is critical as well. Sometimes learners are just told that the answer is wrong, without being given the correct feedback. Then they return to the item, knowing that whatever thought process or strategy they used was wrong, and can get stuck, trying to remember what their wrong response was, trying to choose the correct answer. If they have to repeat the process multiple times, they do not come away with a strong sense of knowing the correct answer, they instead feel relief that they finally guessed the right answer and were able to progress. In the example above, the incorrect feedback statement is close to the selection, and the correct choice is highlighted with a feedback confirmation next to it on the screen. Additionally, more feedback may be appropriate adding context.
When creating multiple choice items, ask yourself these questions:
- Is your goal just to have learners take courses, or are you trying to ensure they learn something?
- If you need someone to demonstrate mastery of procedures, multiple items may be an inappropriate mechanism for them to demonstrate that they can perform.
If you decide to use multiple choice items to promote learning, here are our recommendations:
- Provide the correct answer after a wrong response.
- Supplant the incorrect visual, with a bolder visual of the correct response.
- Use a tracking system which requires the learner to answer a certain percentage (or all) questions correctly.
- Provide personalized, meaningful feedback in-place on-screen whenever possible.
We all think knowledge checks are innocent enough — important segues in the content sequence — little “breaks” that let the learner pause and think about what they just consumed. This can be a good thing as long as you make sure you’re putting forth the appropriate test item for both the learner and the business. At the end of the day, you don’t want to waste the learner’s time, and you really don’t want to spend precious resources designing learning experiences that don’t have a demonstrable educational gains.
This post was cowritten by Brandon Carson of the Total Learner Experience.
