What Do You Know: What Should We Include in Feedback?

In my last post about feedback, I discussed how research shows that feedback can improve learning. But it doesn’t always. Hattie and Timperley (2007) explain that feedback helps learning most when it deals with three specific learning problems: 

1. Misunderstandings. For example, people may think that saving is important, but that it doesn’t matter when you save. (Wrong! It matters because of compound interest!) 
2. Forgotten information or strategies. For example: Which is more important? Saving for retirement or paying off debt? (It depends on a number of factors.) 
3. Performance difficulties. For example, giving in to “tricks of the trade” may increase what you pay without much benefit (such as glass etching on windshields and extended warrantees on most products). 

Now that we know the kind of feedback that most improves learning, let’s discuss what research tells us we should include in feedback. But first, review the following question and select the answers you think are correct. 
Which of the following is valuable as feedback? Select all that apply. 

A. Whether the answer is correct or incorrect 

B. (If incorrect) Try again? 

C. The correct answer 

D. Hints to improve recall of the correct answer, such as applicable rules and facts 

E. Hints on useful strategies, typical misunderstandings, and sources of typical mistakes 

F. Analysis of errors made (what is the misunderstanding or source of mistake) 

G. Where training discussed this information H. Worked examples showing how to get to the correct answer, such as steps or rationale 

I. Interesting tidbits to motivate use of feedback 

Do We Need Feedback? 

Writing proper assessments and feedback is time-consuming. Many designers tell me that they have difficulty with these skills. It’s understandable that people would love to get rid of these tasks. But when these tasks are done well, they add a great deal of value to instruction. (I will write about interesting outcomes from testing soon.) 

Feedback helps when it offers the right information at the right time and in the right way. In the rest of this post, we’ll discuss what to include in feedback. and a little bit about how to offer it. This information is critical for learning designers and instructors because bad feedback can harm learning. And no feedback leaves people stuck with whatever misunderstandings, forgotten information, and difficulties they have. Most learning builds on prior knowledge, so not making sure that people “get it” is simply not OK. 

So yeah, valuable feedback is kind of important. 

What Should Feedback Do? 

According to Narciss and Huth (2004), whose chapter in Instructional Design for Multimedia Learning I am discussing in this and the following sections, the most important function of feedback is to help participants understand and control their own learning. In other words, feedback should help people assess what they know, don’t know, and need to know. We can add this to Hattie and Timperley (2007)’s feedback goal of reducing the gap between existing knowledge and skills and needed knowledge and skills. 

Narciss and Huth (2004) offer us a way to think about types of feedback that I found quite helpful. The two types are quantitative and qualitative (they used different terms but I’m using terms that I think are easier to understand).  

• Quantitative feedback tells people how many. It supplies numbers or percentages, such as how many correct and incorrect responses, what percentage of answers are correct and incorrect, and what percentage of content mastered.
• Qualitative feedback tells people why. It supplies information to improve understanding of the topic, task, error, or solution. 

Table 1 discusses the different content elements we might include in quantitative and qualitative feedback. 
Table 1. The Value of Different Feedback Content 

	Feedback Content	Valuable? Why?
quantitative	Whether the answer is correct or incorrect   Advertisement Incorrect	Correct or Incorrect is more valuable to those with incorrect answers. But it isn’t enough information.
	Try again   That is the incorrect answer. Try again.	Try again too often allows people to guess at another answer or keep trying until they get the correct answer. Unless a different question is presented that tests the same information, this typically isn’t valuable.
	The correct answer   The correct answer is B. Ask a manager to sign off on the withdrawal.	Supplying the correct answer tells which answer is correct but leaves out why.
	Quantitative feedback content rarely offers adequate information to close knowledge and skills gaps. It doesn’t help people improve their ability to think through their understanding and how to fix it. We can consider most quantitative feedback as insufficient.
qualitative	Hints to improve recall of the correct answer, such as applicable rules and facts   Hint: We discussed two different approaches for deciding what order to pay off debts in module 2. Do you remember the order suggested for those with greater than $20,000 in debt?	The types of qualitative feedback content that can be valuable for helping people do one or both of these: Bridge the gap between current knowledge and skills and needed knowledge and skills. Help people assess what they know, don’t know, and need to know.   Which qualitative feedback elements to choose depends on which problem the wrong answer reveals: misunderstanding forgotten information or strategies performance difficulties.   It is our job to analyze which of the three types of learning problems the incorrect answer reveals. Then we can tailor feedback content to that specific problem. Too much information risks overwhelming the learner and reduced effort. Too little information risks not fixing the problem.   Narciss and Huth (2004) list research findings that help us select the best combination of feedback content. Advertisement Don’t provide any feedback before people have tried to solve it on their own. When we offer feedback too soon, we often reduce effort and learning. Don’t provide the correct response and more information (qualitative) feedback together. When we do this, we often reduce effort and allow very shallow learning. Provide qualitative feedback in manageable chunks. This allows people to take in as much as they can handle before taking in more information. Reduce cognitive overload with complex feedback. For example, if you are showing how something works, explain using audio rather than in text. (I’ll explain media effects another time.) Point to the relevant aspects of the screen as needed.
	Hints on useful strategies, typical misunderstandings, and sources of typical mistakes   Hint: There are some common mistakes people make when creating a budget. You left a key category out of your list. Do you remember which category you need to add?
	An analysis of errors made (what is the misunderstanding or source of mistake)   There are some common mistakes people make when creating a budget. You didn’t add a category for unexpected maintenance. Add it now?
	Where training discussed this information   We discussed information about unexpected maintenance in the fourth section. Would you like to revisit that section now before answering more questions?
	Worked examples showing how to get to the correct answer, such as steps or rationale   In the section on creating a budget, we offered six examples of budgets for families with different needs and incomes. Would you like to review them now?
	Interesting tidbits to motivate use of feedback	Don’t include! Extra information is likely to result in overload (extraneous cognitive load).

So what is the answer to the question I posed? Everything except I. To see if you understand this post, I’ll ask the following questions: 

1. What are the two goals or purposes of feedback? 
2. Quantitative feedback is insufficient feedback because____________________________________. 
3. Qualitative feedback should address __________________________________________. 

References 

Hattie, J., and H. Timperley. 2007. “The Power of Feedback.” Review of Educational Research 77(1): 81-112. www.columbia.edu/~mvp19/ETF/Feedback.pdf 

Kulhavy, R.W. 1977. “Feedback in Written Instruction.” Review of Educational Research 47(1): 211-232. http://journals.sagepub.com/doi/pdf/10.3102/00346543047002211 

Narciss, S. 2008. “Feedback Strategies for Interactive Learning Tasks.” In Handbook of Research on Educational Communications and Technology, 3rd edition, edited by J.M. Spector, M.D. Merril, J.J.G. van Merriënboer, and M.P. Driscoll, 125-144. Mahwah, NJ: Lawrence Erlbaum. www.phil.uni-passau.de/fileadmin/dokumente/lehrstuehle/narciss/Narciss__S.__2008_._Feedback_strategies_for_interactive_learning_tasks.pdf 

Narciss S., and K. Huth. 2004. “How to Design Informative Tutoring Feedback for Multimedia Learning.” In Instructional Design for Multimedia Learning, edited by H.M. Niegemann, D. Leutner, and R. Brunken, 181-195. Munster, NY: Waxmann. www.studierplatz2000.tu-dresden.de/toolkit/presentations/CD/Literatur/Publikationen/ID_nahu.pdf