On Timely Feedback During Test Taking

We've all been there. You go in, take a test, and then wait for the results. And wait. And wait. And eventually, a month or two later when you've completely forgotten about the test, and you've started to move on with your life, the results come back.

This is certainly a common phenomenon for many college and high school students, and for somebody like me, who has a terrible case of test anxiety, it turns my life into something of a living hell. So I got thinking. Is there a reason that people might do this beyond the obvious case of "I don't want to grade 40 papers tonight, I'll do it tomorrow"? Is there perhaps some benefit to delayed feedback that we, as test-anxious students might be missing?

So I started, of course, by googling the problem at hand. "Is timely feedback necessary for students?" And the answers were discouraging for our tardy professors. One site remarked: Research has shown that the sooner students receive feedback after submitting work the more effective it is for their learning.

Ok, I thought. What research? They provided a link to [1], a paper which I was able to find, and read. The only mention in this paper, however, of the necessity of timely feedback was a reference to another paper, [2]. So, like any good researcher would do, I found that paper, and read that one as well. While they did have the following Likert scale:

They didn't really have a lot to say about the problem of instant feedback. In fact, they mentioned the following:

For ... feedback to be effective it needs to be provided early in the learning process (Brown & Knight 1994) and give guidance for improving performance (William & Black 1996, p. 543).

Ok. So two more references. But one of the interesting things to note about this statement is that it came with respect to "Web-Based" courses, and the "Test and rapidly retake" methodology. So really, there's nothing here about how instant feedback really helps students, just a few claims that it needs to be provided early in the learning process, and a few references.

So, given the breadcrumbs that we have, let's follow them - starting with Brown et al. [3]. This work has a more promising title: "Assessing learners in higher education" but it's longer. A lot longer. Actually it's a book. Not only is a book, the google books preview had a promising chapter title: "Chapter 8: Speedier Assessment."

Unfortunately, I wanted to write this blog post tonight, and it's in UCB's NRLF storage facility. Which means I probably couldn't get my hands on it until Monday. Thus, we'll have to do without.

Let's turn to the William & Black article [4], "Meanings and Consequences: A Basis for Distinguishing Formative and Summative Functions of Assessment?" With a title like that, how could it not be delightful?  It turns out there's nothing here. Well, there is, but it's not that helpful - all it says is that feedback should be useful for improving performance. Nothing at all about timeliness.

Thus, we return once again to the original google search. There's plenty of opinion out there:

Feedback should be within 24 to 48 hours. This idea seems overwhelming for a teacher who sees 150 to 200 students in a day. But if students wait too long for feedback, they risk losing the context for the valued learning of the work. Strive to attain this response time as a goal, not a policy. With diligence, successes will happen when students need them most.

(That one is from: https://www.edutopia.org/blog/timely-feedback-now-or-never-john-mccarthy)

Or

In most cases, the sooner I get feedback, the better. I don't want to wait for hours or days to find out whether my students were attentive and whether they learned, or which part of my written story works and which part doesn't. I say "in most cases" to allow for situations like playing a piano piece in a recital. I don't want my teacher or the audience barking out feedback as I perform. That's why it is more precise to say that good feedback is "timely" rather than "immediate."

(This one is from: http://www.ascd.org/publications/educational-leadership/sept12/vol70/num01/Seven-Keys-to-Effective-Feedback.aspx)

But really, it's difficult to find anything that's not based on opinion. Eventually, I found something:

Appropriate and timely feedback was considered to be the weakest area for more than 40% of Business institutions, with feedback noted as being provided too late to be useful. This was only a minor point across the Art & Design institutions, because of their tradition of oral feedback through tutorials (QAA, 2000).

This paper [4] referenced [5], the UK quality assessment in education. But it was our first real set of results. In business institutions in the UK in 2000, it was a problem. However in Art programs at the same time, it was less of an issue but only because other feedback was provided earlier. While it's too early to quit, I think we may finally be on the right track.

And then finally, I found something. "Timing Matters: The Impact of Immediate and Delayed Feedback on Artificial Language Learning" [6]. In this experiment, they taught two grammars, however one group received immediate feedback when they answered questions, while another received a 1s delay. Unfortunately, there's an issue with this. When you get test results back, it's usually much longer than a single second of delay - and nothing is immediate (barring modern web results). Further, it seems like theres another claim to consider:

However, feedback that's too immediate may cause students to rely on teachers for answers rather than persevering and figuring out problems on their own (Bangert-Drowns, Kulik, Kulik, & Morgan, 1991).

Hmm. Perhaps some counter-evidence? In their paper "The Instructional Effect of Feedback in Test-Like Events" [7] they reference another meta-review [8]. Finally - gold. In this meta-review of 53 studies, they split the set of studies into three varieties:

Applied studies measured learning on criterion examinations made up of items different from those used as learning material.

Experiments on Acquisition of Test Content used criterion measures made up of items identical to those used as learning material.

List-Learning Experiments used number of trials to acquisition or number of errors made during those trials as a criterion of learning.

• In applied studies, immediate feedback was usually superior to delayed feedback (average effect size = 0.28).
• In experiments on acquisition of test content, however, immediate feedback was almost always inferior to delayed feedback (average effect size = -0.36)
• In list-learning experiments, effects of immediate feedback were moderately positive (average effect size = 0.34) but highly variable.

In more detail, for Applied Studies:

Nine of the 11 studies of this type found that students achieved more in classrooms where they received immediate rather than delayed feedback. In 4 of these 9 studies, the difference in achievement with immediate versus delayed quiz feedback was reported to be significant. In 2 of the 11 classroom evaluations, examination performance was better when students were taught with delayed feedback, but the gain associated with delayed feedback was not statistically significant.

Yet, when we are working with learning content, in Acquisition we find that:

The 14 studies using this type of measure produced fairly consistent results. In 13 studies, students who received immediate feedback performed less well than did those who received delayed feedback, and in 7 of these studies the result was reported to be statistically significant. In 1 study, students performed better with immediate feedback, and the result was reported to be statistically significant.

Hunh. So maybe it is possible that delayed feedback helps some in certain scenarios. In addition, there's some more surprising results:

Results on follow-up tests were consistent with results on the original measures of learning. The average effect size in the eight studies that used a follow-up measure was -0.44. That means that in the average study retention scores of those receiving immediate feedback were 0.44 standard deviations lower than the retention scores of those receiving delayed feedback.

Yet, there's something to note: These were measures administered between 5 and 7 days after the original criterion measures were administered. 5-7 days seems to be helpful. Yet, what about longer than that? What about an entire semester?

It is from here that I wasn't able to find too many studies testing these effects. The closest I got was this figure from [7]:

What this says is that the effect of the four studies that returned criterion in greater than a week showed decreasing mean effect of feedback on performance. This is keeping with the common opinion that the longer you keep the feedback, the less useful it becomes.

Conclusion

So, after all this, what do those professors have to say for themselves? If you haven't gotten feedback all semester, you're probably in the scientific right - feedback is important, and every study makes that clear. But if you're upset about waiting a few hours, or a few days, science says be patient, the professor might not be lazy after all! Just encouraging you to get higher scores on your next exam :)

References

[1] Irons, Alastair. Enhancing learning through formative assessment and feedback. Routledge, 2007.APA

[2] Wang, Kua Hua, et al. "Learning styles and formative assessment strategy: enhancing student achievement in Web‐based learning." Journal of Computer Assisted Learning 22.3 (2006): 207-217.

[3] Brown, Sally, and Peter Knight. Assessing learners in higher education. Routledge, 2012.

[4] Melanie R. Weaver (2006) Do students value feedback? Student perceptions of tutors’ written responses, Assessment & Evaluation in Higher Education, 31:3, 379-394, DOI: 10.1080/02602930500353061

[5] Quality Assurance Agency (QAA) (2000) Code of practice for the assurance of academic quality and standards of higher education. Section 6: Assessment of students. Available online at: http:// www.qaa.ac.uk/public/cop/ copaosfinal/COP_AOS.pdf (accessed June 2003).

[6] Opitz, Bertram, Nicola K. Ferdinand, and Axel Mecklinger. "Timing matters: the impact of immediate and delayed feedback on artificial language learning." Frontiers in human neuroscience 5 (2011): 8.

[7] Bangert-Drowns, Robert L., et al. "The instructional effect of feedback in test-like events." Review of educational research61.2 (1991): 213-238.

[8] Kulik, James A., and Chen-Lin C. Kulik. "Timing of feedback and verbal learning." Review of educational research 58.1 (1988): 79-97.