Research Bites: Watch Out Washback!

High-stakes testing policies, such as those in the USA related to No Child Left Behind and the Common Core, have left a stale taste in the mouths of many educators, parents, and students. The problem here is that there has been a educational shift to “teaching to the test.” In educational literature, especially applied linguistics, this is called “washback.” It used to be called “backwash,” but that was probably too bold a word to call something that was becoming an everyday occurrence in classrooms around the world. It was probably best to stick to a more palatable word.

I’m writing about washback because I have been thinking about my students’ needs lately. In particular, a special group of students in my program have been struggling to pass requirements to enter graduate school within a certain period of time before funding runs out. They constantly spoke to me about wanting more TOEFL or IELTS practice, or they lamented about our program not preparing them for the tests.

My argument was that we are not a test-prep program; we are preparing students for the academic rigors of university studies, and, the skills we are teaching are by extension applicable for their tests. They weren’t buying it. And after a while, I wasn’t sure about it either because they had seemed to hit a testing ceiling. I was (and still am to some degree) not entirely sure about the best ways to help them.

However, in terms of their complaints about the focus of our program, it turned out my logic was dead on. The conclusion from the study summarized below states that, whether students are in a general EAP course, an IELTS test-prep course, or a mixed course, all students will gain and there are no significant differences between course types. In addition, there are other predictive factors that need to be taken into account.

Twitter Summary

Study shows that students gain the same whether in test-prep or EAP courses. #researchbites http://www.anthonyteacher.com/blog/researchbites/research-bites-watch-out-washback

Article

Green, A. (2007). Washback to learning outcomes: a comparative study of IELTS preparation and university pre‐sessional language courses. Assessment in Education, 14(1), 75-97.

Study Design

  • Participants
    • UK-based
    • Type 1 Course: IELTS prep courses at 7 institutions totaling 85 students
    • Type 2 Course: “pre-sessional EAP” courses at 3 institutions totalling 331 students (Type 2)
    • Type 3 course: combined IELTS/EAP courses at 5 institutions with 60 students total
    • all courses were 8-9 weeks, between 21-24 hours per week
  • Main Instruments
    • Pre/Post IELTS writing tasks 1 and 2
    • Questionnaire to collect “participant variables” such as nationality, motivation, etc before taking the test
    • Questionnaire to collect “process variables” such as learning strategies, use of English outside of class, self-assessed gains, etc.
  • Analyses
    • T-Tests were used to check for gains within groups
    • MANOVA was used to check for differences between groups
    • neural network analysis was used to analyze the correlations between the participant and process variables, as well as to create an interactive predictive model based on these variables
  • Major Findings
    • Students had significant gains in all three course types.
    • There were no significant differences between the course types.
    • The average gain was 0.2 of a band.
    • Those who began with lower levels made the greatest gains.
      • Conversely, those starting with higher levels made lower gains, which indicates a possible test ceiling or plateau effect.
    • Participant variables tend to predict more success than process variables
      • Participant variables accounted for 45% of the variance of scores
      • Process variables only accounted for 31%
    • Variables that contribute most to prediction include:
      • low initial writing and grammar scores
      • longer courses
      • educated beyond secondary level
      • believed that they were good at learning to write in English
    • Variables that contributed somewhat (but perhaps not in any meaningful way) include:
      • positive orientation to host culture
      • test-taking strategies

Implications/Conclusions

In 1993, a seminal study on testing and teaching asked “Does washback exist?” (Alderson & Wall). The research has been very clear that it does, and that there are both positive and negative types of washback. Green’s research asked something akin to “Does incorporating washback create greater gains?” The answer: yes BUT no more than other courses that prepare students for academic success. This finding is also not unique: Perrone (2010; cited in Cheng, Sun, & Ma, 2015) had the same results with students studying for the FCE.

What does this mean in the classroom? Well, for one, I can’t print Green’s article and make them dry their tears with it. That won’t do any good. I could explain the research, but would it convince them? I don’t think so. What I can do, is keep working on building strong academic skills. IELTS tests such a narrow range of skills and these skills are by nature covered in EAP courses (e.g. data analysis [task 1], argumentative writing [task 2]) – or, at least should be.

Nevertheless, student needs, perceived or real, are something that must be addressed in the classroom. From Green’s research, here is a situation where I am neither damned if I do nor damned if I don’t. This leads me to believe it is OK to satisfice: to keep providing EAP while integrating more test-focused practice so long as it is principled, that is, is meant to bolster proficiency not test scores. Following this choice is a detriment to no one and allows students to feel better prepared for their exams.

And so, this begins the introduction to a series of posts I’m going to call “Principled Washback” wherein I will introduce an activity or idea that can be used to integrate EAP and test-focused practice.

References

Alderson, J. C., & Wall, D. (1993). Does washback exist?. Applied linguistics,14(2), 115-129. Retrieved from http://files.eric.ed.gov/fulltext/ED345513.pdf.

Cheng, L. Sun, Y., and Ma, J. (2015). Review of washback research literature within Kane’s argument-based validation framework. Language Teaching 48(4), 436-470. [$link]

Perrone, M. (2010). The impact of the First Certificate of English (FCE) on the EFL classroom: A washback study. Unpublished doctoral dissertation, Teachers College, Columbia University. [$link]

2 thoughts on “Research Bites: Watch Out Washback!

  1. Thanks for this post, Anthony. I really like the following line: “This leads me to believe it is OK to satisfice: to keep providing EAP while integrating more test-focused practice so long as it is principled, that is, is meant to bolster proficiency, not test scores.” I totally agree. I’m currently preparing a group of students for a high-stake exam and I constantly question the quality of the course. Not that my students have complained; I just had a very unpleasant experience in the past when a student gave me a negative feedback after a course, saying that “The teacher didn’t include enough of this or that type of exam practice”. She added: “Had I not attended my extra evening courses, I wouldn’t have passed the exam”. This really hurt and I think that’s why I’m more cautious now. I’ve actually become somewhat paranoid and I try to make sure that my students are getting plenty of exam prep and, more importantly, that they know that they are getting the prep. However, I do my best to design the activities so that they bolster proficiency, as you mentioned, and I think it’s the best strategy.

    • Anthony Schmidt says:

      Sorry for the very late reply Hana. First, thank you for reading and commenting on my post. I’ve had that experience too, which is why I began investigating this in the first place. As teachers, we may rail against tests, but we also have to accept that they are a reality, so we need to address them in the classrooms. But, as you said, it’s important “that they bolster proficiency.” I couldn’t agree more.

      Do you have any tips and tricks for integrating test practice into the classroom? Do you set aside test prep time?

Leave a Reply