Summaries, Responses, and Short Answers…oh my! – Using Student Samples in Writing Instruction

Last year at the 2016 SETESOL conference in Louisville, Kentucky, I attended a presentation on using student samples in summary writing instruction. The presentation was given by Dr. Cui Zhang, and it consisted of a literature review and her own action research. I was intrigued by the idea because, unlike peer review where effectiveness is hit or miss and the focus could be on anything from grammar to structure, analyzing student examples allows for the precise identification and evaluation of specific aspects of a writing. This type of analysis allows students to see various ways students were able to successfully or unsuccessfully achieve a specific goal, one which they also have attempted. I recently incorporated Zhang’s ideas with not only summary writing but also response writing and short-answer writing, and I saw immediate positive results in student revisions and subsequent writing. Therefore, I wanted to share these ideas with you.

What Does the Research Say?

  • Baba, 2009: Reading comprehension plays a large role in successful summary writing, while the role of lexical proficiency varies. However, “well-structured semantic network of words and the ability to productively use this network as well as the L2 writer’s metalinguistic knowledge” also has an influence.
  • Keck, 2006: L2 writers paraphrase less and copy more of a source text than L1 writers.
  • Demaree et al., 2008: Students feel that summary writing is useful, and it is better done when there is an authentic purpose (such as preparing for an exam). Students feel the only summary writing audience is themselves and it is not very helpful for others.
  • McDonough et al., 2014: Summary writing improves over time, but requires explicit instruction and may be a lengthy process. The authors looked at reference to the source (increase), verbatim copying (increase in frequency, decrease in length), a “phrase-level modifications” (no change). According to the authors: “the path toward eliminating textual misappropriation may be both indirect and lengthy.”
  • Becker, 2016: Students who develop or practice applying a rubric show greater increases in summary writing performance.

Zhang’s Action Research

The goal of Zhang’s research was to see if students could reliably judge summaries written by their peers and then use these judgments to improve their own summary writing. Zhang worked with 9 students in a university-level ESL course. After a text was read and summaries were written, Zhang collected the summaries and chose several for analysis. They analyzed the summaries without a rubric and discussed their judgments. They read a second article and then produced another summary.

Overall, Zhang found that students could all find the weaknesses or strengths in the summaries and their own summary writing did improve, though not to the point of perfection. She recommends that summary improvement will take time. She also recommends that using previous students’ writing rather than writings from the current students may reduce some reluctance to judge their peers.

My Experiences

Summaries – In my classes, I followed a similar procedure for this article. However, I used a modified checklist rubric to help students evaluate the summaries. I gave students a handout with 4 summaries collected from students. They were modified for clarity (grammar, spelling) and were chosen because they represented very poor, fair, and great summaries. Here is an example (note: the bullets on the right were actually check boxes):

Although sleepiness is a part of life, it seems difficult for schools to start school late. The students can change their schedule to get enough sleep. Whether someone likes it or not, adequate sleep is important for our lives, and it’s especially necessary for children. The more sleep, the healthier and happier life people will have. This summary…

  • …introduced the article and the author.
  • …contained the overall main idea in the second sentence.
  • …contained all the main ideas:
    • not enough sleep
    • they are busy, puberty,
    • school should start later
    • starting school later is difficult
  • …had no extra details.
  • …had no change of meaning
  • …was written in the student’s .own words.

From the four different examples, most students were able to identify the best summary and understand what it had that the others were lacking. After the group discussions, a class discussion of each summary entailed, each time highlighting the elements that were missing or included. This was an attempt to be explicit and reinforce what a good summary contained.

After this activity, students revised their summaries. About a week later, they also wrote new summaries, and for many I saw great improvement. In particular, there were more references to the original source text (According to [author], in [title],…) and less verbatim copying. However, there were still issues with including main ideas and excluding irrelevant details. This showed me that being able to identify what is important was something that needed to be focused on more in class.

Responses – A summary is a pretty straight forward genre that requires students to simply retell important details using new words. Responses, on the other hand, are more varied in terms of content. With only minor directions (“Give your evaluation of the article”) and no instruction, student responses to this article went from clear evaluation of the original text to complete departures and explorations of students’ own, often unrelated, opinions. I saw another chance for students to analyze student samples and improve their writing.

Since no students completed the assignment correctly, I collected 3 student samples and wrote a fourth. I then created another check list rubric that students could use to evaluate the articles. Students discussed the responses together and then we discussed them as a class. Here is an example from the handout:

I agree with the PRO statement that people should eat less meat. First, eating less meat is healthy for us. People will be less obese and avoid disease. Second, we should eat more fruit, vegetables and cereals. These plants need to use machines and they need to use the power. People should use more solar energy, wind energy and water energy to generate electricity. Finally, we should plant more trees because the trees can help reserve the water and prevent soil erosion.
  • The student refers to a claim in the article
  • The student states whether they agree or disagree with a claim from the article.
  • The student gives reasons why they agree or disagree.
  • These reasons show good evaluation of the claim.
  • The reasons focus on the claim and not unrelated ideas.

For this rubric, I was trying to direct students to the fact that a response to an article is not simply an opinion of the topic but an analysis of the ideas contained in the article. In other words, the focus should still be on the article, not only the student’s opinion. And even when the opinion is given, it must be clearly related to the ideas in the article. This seemed like the first time students encountered such an assignment and the evaluation clearly – hence me writing a fourth example. As with the summaries, student revisions and subsequent writings showed some improvement.

Short Answer – Seeing a pattern in students writing and their familiarity with writing assignments, I preempted difficulty with short answer writing assignments and gave students explicit and step-by-step instructions in both understanding the question and writing the answer. Working with this text on driverless cars, we first looked at the default writing prompt from Newsela:

Summarize the central idea of either the PRO or the CON article in a few lines. What claims made by the author of the chosen article are not supported by evidence? Give two-three examples from the text to better illustrate your point.

We analyzed this assignment by breaking it down into parts:

  1. In your own words, write the main idea of the PRO or CON article in a sentence or two
  2. Answer this question: What claims are weak because they lack evidence?
  3. Answer this question: What are two or three examples that show there is a lack of evidence.

Students seemed genuinely surprised that the question was very complex. Therefore, this question analysis proved to be very valuable. We then discussed how to answer this question in a paragraph and wrote a model answer together. For homework, I had students consider the driverless car article as well as this article about a horseless carriage. I then gave them the choice of answering ONE of these questions:

  1. What similarities exist between horseless carriages and driverless cars? Provide two or three examples from the text to help support your point.
  2. How do technological advances like new types of automobiles affect everyday life? Use one or two examples from each article to explain past or future changes.
  3. Do you think the author of this article would share similar opinions (or tones) as the PRO or CON author? Provide two or three examples from the text to help support your point. (no students answered this question).

I collected the student examples in the next class and redistributed them to students individually. I then gave each student the following rubric:

QUESTIONS/CRITERIA POINTS
Did they try to answer all parts of the question?
Only one part: 1 point | Both parts: 2 points
Did they provide evidence from both articles?
Only one article: 1 point | Both articles: 2 points
Did they do a good job answering the question?
Yes (3 pts) No (1 pt) Maybe (2 pts)
(please explain on the back of this paper)
Did they use phrases such as “according to” or “the author states”?
Yes (1 pts) No (0 pts)
Did they give extra details that were unnecessary?
Yes (-1 pt) No (1 pt)
Did they write a summary?
Yes (-2 pts) No (1 pt)
Did they give an opinion that was unrelated to the questions?
Yes (-2 pts) No (1 pt)
Was the answer easy to understand?
Yes (1 pts) No (0 pts)
TOTAL  __ / 12

Students had about 20 minutes to read and analyze the answer they were given. I assisted students with answering questions, and I prompted students to leave clear feedback on the back of the paper. As students worked, I made sure their analyses were accurate, and if I disagreed with a student, I asked them to provide justification for me. Sometimes I had to gently nudge students to fix their analysis because they had clearly misunderstood something. However, more times than not, students noticed something that I had overlooked.

After the 20 minutes, I collected the answers and the rubrics and redistributed them to the appropriate students. I then gave students the rest of class to revise their answers, if necessary, and ask me any questions to clarify or improve their writing. For most students, there was immediate improvement. On a subsequent reading test that involved a short-answer question, I saw more answers that fully answered all parts of the questions, something they had been previously lacking.

Overall Impressions

I found that getting students to analyze student samples was very effective at not only understanding what good writing should contain, but also at helping to clarify writing expectations, something that is often hard to communicate, especially with unfamiliar genres or complex assignments. For most of these assignments, I provided rubrics beforehand, but students often do not pay attention to them. However, even if students had focused on them, I believe that providing rubrics afterward, focusing greater applied attention on them, and then allowing students to revise their writing could have a great positive impact on their writing.

References

Baba, K. (2009). Aspects of lexical proficiency in writing summaries in a foreign language. Journal of Second Language Writing, 18(3), 191-208.

Becker, A. (2016). Student-generated scoring rubrics: Examining their formative value for improving ESL students’ writing performance. Assessing Writing, 29, 15-24.

Demaree, D., Allie, S., Low, M., & Taylor, J. (2008, October). Quantitative and qualitative analysis of student textbook summary writing. In C. Henderson, M. Sabella, & L. Hsu (Eds.), AIP Conference Proceedings (Vol. 1064, No. 1, pp. 107-110). AIP.

Keck, C. (2006). The use of paraphrase in summary writing: A comparison of L1 and L2 writers. Journal of Second Language Writing, 15(4), 261-278.

McDonough, K., Crawford, W. J., & De Vleeschauwer, J. (2014). Summary writing in a Thai EFL university context. Journal of second language writing, 24, 20-32.