MarkWrite : standardised feedback on ESL student writing via a computerised marking interface

Abstract

The research reported on in this thesis forms part of the foundation of a bigger research project in which an attempt is made to provide better, faster and more efficient feedback on student writing. The introduction presents the localised and international context of the study, and discusses some of the problems experienced with feedback practice in general. The introduction also gives a preview of the intended practical implementation of the research reported on in this thesis. From there on, the thesis is presented in article form with each article investigating and answering a part of two main guiding questions. These questions are: 1. Does feedback on student writing work? 2. How can feedback on student writing be implemented as effectively as possible?

The abstracts for the five individual articles are as follows: Article 1 Article 1 presents a rubric for the evaluation of Computer-Assisted Language Learning (CALL) software based on international recommendations for effective CALL. The rubric is presented after a brief overview of the pedagogical and implementation fundamentals of CALL, and a discussion of what needs to be included in a needs analysis for CALL evaluation. It is then illustrated how the evaluation criteria in the rubric can be used in the design of a new CALL system. Article 2 Providing feedback on student writing is a much-debated topic. One group of researchers argues that it is ineffective and another group remains convinced that it is effective, while at ground level teachers and lecturers simply carry on “marking” texts. The author of this article contends that both arguments have valid contributions to make and uses the arguments both for and against feedback to create a checklist for effective feedback practice. Adhering to this checklist should counter most of the arguments against feedback while supporting and improving the positive arguments in favour of feedback. Article 3 This article reports on an experiment which tested how effectively standardised feedback could be used when marking L2 student writing. The experiment was conducted using a custom-programmed software tool and a set of standardised feedback comments. The results of the experiment prove that standardised feedback can be used consistently and effectively to a degree, even though some refinements are still needed. Using standardised feedback in a standard marking environment can assist markers in raising their awareness of errors and in more accurately identifying where students lack knowledge. With some refinements, it may also be possible to speed up the marking process. Article 4 This article describes an experiment in which Boolean feedback (a kind of checklist) was used to provide feedback on the paragraph structures of first-year students in an academic literacy course. The major problems with feedback on L2 writing are introduced and it is established why a focus on paragraph structures in particular is of importance. The experiment conducted was a two-draft assignment in which three different kinds of feedback (technique A: handwritten comments; technique B: consciousness raising through generalised Boolean feedback; and technique C: specific Boolean feedback) were presented to three different groups of students. The results indicate that specific Boolean feedback is more effective than the other two techniques, partly because a higher proportion of the instances of negative feedback on the first draft were corrected in the second draft (improvements), but more importantly because in the revision a much lower number of changes to the text resulted in negative feedback on the second draft (regressions). For non-specific feedback, almost as many regressions occurred as improvements. In combination with automatic analytical techniques made possible with software, the results from this study make a case for using such checklists to give feedback on student writing. Article 5 This article describes an experiment in which a series of statements, answerable simply with yes or no (labelled Boolean feedback), were used to provide feedback on the introductions, conclusions and paragraph structures of student texts. A write-rewrite assignment (the same structure as in article 4) was used and the quality of the student revisions was evaluated. The results indicate that the students who received Boolean feedback showed greater improvement and fewer regressions than students who received feedback using the traditional method. The conclusion provides a brief summary as well as a preview of the immense future research possibilities made possible by this project.