Developing a Coherent System for the Assessment of Writing Abilities: Tasks and Tools

  • Ana Patricia Muñoz Universidad EAFIT
  • Jonathan Mueller North Central College, Illinois
  • Martha E. Álvarez Universidad EAFIT
  • Sandra Gaviria Universidad EAFIT
Keywords: writing standards, construct validity, writing prompts, inter-rater reliability, rubrics

Abstract

This article discusses the design and validation of a writing assessment system. Two main processes are addressed: validation of the construct and the writing test prompts, and design and validation of scoring instruments. Construct validation included defining and aligning the construct, the standards, and the nature of the assessment tasks. The validation of tasks was determined by analyzing task content against a set of criteria to design appropriate writing tasks and by surveying students' understanding of the prompts. Finally, scoring rubrics were designed based on the standards and the construct. To validate the rubrics, their alignment with the construct and the standards was established, as well as the estimation of their inter-rater reliability (r = > 0,7).

Received: 09-03-06 / Accepted: 19-07-06

How to reference this article:

Muñoz, A. P.; Mueller, J.;  Álvarez, M. E. & Gaviria, S. (2006). Developing a Coherent System for the Assessment of Writing Abilities: Tasks and Tools. Íkala. 11(1), pp. 265 – 307 

|Abstract
= 288 veces | PDF
= 331 veces|

Downloads

Download data is not yet available.

Author Biographies

Ana Patricia Muñoz, Universidad EAFIT
Ana Muñoz is the Research and Teacher Education Coordinator at the Language Center, EAFIT University. She holds an M.A. in TESOL from Eastern Michigan University, USA and has a B.A. in Languages - English and French from Universidad Pontificia Bolivariana, Medellín, Colombia.
Jonathan Mueller, North Central College, Illinois
Jonathan Mueller is a professor of psychology at North Central College, Illinois. He holds a Ph.D in Personality and Social Psychology from Northern Illinois University. For more than ten years he has consulted with teachers, schools and districts, at the K-12 and post-secondary levels, on the development, evaluation and revision of assessments, goals and standards.
Martha E. Álvarez, Universidad EAFIT
Martha E. Alvarez is an Industrial Engineer from Universidad Nacional, Medellín. She holds a postgraduate degree in Information Systems from Universidad EAFIT and has been a teacher at the School of Engineering in this institution since 1992. She has also taught and coordinated the Statistics I and II programs.
Sandra Gaviria, Universidad EAFIT
Sandra Gaviria is the Coordinator of the Adult English Program and researcher at the Language Center, EAFIT University. She holds an M.A. in TESOL from West Virginia University, USA and has a B.A. in Modern Languages from Universidad Pontificia Bolivariana, Medellín, Colombia.

References

Alderson, J. C. & D. Wall, 2001, Language test construction and evaluation, Cambridge, Cambridge University Press.

Bachman, L. & A. Palmer, 1996, Language testing in practice: Designing and developing useful tests, Oxford, Oxford University Press.

Bachman, L., 1990, Fundamental considerations in language testing, Oxford, Oxford University Press.

Bailey, K. 1996, “Working for washback: a review of the washback concept in language testing”, Language Testing 13, 257–79.

Bereiter, C. and M. Scardamalia, 1987, The psychology of written composition, Hillsdale, NJ, Lawrence Erlbaum Associates.

Berkowitz, D. et al., 2000, The use of texts as part of High-Stakes decision-making for students: A Resource Guide for educators and policy-makers, Washington, DC, U.S. Department of Education.

Canale, M. and M. Swain, 1980, “Theoretical bases of communicative approaches to second language teaching and testing”, Applied Linguistics 1, (1), 1- 47.

Carr, J. F., & D. E. Harris, 2001, Succeeding with standards: Linking curriculum, assessment, and action planning, Alexandria, VA, ASCD.

Council of Europe, 2001, Common European framework of reference for languages: Learning, teaching, assessment, Cambridge, Cambridge University Press.

Douglas, D., 2000, Assessing language for specific purposes, Cambridge, Cambridge University Press. 294 Íkala, revista de lenguaje y cultura Vol. 11, N.º 17 (ene.-dic., 2006)

Grabe, W. and Kaplan, R. B., 1989, “Writing in a second language: Contrastive rhetoric”. In D. M. Johnson and D. H. Roen (eds.), Richness in writing. New York and London: Longman.

Green, S., 2002, Criterion referenced assessment as a guide to learning – The importance of progression and reliability. A paper presented at the association for the study of evaluation in education. Southern Africa International Conference.

Guskey, T. R., 2001, “Helping standards make the grade”, Educational Leadership, 59, 20-27.

Hale, G., et al., 1996, A study of writing tasks assigned in academic degree programs, TOEFL Research report No. 54, Princeton, NJ, Educational Testing Service.

Hughes, A. 1989, Testing for language teachers, New York: Cambridge University Press.

Hyland, K., 2003, Second language writing, Cambridge: Cambridge University Press.

Hymes, D. 1972, “On communicative competence”, in J. Pride and H. Holmes (eds.), Sociolinguistics, NY, Penguin, 269 – 93.

Marzano, R. J., et al., 2001, Classroom instruction that works: Research-based strategies for increasing student achievement, Alexandria, VA, ASCD.

Messick, S. 1996, “Validity and washback in language testing”, Language Testing 13(3), 241-256.

Messick, S., 1989, “Meaning and values in test validation: the science and ethics of assessment”, Educational Researcher 18 (2), 5-11.

Mueller, J., 2004, Authentic Assessment Toolbox, [Retrieved: March, 2005] Natriello, G., & S.

M. Dornbusch, 1984, Teacher evaluative standards and student effort, New York, Longman.

Purves et al., 1984, “Towards a domain-referenced system for classifying assignments”, Research in the Teaching of English. 18 (4), 385 – 416.

Rea-Dickins, P. & K. Germaine, 1992, Evaluation, Oxford: Oxford University Press. Reid, J. and B. Kroll, 1995, “Designing and assessing effective classroom writing assignments for NES and ESL students”, Journal of second language writing 4 (1), 17-41.

Stansfield, C., J. & Ross, 1988, “A long-term research agenda for the Test of Written English”, Language Testing, 5, 160-186.

Troman, G. 1989, “Testing tension: The politics of educational assessment”, British Educational Research Journal, 15, 279-295

Weigle, Sara, 2002, Assessing Writing, Cambridge, Cambridge University Press.

Weigle, S., 1994, “Effects of training on raters of ESL compositions”, Language Testing, 11, 197-223.

Weir, C. J., 1990, Communicative language testing, London, Prentice Hall. Widdowson, H., 1979, Explorations in applied linguistics, Oxford, Oxford University Press.

Wiggins, G., 1998, Educative assessment: Designing assessments to inform and improve student performance, San Francisco, CA, Jossey-Bass Publishers.

Published
2009-11-11
How to Cite
Muñoz, A. P., Mueller, J., Álvarez, M. E., & Gaviria, S. (2009). Developing a Coherent System for the Assessment of Writing Abilities: Tasks and Tools. Íkala, 11(1), 265-307. Retrieved from https://revistas.udea.edu.co/index.php/ikala/article/view/2791
Section
Empirical Studies