The development and validation of Primary Trait Writing Rubrics for secondary school students' self-assessment in Malaysia using many facet Rasch Model

Abstract

With the integration of The Common European Framework of Reference (CEFR) into Malaysia’s English curriculum, teachers face increasing pressure to promote self directed learning. However, the lack of effective evaluation tools hinders self assessment, leaving students without clear criteria to gauge their performance. This ambiguity may lead to confusion, inconsistent grading, and diminished motivation. To address these challenges, this study aimed to develop and validate four Primary Trait Writing (PTW) rubrics for self-assessment. A mixed-method approach with an exploratory sequential design was employed, comprising rubric development stage, rubric trialing stage, and rubric implementation stage. The development of these rubrics was guided by Davis’s (2018) Primary Trait Rubric Development Model, while their validation followed Bachman’s (2005) Assessment Use Argument (AUA) framework, focusing on test usefulness qualities: (a) construct validity, (b) reliability, (c) authenticity, (d) impact, and (e) practicality. A total of 149 Form 4 students representing low, average, and high levels of English proficiency, were purposively sampled to produce essays and use the rubrics for self-assessment. Additionally, three expert teachers, selected through convenience sampling, evaluated the students’ essays. Four primary traits: (a) Content, (b) Format, (c) Cohesive Devices, and (d) Sentence Fluency were identified as suitable components for the PTW rubrics. The Many-Facet Rasch Model was utilized to assess the construct validity, reliability, and authenticity of the rubrics. Furthermore, qualitative data from interviews with teachers and students provided insights into the rubrics’ impact and practicality. The findings demonstrated that the rubrics were reliable and valid for self-assessment, though notable concerns emerged regarding the Cohesive Devices trait. The rubrics were also deemed authentic, as they aligned with teacher assessments. Additionally, they were found to have a positive impact and were considered practical for student use. The findings have significant theoretical, methodological, and pedagogical implications for self-assessment in the English as second language learning (ESL) classroom. Future research could explore the development of primary-trait scoring rubrics for peer-assessment activities and investigate co-constructed rubrics, wherein students play an equal role alongside researchers in the development process.