Intended Versus Demonstrated Language Assessment Literacy: A Mixed-Method Study of Indonesian Pre-Service English Teachers

Abstrak

Language Assessment Literacy (LAL) is one of core competencies for English language teachers, although there is a gap between what pre-service teachers learn in assessment courses and their performances during teaching practicum period. This mixed-methods convergent parallel study investigated the effectiveness of the Language Assessment course at Universitas Hindu Negeri I Gusti Bagus Sugriwa Denpasar by assessing the gap between planned learning objectives and implemented LAL. Participants were 36 final year pre-service teachers and 7 university supervisors. Data were obtained using an LAL Self-Report Questionnaire, an Observation Checklist completed by supervisors, and document analysis of course syllabi and 108 assessment artifacts (summative tests, formative quizzes, scoring rubrics). A one sample t-test was used to compare the mean demonstrated LAL to the benchmark planned outcome score of 4.2. Qualitative data was investigated using thematic analysis. A t-test found a statistically significant difference (M = 2.8, t(35) = -14.23, p < .001, Cohen’s d = 2.91) with just 8.3% of pre-service teachers meeting the required standard. Pre-service teachers over-estimated their competency (self-report M = 3.9 compared to observed M = 2.7). The qualitative results showed five themes: general flaws in rubric design (75%), lack of statistical analysis (88.9%), generic unactionable feedback (69.4%), over-reliance on selected-response formats (80.6%), and total absence of digital assessment tools (100%). The Language Assessment course is not meeting its objectives. There is a need for immediate reforms that focus on practice-based curriculum, training on metacognitive self-assessment and integration of digital tools.