NOMAD 13(2), 2008. Interrater reliability in a national assessment of oral mathematical communication

Skapad: 2008-06-13. Ändrad: 2008-06-13  

NOMAD 13(2), 2008. Interrater reliability in a national assessment of oral mathematical communication

TORULF PALM

Abstract

Mathematical communication, oral and written, is generally regarded as an important aspect of mathematics and mathematics education. This implies that oral mathematical communication also should play a part in various kinds of assessments. But oral assessments of subject matter knowledge or communication abilities, in education and elsewhere, often display reliability problems, which render difficulties with their use. In mathematics education, research about the reliability of oral assessments is comparably uncommon and this lack of research is particularly striking when it comes to the assessment of mathematical communication abilities. This study analyses the interrater reliability of the assessment of oral mathematical communication in a Swedish national test for upper secondary level. The results show that the assessment does suffer from interrater reliability problems. In addition, the difficulties to assess this construct reliably do not seem to mainly come from the communication aspect in itself, but from insufficiencies in the model employed to assess the construct.

Sammanfattning

Matematisk kommunikation, muntlig och skriftlig, ses i allmänhet som en viktig aspekt av matematik och matematikutbildning. Detta medför att muntlig matematisk kommunikation också bör bedömas i olika sorters prov. I både utbildningssammanhang och inom andra områden är dock muntliga prov av ämneskunskap eller kommunikationsförmåga behäftade med reliabilitetsproblem vilket orsaker svårigheter med dess användning. När det gäller matematikutbildning är reliabilitets­studier om muntliga prov dock relativt ovanliga, och detta gäller speciellt vid bedömning av matematisk kommunikationsförmåga. Denna studie analyserar interbedömarreliabiliteten för bedömningen av muntlig matematisk kommunikation i ett svenskt nationellt matematikprov för gymnasial nivå. Resultaten visar att provet var behäftat med reliabilitets­problem. Det verkar dock inte vara kommunikationsaspekten i sig själv som gör att denna förmåga var svår att bedöma reliabelt utan
otillräckligheter i den då använda bedömningsmodellen.

TORULF PALM
Torulf Palm, PhD, is a member of Umeå Mathematics Education Research Centre (UMERC) at Umeå University, Sweden and works at the Department of Mathematics, Technology and Science Education. His special research interests are assessment, mathematical reasoning, mathematical modelling and the authenticity of word problems.