How Computers assess Language Skills with Validity and Reliability

by Rose Cleshman

Speaker 1

Abstract

This session will describe and demonstrate how research and advances in computer-based testing (CBT) has changed the way English Language can be assessed for summative and formative purposes across the world. Public perception of CBT has often focused on the lack of human interaction and judgement when using these systems. This session will demonstrate that in many ways, the opposite is true. Pearson’s use of AI in tests such as PTE Academic and PEIC allows the judgement of hundreds of human assessors to work in unison, increases content representation, and allows test takers multiple safe and secure test taking options, yet still ensuring accurate and trustworthy outcomes. Through years of research and development, Pearson’s language tests provide evidence of not only surface features of language proficiency but also essential features such as pronunciation, fluency, mediation and intelligibility in speaking and organisation of ideas, style, tone and grammar in writing. Skills are assessed through a combination of single and integrated skill tasks. This session will demonstrate how these systems work, and the various ways in which these tests can now be taken, ensuring valid and reliable language assessment.




About the author

Rose Cleshman

Dr. Rose Clesham is Pearson’s Director of Academic Standards and Measurement. Her career started in teaching, and teacher education, before moving on to Governmental positions, responsible for running national assessment programmes in the UK. Issues surrounding the content, standards, reliability and validity of assessments are central to her work, in particular to meet the needs of evolving international educational landscapes and requirements. Her research interests include the emerging development of e-assessment and Artificial Intelligence in particular, and on-going educational strategies and policy. She holds a Master’s degree in formative and summative assessment from Cambridge University and a Doctorate in educational assessment. She is a Fellow of the Association for Education Assessment – Europe (AEA-E) and is also a Fellow at the University of London (UCL). Rose lectures on educational standards, validity and reliability issues both nationally and internationally.