Sparx Science is an online homework system used weekly by 400,000+ students in England, part of the broader Sparx portfolio which serves 2,600+ schools and 2.2m+ students.

Traditionally, open-response questions—requiring students to answer with anything from a few words to full paragraphs—have required marking by teachers. As a result, online homework systems typically relied on constrained question types, such as multiple-choice.
However, Large language models (LLMs) have opened the door to reliably mark and provide meaningful feedback on open-response tasks, supporting effective retrieval practice and challenging students to think deeply.

In this talk, we'll share our approach to using LLMs to automatically mark prose in Sparx Science, including: fine-tuning, prompt engineering, and everything in between.

Data Science in Education

Simon Woodhead

Digory

Derek W

Francisco Diaz

Bedriye Akson

Rob Lucas

jake.amey

saxenapawan224

roksana mirzayi

Timo Hannay

Mehdi Moussaoui

liz pettit

Dirk Luedtke

Joyce Zhang

Farzam

baptiste moreau

Aldo

Scott Dooley

Deeper Thinking Through Open Response: Using LLMs for Marking and Feedback

Online event

As high-stakes exams move online, ensuring their integrity poses both technical and human challenges. This talk focuses on quantifying the quality of human judgment in proctoring—a critical but often overlooked aspect of test security. Like other professionals, proctors make high-stakes decisions under uncertainty, with judgments influenced by lapses in attention, inconsistent training, and individual biases. We introduce the Professional Calibration Tool (PCT), a psychometric system that both measures and increases decision accuracy and reliability by embedding stealth assessments—pre-adjudicated cases—into live workflows and delivering real-time feedback based on expert and peer consensus.

gorgun

Joseph 

Jonathon Speed

mark house

shahtaj nazmeen

Jan Meppe

Abby

From Gut Feelings to Data: Quantifying Proctor Judgment in High-Stakes Exams

Pesquisar {networkName}

Grupos

{PRO_NETWORK_NAME} está no Meetup Pro com mais de {PRO_MEMBER_COUNT} membros em {PRO_GROUP_COUNT} Meetups em todo o mundo. Meetup Pro é a ferramenta profissional para organizar e comunicar uma rede de usuários, parceiros, colaboradores e membros.

Simon Woodhead

Simon Woodhead

Quem nós somos

Pesquisar Simon Woodhead

Próximos eventos

Grupos