In higher education, many institutions use algorithmic alerts to flag at-risk students and deliver advising at scale. While much research has focused on the accuracy of algorithmic alert systems, less is known about how advisors translate risk predictions into effective interventions.

In this talk, Kara will present evidence that advisors' *discretionary* judgments in acting on predictions may be key to these tools' success, using rich quantitative and qualitative data from a randomized controlled trial of an algorithm-assisted advising program at Georgia State University, the Monitoring Advisor Analytics to Promote Success (MAAPS) experiment. We present a framework combining causal modeling, statistical testing, and qualitative analysis to study one kind of discretionary judgment by advisors, the incorporation of non-algorithmic context to "expertly" target interventions to students, in the MAAPS experiment.

The results suggest advisors incorporate diverse non-algorithmic information into their intervention decisions, with an estimated 2 in 3 interventions in the MAAPS experiment that may have been "expertly targeted" by advisors. The findings shed light on the role of advisor expertise in a real-world algorithm-assisted advising program and underscore the importance of accounting for human discretion in the design, evaluation, and implementation of algorithmic decision systems in education.

This talk is based on joint work with Sofiia Druchyna, Benjamin Brandon, Jenise Stafford, Hannah Li, and Lydia Liu.

Data Science in Education

Digory

Simon Woodhead

Robert Murphy

Matthew Darlison

David Lobo

Timo Hannay

Martin Hlosta

Abhijit Mandal

Will Belzak

Discretion in the Loop: Human Expertise in Algorithm-Assisted College Advising

Online event

Pesquisar {networkName}

Grupos

{PRO_NETWORK_NAME} está no Meetup Pro com mais de {PRO_MEMBER_COUNT} membros em {PRO_GROUP_COUNT} Meetups em todo o mundo. Meetup Pro é a ferramenta profissional para organizar e comunicar uma rede de usuários, parceiros, colaboradores e membros.

Simon Woodhead

Simon Woodhead

Quem nós somos

Pesquisar Simon Woodhead

Próximos eventos

Grupos