In higher education, many institutions use algorithmic alerts to flag at-risk students and deliver advising at scale. While much research has focused on the accuracy of algorithmic alert systems, less is known about how advisors translate risk predictions into effective interventions.

In this talk, Kara will present evidence that advisors' *discretionary* judgments in acting on predictions may be key to these tools' success, using rich quantitative and qualitative data from a randomized controlled trial of an algorithm-assisted advising program at Georgia State University, the Monitoring Advisor Analytics to Promote Success (MAAPS) experiment. We present a framework combining causal modeling, statistical testing, and qualitative analysis to study one kind of discretionary judgment by advisors, the incorporation of non-algorithmic context to "expertly" target interventions to students, in the MAAPS experiment.

The results suggest advisors incorporate diverse non-algorithmic information into their intervention decisions, with an estimated 2 in 3 interventions in the MAAPS experiment that may have been "expertly targeted" by advisors. The findings shed light on the role of advisor expertise in a real-world algorithm-assisted advising program and underscore the importance of accounting for human discretion in the design, evaluation, and implementation of algorithmic decision systems in education.

This talk is based on joint work with Sofiia Druchyna, Benjamin Brandon, Jenise Stafford, Hannah Li, and Lydia Liu.

Data Science in Education

Digory

Simon Woodhead

Robert Murphy

Matthew Darlison

David Lobo

Timo Hannay

Martin Hlosta

Abhijit Mandal

Will Belzak

kkamau

Philip Greenwood

Evan Simpson

zx2393

Heng-Yi

Naley Nancy Lionelle

Sudhir Gupta

Sammy White

Lucas Dove

Discretion in the Loop: Human Expertise in Algorithm-Assisted College Advising

Online event

Buscar {networkName}

Grupos

{PRO_NETWORK_NAME} está en Meetup Pro y tiene más de {PRO_MEMBER_COUNT} miembros en {PRO_GROUP_COUNT} grupos de Meetup en todo el mundo. Meetup Pro es la herramienta profesional para organizar redes de usuarios, socios, colaboradores y miembro, y comunicarse con ellos.

Simon Woodhead

Simon Woodhead

Quiénes somos

Buscar Simon Woodhead

Eventos previstos

Grupos