Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories...
Details
We are delighted to welcome Millicent Ochieng and Felermino Ali from Microsoft Research Africa.
Millicent Ochieng is a Research Scientist at Microsoft Research Africa and a PhD researcher in Computer Science at the University of the Witwatersrand. Her research focuses on multilingual and multicultural AI, with particular interests in multilingual tokenization, cultural alignment, multimodal AI, AI evaluation, and equitable language technologies. Her work explores how AI systems can better represent linguistic and cultural diversity across global and underrepresented contexts.
Felermino Ali is a Postdoctoral Researcher at Microsoft Research Africa in Nairobi, specializing in Artificial Intelligence and Natural Language Processing. He holds a PhD in Informatics Engineering from the University of Porto, where his research focused on machine translation for low-resource languages. His research interests include multilingual AI, machine translation, ASR, language models, tokenization, AI evaluation, and language technologies for underrepresented African languages.
They will present their work “Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories Across Five African Communities”. Here is the abstract:
“Generative AI systems can produce fluent and locally plausible content while still failing to reflect the lived practices, relationships, language, values, and visual expectations of the communities they represent. In this talk, we present a community-grounded mixed-methods evaluation of multimodal AI-generated stories across five African communities. We show that cultural alignment depends not simply on recognizable cultural markers, but on how those markers fit social, linguistic, procedural, and visual context. We introduce a taxonomy of five broader cultural marker categories and eight recurring mechanisms of misalignment, and examine whether multimodal LLM judges can approximate community-grounded judgments at scale. Our findings show substantial variation in judge reliability and calibration across communities, motivating community-calibrated evaluation in which automated judges are validated against community judgments rather than assumed to be universally reliable. More broadly, we argue that building more representative AI requires not only representing diverse languages and communities, but also involving those communities in defining whether that representation is actually culturally aligned.”
