Name: Understanding LLM behaviors through interactive probing and visualisations
Start: 2024-10-31T18:00:00Z
End: 2024-10-31T19:30:00Z
Location: Inspace

Large language models (LLMs) have shown impressive capabilities in following natural language instructions to generate fluent text. However, the LLMs' generated text is not always faithful and truthful. These models can generate non-factual content in a convincing way, known as hallucinations, and use the shortcuts in the dataset for predictions. Besides, LLM users know little about how the LLM follows the instructions to arrive at their decisions and how confident the LLM is in the generated content, making it hard for users to justify the generation's correctness and assess the model's capability.

In this talk, I will introduce our recent studies in LLM behavior analysis. First, I will discuss our approach to identifying hallucinations by measuring and communicating the LLM's confidence in long-form generations. Then, I will introduce a counterfactual-based method for explaining LLM local behaviors. Finally, I will summarize the talk and discuss the open questions and research opportunities in interactive LLM behavior analysis.

**Bio**
Dr. **Furui Cheng** is a Postdoc at ETH Zürich, working with Prof. Menna El-Assady. Before that, he earned his Ph.D. at the Hong Kong University of Science and Technology, advised by Prof. Huamin Qu. His research lies in the intersection between data visualization, human-computer interaction (HCI), and explainable artificial intelligence (XAI). He develops and integrates XAI techniques with interactive visualizations to help users probe, understand, and steer machine learning models. His research has been published in IEEE TVCG, IEEE VIS, and ACM CHI, and received a Best Paper Honorable Mention award in IEEE VIS’21 (top 5%) and a Best Abstract Award for an invited talk at BioVis@ISMB’22.
\-\-\-\-\-\-\-\-\-\-\-
The Edinburgh DataVis Meetup is an informal community event open to all, that has been bringing together practitioners, designers, academics and the just plain curious in this highly active field since 2018.
This is a joint event with Newcastle University, and a hybrid event.
Doors open at 1800; talks start at 1815.

Zoom link:
[https://ed-ac-uk.zoom.us/j/82092392703](https://ed-ac-uk.zoom.us/j/82092392703)
Meeting ID: 820 9239 2703
Passcode: V5iprqeB

Xinhuan Shu

Edinburgh Data Visualisation Meetup

Technology

Data Visualization

Data Analytics

Data Science

UX Design

Web Design

Graphic Design

Human-Computer Interaction

Computer Graphics

Understanding LLM behaviors through interactive probing and visualisations

Explainable AI

Inspace

Share

Edinburgh Data Visualisation Meetup

Understanding LLM behaviors through interactive probing and visualisations

Edinburgh Data Visualisation Meetup

Details

Members are also interested in