Name: Approaching Human-Level Forecasting with Language Models
Start: 2024-04-04T10:00:00-07:00
End: 2024-04-04T11:00:00-07:00

Can AI forecast geopolitical events or any other major events that potentially can have a huge impact on the world? This is what researchers from UC Berkeley attempted to achieve in their research work.

They have built an LM (language model) pipeline for automated forecasting. Given any question about a future event, the model retrieves and summarizes relevant articles, reasons about them, and predicts the probability that the event will occur. In some cases, AI even surpassed human predictions.

In this talk, our guest, Danny Halawi, will share the technical details behind this work.

He and his collaborators developed a retrieval-augmented LM system designed to automatically search for relevant information, generate forecasts, and aggregate predictions.

They collected a large dataset of questions from competitive forecasting platforms. Under a test set published after the knowledge cut-offs of LMs, they evaluated the end-to-end performance of the system against the aggregates of human forecasts.

On average, the system nears the crowd aggregate of competitive forecasters, and in some settings surpasses it.

Sophia Aryan

BuzzRobot

Technology

Artificial Intelligence

Machine Learning

Data Science

Deep Learning

Data Science using Python

Deep Reinforcement Learning

Machine Learning with Python

Computer Programming

Programming Languages

Open Source Python

JavaScript

Golang

Open Source

Danny Halawi is working on developing AI systems that can accurately forecast future events, including advancements in AI capabilities themselves. He studies at UC Berkeley under the mentorship of Jacob Steinhardt. 

Prior to pursuing AI research, he worked on computer security roles at Robinhood and Google, and contributed to projects at PlayStation for the PS-5, and collaborated on digital humanities projects with Mairaj Syed.

Danny Halawi

Approaching Human-Level Forecasting with Language Models

Online event

Share

BuzzRobot

Approaching Human-Level Forecasting with Language Models

BuzzRobot

Details

Members are also interested in