Name: [AI Alliance] Introducing Gneissweb: A State-Of-The-Art LLM Pre-training Dataset
Start: 2025-03-06T18:00:00+01:00
End: 2025-03-06T19:00:00+01:00

**Agenda**

* Quick intro about AI Alliance (5 mins)
* GneissWeb presentation (40 mins)
* Q&A (10 mins)
* Wrapup

**Session: Introducing GneissWeb - a state-of-the-art LLM pre-training dataset**
At IBM, responsible AI implies transparency in training data: Introducing GneissWeb (pronounced “niceWeb”), a state-of-the-art LLM pre-training dataset with \~10 Trillion tokens derived from FineWeb, with open recipes, results, and tools for reproduction!

In this session we will go over how we created GneissWeb and discuss tools and techniques used. We will provide code examples that you can try at your leisure.

👉 > 2% avg improvement in benchmark performance over FineWeb
👉 [Huggingface page](https://huggingface.co/datasets/ibm-granite/GneissWeb)
👉 [Data prep kit detailed recipe](https://github.com/IBM/data-prep-kit/blob/dev/examples/notebooks/GneissWeb/GneissWeb.ipynb)
👉 [Data prep kit bloom filter for quick reproduction](https://github.com/ian-cho/data-prep-kit/blob/dev/transforms/universal/bloom/bloom_python.ipynb)
👉 [Recipe models for reproduction](https://huggingface.co/collections/ibm-granite/granite-data-67acf1780ddf9261f2c12b47)
👉 [announcement](https://research.ibm.com/blog/gneissweb-for-granite-training)
👉 [Paper](https://arxiv.org/abs/2502.14907)

**Session Type**
Presentation

**Audience**
LLM app developers, data scientists, data engineers

**Technical Level**
Beginner – Intermediate

**Prerequisites**
None

**Speaker: Shahrokh Daijavad, Research Scientist @ IBM Almaden Research Center**
Shahrokh Daijavad, a distinguished Research Scientist in the Watsonx Data Engineering group at IBM Almaden Research Center, has a rich background in Edge Computing and Data Engineering. He earned his B.Eng. and Ph.D. in electrical engineering from McMaster University and spent years at IBM T. J. Watson Research Center. His recent research focuses on AI@Edge and Data Engineering for IBM Watsonx AI offerings.

**About the AI Alliance**
The [AI Alliance](https://thealliance.ai) is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.

IBM Community

IBM Developer Poland

Technology

Java

Web Design

Software Development

New Technology

Internet Professionals

Open Source

JavaScript

Web Technology

Cloud Computing

Web Development

Computer Programming

HTML5

JavaScript Frameworks

Blockchain

[AI Alliance] Introducing Gneissweb: A State-Of-The-Art LLM Pre-training Dataset

Artificial Intelligence

Online event

Share

IBM Developer Poland

[AI Alliance] Introducing Gneissweb: A State-Of-The-Art LLM Pre-training Dataset

IBM Developer Poland

Details

Members are also interested in