Open source data ingestion for RAGs with Dlt

About the event

In this hands-on workshop, we’ll learn how to build a data ingestion pipeline using dlt to load data from a REST API into LanceDB so you can have an always up-to-date RAG.
​​We’ll cover the following steps:

• ​​Extract data from REST APIs
• ​​Loading and vectorizing into LanceDB, which unlike other vector DBs stores the data and the embeddings
• ​​Keeping your data up to date with incremental loading

​​By the end of this workshop, you’ll be able to write a portable, OSS data pipeline for your RAG that you can deploy anywhere, such as Python notebooks, virtual machines, or orchestrators like Airflow, Dagster, or Mage.
​
​​This event is sponsored by dlthub.
​
​​​​​DataTalks.Club is the place to talk about data. Join our slack community!