cs329z Agentic AI Working Group--RAG programming session
Details
Agents and agentic automation.
There are thousands of agentic programming jobs out there. You can use these to help guide the development of your own portfolio
https://cs329z.stanford.edu/
There are some important slides in the lectures. Use them for your own portfolio.
Evals: will cover why evals are so hard with a LLM
https://hamel.dev/blog/posts/evals-faq/. Different evals require different archiectures. Simple 1 - 2 document queries vs. multi document multi hop agents in hapi/fraud.
Context improvement of chunks using anthropic blog. (2024)
Dental persona chatbot with state-machine RAG and a policy Q&A with JEV acting as a guard against harmful content.
https://github.com/dougc333/geha/tree/main/dspy_dental_chatbot
Data Ingestion Workflow: React app demo on 2 image only files, a EY pdf with vector graphics and a synthetic bill of lading showing how to use a prompt to do layout blocking. We covered this last lecture. This was the only one working. Although current models can do this zeroshot, there is still a demand for data ingestion processing because many records can't be sent over the internet. Pair data ingestion pipeline with your own data source and with Recursive Self Improvement of a teacher/student open source model for an application use case.
RAG system architecture. We will step through a basic RAG system and a AWS implementation for research papers.
We will cover how to debug different stages of the RAG pipeline.
Context Retrieval React App. How to add LLM context to a docling chunk from the Anthropic blog and how to use prefix caching to reduce the overall cost. How to monitor this w a react app.
Context Assembly and Answer Completeness RAG React App demo. When combining chunks from different sources, how to detect and debug retrieval and assembly failures.
https://github.com/dougc333/geha/tree/main/context_assembly_benchmarks
Recursive Self Improvement React App How to have the teacher LLM create the prompt for a student model. Warning: can go to 1M tokens. Watch token budget.
Create a deck of your portfolio. https://docs.google.com/presentation/d/1HesLhw4w2OSBW7LTL1-tvZL2qx9AXJuO/edit?usp=sharing&ouid=114597187980442155549&rtpof=true&sd=true