SAVE THE DATE! Our next Meetup is coming in June, 18th. 🚀
This time we will be hosted by Fandom whose Poznań office is located on Baraniaka 6 Street.
We will have two amazing presentations followed by another awesome networking with pizza 🍕. Can't wait to meet you all! 😊
Neon Zhang (Fandom)
Metabites - Embeddings in User Reviews
#embeddings #textanalysis #UAE-Large-V1 #FAISS
As a user, I want to compare other people’s reviews of a game to get validation of my own opinion. But looking through all of the reviews is not rewarding, what should I do? Here, we produce bite-sized keywords to evaluate Metacritic’s user reviews and capture the overall user sentiment of a given game using Embeddings.
Maria Knorps (Tweag/Modus Create)
Evaluation framework for Retrieval-Augmented Generation (RAG) systems
Maria will present the work of the GenAI team at Tweag, the innovation arm of Modus Create, where the team developed an evaluation framework for Retrieval-Augmented Generation (RAG) systems. RAG systems provide an easy and low-cost way to extend the knowledge of Large Language Models (LLMs), but measuring their performance is challenging.
The presentation will review existing evaluation frameworks, ranging from those based on the traditional ML approach using ground truth datasets, including Tweag's framework, to those that use LLMs to compute evaluation metrics. It will also delve into the practical implementation of Tweag's chatbot over two distinct document datasets, providing insights on chunking, embedding, and comparing open-source and commercial LLMs.