Evaluation Metrics for GenAI Apps

Watch the livestream of the session here: meet.google.com/wto-kxrm-ujg

-----------------------------

In an age where Large Language Models (LLM) are evolving at a rapid pace, understanding how to effectively evaluate the output of such models is crucial for ensuring their reliability and utility in actual real-world deployments. In this session, we delve into the intricacies of measuring performance and assessing the efficacy of LLM applications, and also look at best practices to test LLM-driven apps.

This event will happen in person in Bengaluru and will also be livestreamed online. Please see Registration section below for details.

AGENDA

• Literature review of the existing LLMs
• Dimensions of evaluation of LLMs
• Automated test frameworks Vs Human-based ranking systems
• LLM-based application building process
• Custom evaluation for apps and how to build pipelines

SPEAKERS
Sai Sundarakrishna, currently in a dual mode related to self initiated startup and academia, works with medical fraternity shaping patents, products and research across multiple medical specialties. Before leading DeepmAGIc, a multimodal AI startup, he was the VP of Dream11 Data science (Mumbai), Exec. Director of Wavicle AI products (Montreal) and Senior Researcher/Leader at General Motors and Caterpillar. He is active in Bangalore AI ecosystem, global forums (engg.editorship), conferences, and lectures regularly at several Bangalore meetups on AI. Sai is a Columbia University, NY and Virginia Tech grad in the fields of business, AI and Data Science.
LinkedIn: https://www.linkedin.com/in/sai-sundarakrishna-7b87625

Ronith S Kumar has hands-on expertise with Large Language Models (LLMs), having been working on this from the time of GPT-2. He has developed numerous LLM-based apps, including a code interpreter for a renowned GPT chat clone, and has pioneered one of the initial implementations of a general LLM on the Android platform. He is currently in his third year pursuing B.Tech AIE at Amrita Viswa Vidyapeetham.

FEE
This workshop is FREE to attend but seats are limited and available on an invite-only basis. Prior registration is required for receiving an invitation, as per the below process.

REGISTRATION
To register to attend the event in person in Bengaluru or online, please do BOTH of the following:
1. Fill in this Event Registration Form: https://lu.ma/pkw9m7fy (attendees will be selected and invited based on this EXCLUSIVELY)
2. RSVP here in Meetup (this is only for communicating any updates about the event)
Please note that we will not be able to accommodate walk-ins.

Please reach Nihal at 9663374431 if you need any clarifications or have any challenges in registration. We look forward to seeing many of you there!

We thank Microsoft Reactor for co-hosting this session with us.