LLMs in Production and Semantic Search Engine

Welcome to our in-person ML monthly meetup in San Francisco. Join us for deep dive tech talks on AI/ML, food/drink, networking with speakers&peers developers, and win lucky draw prizes.

[Important update] We have reached the room capacity and closed the registration.

[RSVP instructions]

• Register at the event website: https://www.aicamp.ai/event/eventdetails/W2023040417 (correct/full name is required for badges and check in. NO walk-ins, NO access without badge)
• Contact us to submit topics and/or sponsor the meetup on venue/food/swags/prizes. https://forms.gle/JkMt91CZRtoJBSFUA
• Community on Slack for events chat, speakers office hour and sharing learning resources, job openings and projects collaboration. join slack (join #sanfrancisco channel)

Agenda:

• 5:00pm~5:30pm: Checkin, Food and Networking
• 5:30pm~5:45pm: Welcome/Sponsor intro
• 5:45pm~7:30pm: Tech talks
• 7:30pm~8:00pm: Open discussion, Lucky Draw & Mixer

Tech Talk 1: Ray as the Common Infrastructure for LLM and Generative AI
Speaker: Zhe Zhang, Head of Ray OSS @Anyscale
Abstract: Generative AI exposes many new and exciting challenges to the underlying compute infrastructure. In this talk, we will introduce how Ray, a leading solution for scaling ML workloads, tackles these challenges (from training and fine tuning, to inference and deployment).
Because of its flexibility and architectural advantages, Ray is used by leading AI organizations to train large language models (LLM) at scale (e.g., by OpenAI to train ChatGPT , Cohere to train their models, EleutherAI to train GPT-J, and Alpa for multi-node training and serving). Meanwhile, there’s also a fast increasing demand for users to orchestrate their own “open source version” of generative AI workloads, without needing to be trained from scratch.
We will dive into how Ray can be best used in both scenarios. We will finish with a roadmap for improvements we’re undertaking to make things even easier.

Tech Talk 2: LLMs for the rest of us.
Speaker: Chenggang Wu, Co-founder and CTO @Aqueduct
Abstract: Large language models (LLMs) and other foundation models have made an unprecedented step forward in AI. Unfortunately, the infrastructure required to run these models is so overwhelmingly complex that only a few select companies have the requisite capabilities. Infrastructure challenges range from managing large amount of data, deploying complex pipelines, and managing compute services. In this talk, we will discuss:
- An overview of the infrastructure challenges of running LLMs in the cloud
- A demo/walkthrough of deploying an LLM on existing cloud infrastructure
- How Aqueduct seamlessly takes a LLM-powered workflow from prototype to production.

Tech Talk 3: Open Source Semantic Search Engine
Speaker: Dan Dascalescu @Weaviate
Abstract: Today, we've gotten used to natural language search and recommendation systems. We expect to get what we search for without remembering the exact keywords. To enable this, we use machine learning models that represent data as vectors that capture the semantics of the data. To do this at scale we need a way to efficiently store and retrieve large amounts of vector and non-vector data. Scaling ML models to work reliably in production is complex and implementing efficient vector search while keeping real-time CRUD support that is expected from databases is even harder. Vector search engines form an elegant solution to these challenges. A vector search engine helps you create machine-learning-first search and recommendation systems based on your data. It searches through your data super-fast with Approximate Nearest Neighbor (ANN) search, while also supporting all CRUD operations and data mutability.

In this session, you will learn what vector search is, why and when you would need it and you will see vector search in action during live demos.