What we’re about
Hands-on project-oriented data science, with a heavy focus on machine learning and artificial intelligence. We're here to get neck-deep into projects and actually do awesome things!
Join us on slack! https://join.slack.com/t/boulderdatascience/shared_invite/zt-20t147vcy-SIo7is6meTWOfHa1ENlJeA
The meetup consists of:
- recurring study groups (if you want to start one, just notify Ben to be made a meetup co-organizer).
- intermediate/advanced working groups (starting in 2019)
- occasional talks and gathering (aiming for at least quarterly starting in 2019)
Upcoming events (2)
See all- Unveiling the Power of MergeKitLink visible for attendees
Event Description:
We'll be exploring MergeKit, a groundbreaking tool for blending and creating hybrid llm models from existing llms. MergeKit offers an exciting approach by facilitating the merger of pre-trained language models to forge advanced configurations efficiently, without demanding high GPU resources. This session is dedicated to unraveling the potentials of llm model merging, its broader implications, and how you can can use this open source tool yourself.Why MergeKit?
As the generative AI field rapidly progresses, MergeKit heralds a novel paradigm in model development. It empowers researchers and developers to blend the strengths of various models, leading to the creation of "frankenmerges." These hybrid models are not only unique but also boast enhanced efficiency and effectiveness. MergeKit's approach to model merging paves new avenues for AI research and application, democratizing access to sophisticated model development across the AI community.
Agenda:
- **The Essence of Model Merging:** An introductory overview highlighting the significance and foundational concepts of model merging in today's AI ecosystem.
- **Exploring the Intricacies of MergeKit:** A thorough examination of MergeKit's capabilities, including support for models like Llama, Mistral, GPT-NeoX, StableLM, and running through a simple example on how to use it.
- **The Advent of Frankenmerges:** A deep dive into the concept of frankenmerges, their creation process via MergeKit, and their prospective role in reshaping AI development. We will discuss miqu120B as an example of how to create a 120B parameter llm from a 70B base model.
- **Live Demonstration:** An interactive session demonstrating MergeKit's functionality, illustrating model merging processes, and experimenting with the resultant models.
- **Discussion and Q&A Session:** A platform to engage with AI enthusiasts and experts, fostering discussions on the challenges, implications, and opportunities that MergeKit and model merging bring to the table.Reading Materials & Resources:
1. **Linear Mixtures/Model Soups:** Uncover the concept of combining multiple models for improved performance and its relevance to MergeKit's methodology. [Model Soups Paper Link](https://arxiv.org/abs/2203.05482)
2. **SLERP - Spherical Linear Interpolation:** [SLERP Paper Link](https://dl.acm.org/doi/pdf/10.1145/325165.325242)
3. **Task Arithmetic:** [Task Arithmetic Paper link ](https://arxiv.org/abs/2212.04089)
4. **TIES - Towards Integrating Expert Systems:** [TIES paper Link](https://arxiv.org/abs/2306.01708)
5. **DARE - Dynamic and Robust Ensembles:** [DARE paper Link](https://arxiv.org/abs/2311.03099)---
Silicon Valley Generative AI has two meeting formats.
1. Paper Reading - Every second week we meet to discuss machine learning papers. This is a collaboration between Silicon Valley Generative AI and Boulder Data Science.
2. Talks - Once a month we meet to have someone present on a topic related to generative AI. Speakers can range from industry leaders, researchers, startup founders, subject matter experts and those with an interest in a topic and would like to share. Topics vary from technical to business focused. They can be on how the latest in generative models work and how they can be used, applications and adoption of generative AI, demos of projects and startup pitches or legal and ethical topics. The talks are meant to be inclusive and for a more general audience compared to the paper readings.
If you would like to be a speaker please contact:
Matt White