What we’re about
We are a professional organization for AI practitioners in the Silicon Valley. We aim to bring together data scientists, engineers, and business people working in AI and big data area. We host seminars, interactive group meetings, and mentoring sessions. We provide an exchange platform for big data professionals to share their experiences, learn about the newest technologies and explore potential startup opportunities. Join us today. Find like-minded people in AI and grow your career and AI and big data business with us.
Upcoming events (4+)
See all- [Workshop] Mastering RLHF in 1 day (LLM Crash Course #2)Link visible for attendees
This is a paid event. The registration is through Eventbrite (Meetup RSVP is not considered as registration ). Please register here
https://RLHF1Day.eventbrite.comRLHF (Reinforcement Learning with Human Feedback) is the secret weapon behind ChatGPT and Llama. It is a crucial step to enhance the performance of an LLM. By the end of this workshop, you will gain complete understanding of RLHF, the major components including PPO, reward function and supervised finetuning. You will gain the confidence of knowing the complete process of RLHF and how to implement it.
Topics include (see details at the end):
- Supervised finetuning
- Reinforcement learning basics
- PPO and its implementation
- Training a reward function
- Applying RLHF step by step
This workshop is for those who want to enhance their LLM model, and who are interested in RLHF.
What you get from this workshop:
- Real-time interaction with the instructor.
- 4 Live Python notebooks that you can take home.
- Certificate when finishing the class (upon request).
- Join the community of AI builders.
Schedule details:
9-10 am. Overview of the process of finetuning and enhancing an LLM
10-11 am. Supervised finetuning
Understand the supervised finetuning process for LLM, from data formatting to the training procedure.
11-12 pm. Reinforcement learning fundamentals
Introduction to reinforcement learning process, the basic concepts including action, reward, policy and value function.
12 pm-12:30 pm. Deep Reinforcement Learning
Major methods in deep reinforcement learning, particularly the policy gradient method and actor-critic framework.
12:30-1 pm Break
1-2 pm. Introduction to PPO (Proximal Policy Optimization)
The PPO algorithm and its implementation. See it in action in a Python notebook.
2-3 pm. The reward function and its training process
Learning a reward function for language generation. How to train such a function and practical steps.
3-4 pm. Training the LLM with PPO
Apply PPO to enhance language generation. How to reframe the language model in reinforcement learning framework, and train it using PPO.
4-5 pm. Bring everything together and Q&A
- Implement RLHF step by step, and enhance your model with feedback from humans. Starting with a pretrained LLM, we will enhance the model through the 3 steps that we have learned this course.
- Introducing tools for doing RLHF and resources after this workshop.
Instructor: Junling Hu, https://www.linkedin.com/in/junlinghu/
Refund policy:
Risk free purchase. 100% refundable if you are not happy with the class. Simply submit your request for refund within 1 day after the class finishes.The registration is through Eventbrite. Register at Eventbrite here: https://RLHF1Day.eventbrite.com
- [Course] Mastering Transformers in 4 Weeks (LLM Course #1)Link visible for attendees
This is a paid event. The registration is through Eventbrite (Meetup RSVP is not considered as registration ). Please register here https://transformers4weeks.eventbrite.com
Transformers have become the cornerstone of LLMs. By the end of this course, you will gain complete understanding of transformer, its inside architecture and the process of building and training a transformer. You will gain the confidence of knowing all the major concepts behind today’s LLM, and how to implement them.
Topics include (see details at the end):- Neural network and deep learning fundamentals
- Word embedding
- Attention layer and layer normalization
- Building encoders and decoders
- Building a transformer step by step
This course is for those with little background in deep learning but interested in getting deeper into transformers. It gives you real-time interaction with instructor and fellow students. You will be in a community to get help and learn and grow together.
What you get from this workshop:- 4 weeks of live classes, live interaction with the instructor
- 4 weekly Q&A sessions (separate from the class time) to get all your questions answered.
- 4 Take-home Python notebooks that you can run and learn from
- Join a project group where you can learn and practice.
- Join the community of AI builders.
- A certificate when finishing the course.
The class starts on October 1st. The live class happens on Sunday 2-4 pm, and the live Q&A session is on Weds 7-8 pm. You can reach out to the instructor and other students at any time during the week.
Schedule details:
Week 1: Introduction to neural networks and deep learning.
(1) Introduction to neural network basics, activation function, backpropagation, and optimization algorithms such as Adam and AdamW.
(2) Deep learning fundamentals, including dropout, Layer normalization, residual network, and GPU computing. Hands on exercise in building a deep neural network.Week 2: Sub-word tokenization and Word embedding
(1) Introduction to sub-word tokenization methods: BPE (Byte pair encode) and Unigram. Build a BPE tokenizer in a Python notebook
(2) Word embedding and position embedding, basic concepts and implementation.Week 3: Attention layer, encoder and decoders
(1) Understand attention operation, query, key and value, and attention heads. Build Python code to Implement it.
(2) Understand the sublayers of encoder and decoder, feedforward network (FFN), masked attention. Build encoder and decoder in a Python notebook.Week 4: Build a transformer for a practical application.
(1) Create a transformer step by step, and train and test it for a real dataset.
(2) Preview the pipeline of building LLM applications.Instructor: Junling Hu
Refund policy:
Risk free purchase. 100% refundable if you are not happy with the class. Simply submit your request for refund within 1 day after the first class.The registration is through Eventbrite. Please register at Eventbrite here: https://transformers4weeks.eventbrite.com
- Understanding RT-2: Vision-language-action modelsLink visible for attendees
In this talk, I will explain a new method from Deepmind (posted on July 28, 2023). It intends to create a generalized model that incorporates LLM into robotic control.
Paper abstract:
We study how vision-language models trained on Internet-scale data can be incorporated directly into robotic control. We propose to co-fine-tune vision-language models on both robotic trajectory data and Internet-scale vision-language tasks, such as visual question answering. In order to fit both natural language responses and robotic actions into the same format, we express the actions as text tokens and incorporate them directly into the training set of the model in the same way as natural language tokens. We refer to such models as vision-language-action models (VLA) and instantiate an example of such a model, which we call RT-2. Our evaluation shows significantly improved generalization to novel objects, the ability to interpret commands not present in the robot training data, and the ability to perform rudimentary reasoning in response to user commands.Join this event here: https://us02web.zoom.us/meeting/register/tZUtdumppjIpH910MlOei5RT1Y92Ht7EUjFL
Related links:
Paper: https://arxiv.org/abs/2307.15818Agenda:
7pm-7:05pm Meet and Greet
7:05-7:50pm Presentation
7:50-8 pm Q&A and Discussions - Training Diffusion Models with Reinforcement LearningLink visible for attendees
This is a new paper from UC Berkeley. We will review the algorithm, performance and the new Github code.
Paper abstract:
Diffusion models are a class of flexible generative models trained with an approximation to the log-likelihood objective. However, most use cases of diffusion models are not concerned with likelihoods, but instead with downstream objectives such as human-perceived image quality or drug effectiveness. In this paper, we investigate reinforcement learning methods for directly optimizing diffusion models for such objectives. We describe how posing denoising as a multi-step decision-making problem enables a class of policy gradient algorithms, which we refer to as denoising diffusion policy optimization (DDPO), that are more effective than alternative reward-weighted likelihood approaches. Empirically, DDPO is able to adapt text-to-image diffusion models to objectives that are difficult to express via prompting, such as image compressibility, and those derived from human feedback, such as aesthetic quality. Finally, we show that DDPO can improve prompt-image alignment using feedback from a vision-language model without the need for additional data collection or human annotation.Presenter: Junling Hu
Join this event here: https://us02web.zoom.us/meeting/register/tZMlc--urTMpGteRnTC6x2v5f8eQyoLIcF8J
Related links:
Paper: https://arxiv.org/abs/2307.15818
Blog of Carper.ai, Sept 27, 2023, https://carper.ai/enhancing-diffusion-models-with-reinforcement-learning/
Code: https://github.com/carperai/drlxAgenda:
7pm-7:05pm Meet and Greet
7:05-7:50pm Presentation
7:50-8 pm Q&A and Discussions
![[Workshop] Mastering RLHF in 1 day (LLM Crash Course #2) cover photo](https://secure.meetupstatic.com/photos/event/d/5/3/8/event_515634584.webp "[Workshop] Mastering RLHF in 1 day (LLM Crash Course #2)")
![[Course] Mastering Transformers in 4 Weeks (LLM Course #1) cover photo](https://secure.meetupstatic.com/photos/event/3/b/c/a/event_516015306.webp "[Course] Mastering Transformers in 4 Weeks (LLM Course #1)")
