A survey on LLM fine tuning (Lora, Qlora etc) by Cosmin and Yaroslav

Cosmin Negruseri and Yaroslav Bulatov will go over

A survey on LLM finetuning (LoRa, QLoRa etc)

Abstract:

In this session we'll do an intro into Large Language Model fine tuning and will cover LoRa, QLoRa

Resources:

LoRA: Low-Rank Adaptation of Large Language Models https://arxiv.org/abs/2106.09685

QLoRA: Efficient Finetuning of Quantized LLMs https://arxiv.org/abs/2305.14314

LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning https://arxiv.org/abs/2308.03303

This is an online event. Link will be available to previous attendees or by invitation.