This session will cover different ways of aligning a FM.

Covered topics include

• A general overview of FM pre-training
• Fine-tuning - Types of used data: supervised fine-tuning, PEFT, Prompt tuning, soft prompts, Adapter tuning, AdapterHub, Selective finetuning (e.g. BiFit), Reparametrization-based fine tuning (LoRA)
• Preference tuning - RLHF, DPO, RLSF
• Constitutional AI
• Overview of curriculum learning
• Synthetic data generation techniques, their advantages and trade-offs
• Data Flywheel for FMware
• Advanced curriculum generation techniques and benefits, e.g. how curriculum learning is used in the Microsoft Phi and IBM Granite models