DPO#30

Open

shahbuland wants to merge 30 commits intomainfrom

Commits on Sep 17, 2023

Add base trainer for any accelerate model
shahbuland
committed

Commits on Sep 27, 2023

Commits on Jan 23, 2024

Pipeline for DPO
Shahbuland Matiana
committed
Allow for list of images instead of just list of np arrays for sample drawing
Shahbuland Matiana
committed
Add sampler for DPO
Shahbuland Matiana
committed
Add method config for DPO
Shahbuland Matiana
committed
Add DPO trainer initial version
Shahbuland Matiana
committed

Commits on Jan 24, 2024

basic debugs
shahbuland
committed
Remove streaming
shahbuland
committed
minor bug fixes
shahbuland
committed
Moved saving from DDPO trainer to base accelerate
shahbuland
committed

Commits on Jan 25, 2024

LoRA, refactorings, quick bug fixes
shahbuland
committed
small bug fixes
shahbuland
committed
bug fixes
shahbuland
committed
Fix import errors and checkpointing
shahbuland
committed

Commits on Jan 26, 2024

Add base model loss deviation to sampling as metric
shahbuland
committed
Add base model loss deviation to trainer logging as metric
shahbuland
committed
Add non-lora training with memory saving options in config
shahbuland
committed

Commits on Jan 28, 2024

some refactorings to sampling, add rmsprop
shahbuland
committed
Delete old DPO example, push new one
shahbuland
committed

Commits on Feb 13, 2024