Skip to content

About

No description, website, or topics provided.

Resources

Readme
Apache-2.0 license
Activity

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository
BranchesTags

Latest commit

 

History

77 Commits

Folders and files

NameName
Last commit message
Last commit date
input
input
 
 
notebooks
notebooks
 
 
relora
relora
 
 
resources
resources
 
 
scripts
scripts
 
 
source
source
 
 
4_phase_training.md
4_phase_training.md
 
 
CITATION.cff
CITATION.cff
 
 
Eric David Perl - Theophany_ The Neoplatonic Philosophy of Dionysius the Areopagite (S U N Y Series in Ancient Greek Philosophy) (2007).txt
Eric David Perl - Theophany_ The Neoplatonic Philosophy of Dionysius the Areopagite (S U N Y Series in Ancient Greek Philosophy) (2007).txt
 
 
Generate_questions.py
Generate_questions.py
 
 
John M. Dillon - The Middle Platonists_ 80 B.C. to A.D. 220-Cornell University Press (1996).txt
John M. Dillon - The Middle Platonists_ 80 B.C. to A.D. 220-Cornell University Press (1996).txt
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
Untitled.ipynb
Untitled.ipynb
 
 
Untitled2.ipynb
Untitled2.ipynb
 
 
bnb_4bit_finetuning.ipynb
bnb_4bit_finetuning.ipynb
 
 
bnb_4bit_finetuning.py
bnb_4bit_finetuning.py
 
 
bnb_4bit_inference.ipynb
bnb_4bit_inference.ipynb
 
 
bnb_4bit_inference.py
bnb_4bit_inference.py
 
 
bnb_4bit_integration.ipynb
bnb_4bit_integration.ipynb
 
 
bnb_4bit_training-mine.ipynb
bnb_4bit_training-mine.ipynb
 
 
bnb_4bit_training.ipynb
bnb_4bit_training.ipynb
 
 
bnb_4bit_training.py
bnb_4bit_training.py
 
 
chopped_sequences.txt
chopped_sequences.txt
 
 
clean_spo.py
clean_spo.py
 
 
closed-qa-pipeline.py
closed-qa-pipeline.py
 
 
common_imports.py
common_imports.py
 
 
compare.py
compare.py
 
 
expandquotes.py
expandquotes.py
 
 
functions-old.py
functions-old.py
 
 
functions.py
functions.py
 
 
gen_rag_GoT_augmented_finetuning.py
gen_rag_GoT_augmented_finetuning.py
 
 
ie_chunks.py
ie_chunks.py
 
 
installed_packages.txt
installed_packages.txt
 
 
labels.json
labels.json
 
 
main.py
main.py
 
 
mezo-all-phases.py
mezo-all-phases.py
 
 
mezo-finetune-C-I-R.py
mezo-finetune-C-I-R.py
 
 
mezo-finetune-I-R.py
mezo-finetune-I-R.py
 
 
mezo-finetune-pretrain.py
mezo-finetune-pretrain.py
 
 
mezo-inference.py
mezo-inference.py
 
 
mezo-pretrain.py
mezo-pretrain.py
 
 
packages.txt
packages.txt
 
 
parse-jsonl-alpaca.py
parse-jsonl-alpaca.py
 
 
post-embeddings.py
post-embeddings.py
 
 
prep_data.py
prep_data.py
 
 
prep_qa_data.py
prep_qa_data.py
 
 
prep_raw_data.py
prep_raw_data.py
 
 
prep_summ_data.py
prep_summ_data.py
 
 
pyvenv.cfg
pyvenv.cfg
 
 
q-a-facts.py
q-a-facts.py
 
 
requirements.txt
requirements.txt
 
 
results.json
results.json
 
 
run-mezo.sh
run-mezo.sh
 
 
run.sh
run.sh
 
 
split_strings.txt
split_strings.txt
 
 
spo_parse_jsonl.py
spo_parse_jsonl.py
 
 
spo_qa.py
spo_qa.py
 
 
summarize.py
summarize.py
 
 
torchrun_main.py
torchrun_main.py
 
 
train_chopped_sequences.txt
train_chopped_sequences.txt
 
 
trainer.py
trainer.py
 
 
valid_chopped_sequences.txt
valid_chopped_sequences.txt
 
 
vars.py
vars.py
 
 

Repository files navigation

TrainLLMv3

framework for [eventually] achieving closed-book qa

#see 4-phase-training (old document) #atm only doing phase II. Phase I is meant to pre-train

#important files #resources/train.json #vars.py

#pre-training Generate_questions.py #writes outputted json to llm dataset format of contexts, questions, answers prep_raw_data.py

#other files of interest

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user/env/lib/python3.11/site-packages/nvidia/cusparse/lib/ export WANDB_MODE=offline ./mezo-run.sh prep_data.py ./mezo-run.sh mezo-all-phases.py

#after training mezo-inference.py

About

No description, website, or topics provided.

Resources

Readme
Apache-2.0 license
Activity

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Used by

Contributors

Languages