icon	description	cover	coverY
2	다양한 LLM에 대한 자료를 수집합니다.	https://images.unsplash.com/photo-1645839057098-5ea8761a6b09?crop=entropy&cs=srgb&fm=jpg&ixid=M3wxOTcwMjR8MHwxfHNlYXJjaHw1fHxsbG18ZW58MHx8fHwxNzMwNjE5NjcwfDA&ixlib=rb-4.0.3&q=85	0

LLM

Ⅰ. Reference

다양한 LLM 관련 레퍼런스를 소개합니다.

모델학습시 GPU 사용량 계산하는 사이트
- https://huggingface.co/spaces/Vokturz/can-it-run-llm
Open Ai : Transformer Debugger
- https://github.com/openai/transformer-debugger
Langchain Visualizer
- https://github.com/amosjyng/langchain-visualizer
RingAttention
- https://github.com/lhao499/RingAttention
Awesome LLM Tabular
- https://github.com/johnnyhwu/Awesome-LLM-Tabular
Awesome LLM
- https://github.com/Hannibal046/Awesome-LLM
Awesome Korean LLM
- https://github.com/NomaDamas/awesome-korean-llm
MiniGPT-4 (opensource)
- https://github.com/Vision-CAIR/MiniGPT-4
transformer library tutorial
- https://huggingface.co/docs/transformers/v4.39.1/ko/index
- https://github.com/NielsRogge/Transformers-Tutorials
Py2Mojo
- https://github.com/msaelices/py2mojo
Awesome Korean speech recognition
- https://github.com/rtzr/Awesome-Korean-Speech-Recognition
Open Korean instructions
- https://github.com/HeegyuKim/open-korean-instructions
Practical MLOps
- https://github.com/paiml/practical-mlops-book
Embedding (ratsgo 한국어 임베딩 도서 코드)
- https://github.com/ratsgo/embedding

LLM 학습에 대한 레퍼런스를 소개합니다.

WhatsApp : Finetune a LLM to speak like you based on your WhatsApp Conversations
- https://github.com/Ads-cmu/WhatsApp-Llama
KB-ALBERT : KB국민은행에서 제공하는 경제/금융 도메인에 특화된 한국어 ALBERT 언어모델
- https://github.com/KB-AI-Research/KB-ALBERT
딥러닝 파이토치 교과서 - 입문부터 파인튜닝까지 wikidocs
- https://wikidocs.net/book/2788
전이학습 기반 NLP
- https://brunch.co.kr/@learning/12
자연어처리 : 토큰화
- https://real-myeong.tistory.com/41
딥러닝을 위한 자연어 처리 입문
- https://wikidocs.net/book/2155
SentencePiece를 이용한 효과적인 한국어 토크나이저 만들기 (예제)
- https://devocean.sk.com/blog/techBoardDetail.do?ID=164570&boardType=techBlog#none
언어모델의 말솜씨? NEFTune 한 스푼으로 업그레이드 (예제)
- https://devocean.sk.com/blog/techBoardDetail.do?page=&boardType=undefined&query=&ID=165452&searchData=&subIndex=#none

다양한 학습 방법론에 대한 레퍼런스를 소개합니다. (Pre-Training / Fine-Tuning)

PPO
- https://velog.io/@uonmf97/HUFS-RL-%EA%B0%95%ED%99%94%ED%95%99%EC%8A%B5-Reinforcement-Learning-PPO-Proximal-PolicyOptimization
RLHF (Human Feedback)
- https://tech.scatterlab.co.kr/luda-rlhf/
TransformersData Augementation
- https://github.com/varunkumar-dev/TransformersDataAugmentation
Domain Adaptation : Generative Pseudo Labeling(GPL)
- https://github.com/UKPLab/gpl
X Algorithm
- https://github.com/twitter/the-algorithm-ml
MergeKit
- https://github.com/arcee-ai/mergekit?tab=readme-ov-file#merge-methods
- https://bigwaveai.tistory.com/79

LLM 경량화에 대한 레퍼런스를 소개합니다.

PEFT
LoRA
QLoRA
Knwoledge Distillation
- https://velog.io/@qtly_u/%EB%AA%A8%EB%8D%B8-%EA%B2%BD%EB%9F%89%ED%99%94-%EA%B8%B0%EB%B2%95-Knowledge-Distillation

LLM Alignment에 대한 레퍼런스를 소개합니다.

DPO
Instruct Following
OpenAi Alignment handbook
- https://github.com/huggingface/alignment-handbook
LoRA Instruct
- https://github.com/leehanchung/lora-instruct
FlashAttention : Fast and memory-efficient exact attention
- https://github.com/Dao-AILab/flash-attention