This is the backend of WikiBot App project. Server was built using Python and FastAPI framework. Application involes usage of machine learning sequance to sequance model, which is based on transformer architecture, with the usage of tokenization, positional embedding and word embedding. Model was trained or WikiQA dataset and was built with the help of pytorch and pytorch lightning.
Model was trained, with such hyperparameters:
learning_rate = 1e-3
num_epochs = 150
patience = 1000
gamma = 0.995
d_model=128
nhead=4
num_encoder_layers=3
num_decoder_layers=3
dim_feedforward=256
check_val_every_n_epoch = 10
batch_size = 4- 🛜 Easy to use and develop REST API based on FastAPI
- 🤖 Pytorch model, with it's parameters ready to be used in practice
- ⚙️ Docker support for local development
- ⚡ Implementatio of pytorch lightning
- 📈 Tensorboard implementation via pytorch lightning
To work with this project locally or in a containerized environment, use the following commands:
conda env export > environment.yml # generates list of dependencies, which are used by conda
sips -s format jpeg ./evaluation_data/[file_name].HEIC --out ./evaluation_data/[file_name].jpg # converting HEIC -> JPG command on MacOS
python -m model.train # model training command
python -W ignore -m model.eval # model evaluation on your own files
tensorboard --logdir=lightning_logs # tensorboard starting command
uvicorn api.server:app --reload # server development starting command
docker-compose up # 🐳 Run with Docker (backend + frontend)🧠 Tech Stack
- pytorch lightning
- tensorboard
- torchvision