tfs-mt
Transformer from scratch for Machine Translation

▶️ Getting started • 📖 Documentation • 🤗 Hugging Face • 🎬 Demo

---

This project implements the Transformer architecture from scratch considering Machine Translation as the usecase. It's mainly intended as an educational resource and a functional implementation of the architecture and the training/inference logic.

Getting Started

From PyPI

pip install tfs-mt

From source

Prerequisites

• uv [install]

Steps

git clone https://github.com/Giovo17/tfs-mt.git
cd tfs-mt

uv sync

cp .env.example .env
# Edit .env file with your configuration

Usage

Training

To start training the model with the default configuration:

uv run src/train.py

Inference

To run inference using the trained model from the HuggingFace repo:

uv run src/inference.py

Configuration

The whole project parameters can be configured in src/tfs_mt/configs/config.yml. Key configurations include:

• Model Architecture: Config, dropout, GloVe embedding init, ...
• Training: Optimizer, Learning rate scheduler, number of epochs, ...
• Data: Dataset, Dataloader, Tokenizer, ...

Technical walkthrough

The technical breakdown of the architecture and the training/inference workflows can be found here:

• Architecture

• Training and Inference

Model Sizes

The project supports various model configurations to suit different computational resources:

Parameter	Nano	Small	Base	Original
Encoder Layers	4	6	8	6
Decoder Layers	4	6	8	6
d_model	50	100	300	512
Num Heads	4	6	8	8
d_ff	200	400	800	2048
Norm Type	PostNorm	PostNorm	PostNorm	PostNorm
Dropout	0.1	0.1	0.1	0.1
GloVe Dim	50d	100d	300d	-

Documentation

Full documentation is available at https://giovo17.github.io/tfs-mt/.

License

• Source code: licensed under the MIT License.

  • Note: This project includes modified code derived from PyTorch Ignite, which is licensed under the BSD 3-Clause License. See the LICENSE file for the full text of both licenses and original copyright notices.

• Documentation: located in the docs/ directory, licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0). See docs/LICENSE.

Citation

If you use tfs-mt in your research or project, please cite:

@software{Spadaro_tfs-mt,
author = {Spadaro, Giovanni},
licenses = {MIT, CC BY-SA 4.0},
title = {{tfs-mt}},
url = {https://github.com/Giovo17/tfs-mt}
}