MedMini

Runs within < 3GB RAM | 0 vRAM

Inference Time < 3 sec

A Lightweight architecture for an Answering System on medical data based on LLMs, designed to run on edge devices.

Entirely on-device processing

Model Size on Disk: 500 + 250 MB

Installation Instructions

Quickstart - Docker Memory Heavy

• Download the docker-run.sh file from the repository

• sudo chmod +x docker-run.sh

• sudo ./docker-run.sh

  Uninstalling

  • sudo docker image rm medmini

Raw Install Best Performance

• git clone https://github.com/sarthakchittawar/Medmini.git
• sudo chmod +x install.sh ; ./install.sh
• sudo chmod +x run.sh ; ./run.sh
• Need to have a ubuntu>=22.04 or debian>=12 based distro

Future work

1. Improve the RAG algorithm without compromising on efficiency