👉 Check out full documentation on my 🤗 Space

My Notion Companion 🤖

A conversational RAG that helps to chat with my (mostly Chinese-based) Notion Databases.

My Notion Companion is a LLM-powered conversational RAG to chat with documents from Notion. It uses hybrid search (lexical + semantic) to find the relevant documents and a chat interface to interact with the docs. It uses only open-sourced technologies and can run on a single Mac Mini.

Empowering technologies

• The Framework: uses Langchain
• The LLM: uses 🤗-developed HuggingFaceH4/zephyr-7b-beta. It has great inference speed, bilingual and instruction following capabilities
• The Datastores: the documents were stored into both conventional lexical data form and embeeding-based vectorstore (uses Redis)
• The Embedding Model: uses sentence-transformers/distiluse-base-multilingual-cased-v1. It has great inference speed and bilingual capability
• The Tokenizers: uses 🤗's AutoTokenizer and Chinese text segmentation tool jieba (only in lexical search)
• The Lexical Search Tool: uses rank_bm25
• The Computing: uses LlamaCpp to power the LLM in the local machine (a Mac Mini with M2 Pro chip)
• The Observability Tool: uses LangSmith
• The UI: uses Streamlit

What's wrong with Notion's native search?

As much as I've been a very loyal (but freemium) Notion user, search func in Notion sucks. It supports only discrete keyword search with exact match (e.g. it treats Taylor Swift as two words).

What's even worse is that most of my documents are in Chinese. Most Chinese words consist of multiple characters. If you break them up, you end up with a total different meaning ("上海"=Shanghai, "上"=up,"海"=ocean).

My Notion Compnion is here to help me achieve two things:

• to have an improved search experience across my notion databases (200+ documents)
• to have a conversation with my Notion documents in natural language

The system design

See more detailed evaluation and documentation on Fred's 🤗 space!

Quickstart

Step 1: Install tools required

The following tools is required to set up the environments:

1. Conda/Mamba package manager. I used Micromamba. Once you have the right distribution, create the virtual env with the provided env.yml
2. LlamaCpp: to enable GPU acceleration, llamacpp has to be installed separately. See the langchain instruction to set up llamacpp in the virtual environment
3. Redis stack server. Follow the Redis guide to set up

Step 2: Download the LLM

LlamaCpp accepts any gguf binary model files. These files can generally be found in Hugging Face searching model_name.gguf. You can choose one you like, but for running on a single machines, I suggest with models <=7B params and a quantized version.

Step 3: Create and specify various configurations

• create new .token.toml to specify (in key-value pair) the needed tokens for notion and hugging face. You will need to generate them on their website
• create new .env (if you want to use LangSmith as observability tool, see their guide, again you need to generate a token on their website)
• create new .notion_databases.toml, you need to specify (in key-value pair) the databases you want to download, see the langchain guide
• in .config.toml, change the model_path, model_name, model_mapping, embedding_model, redis_schema params according to the downloaded and used models

Step 4: Your chatbot is waiting for you!

run bash run.sh to start the notion companion! Enjoy chatting!

References

• Working Log
• Feature Tracker