Vector DB Microservice Launchpad

This repository consolidates multiple asynchronous API implementations (Dotnet, Node, Python, and Rust) showcasing the use of PosgreSQL with the PgVector extension for elementary vector store and retrieve operations. This project is intended to be a launchpad for practicioners and researchers who need to quickly spin up a RESTful API in their language of choice, while kicking off development of a custom vector database implementation. Each subproject demonstrates how to build and launch the servers, which manage embeddings and execute vector similarity queries, and return JSON responses with additional timing information for the runtime comparison that is detailed below.

Key Features

• Efficient Database Connectivity

  • Seamlessly connect to your PostgreSQL database using connection pooling.
  • Optimized resource utilization for improved concurrency and performance.

• Asynchronous API

  • Fully asynchronous design to maximize throughput and responsiveness.
  • Supports high-load scenarios with non-blocking operations.

• Interactive API Documentation

  • Swagger UI is automatically rendered at the root URL.
  • Easily explore, test, and validate all API endpoints through a user-friendly web interface.

• Consistent API Endpoints

  • Test database connectivity to ensure seamless integration.
  • Insert vector embeddings into PostgreSQL for efficient storage and retrieval.
  • Perform approximate nearest neighbor (ANN) searches for AI/ML applications.

• Performance Metrics

  • Includes precise timing details in the output JSON.
  • Compare runtime performance across different server environments and configurations.

• Multi-Language Implementations

  • The API is implemented in four different programming languages (+ servers):
    • Python (FastAPI)
    • Node.js (Express)
    • Rust (Actix-Web)
    • C# (.NET)

Common Setup

• Install PostgreSQL and enable the pgvector extension. See this post from Microsoft for details on deploying it to a 'Azure Database for PostgreSQL - Flexible Server'
• Configure environment variables (database connection, etc.).

PG_USER=
PG_HOST=
PG_DATABASE=
PG_PASSWORD=
PG_PORT=5432

• Review each subproject's README for language-specific tools and commands.

Performance

The figure below shows the runtime performance of each service over a series of empiracle tests for elementary operations: vector inserts and approximate nearest neighbor search. A sample size of 10 trials were collected for both tests and for vector dimensions of 1536 and 3072 (to approximate performance for text-embedding-003-small and text-embedding-003-large models). Approximate nearest neighbor search was always performed using the PgVector extension implementation. Vector generation was performed using random number generation from a uniform distribution [0,1] with O(N) complexity but package optimizations could influence the vector generation times significantly.

In summary, the C# implementation ran the fastest in all test cases, notably for the ANN search, where performance was markely better than Python/Node/Rust. High latency of the Rust server indicates that the current implementation needs further optimization, as it uses AoT Compilation which is expected to be slightly faster than C#'s JiT compilation. Additionally we may not see the runtime hit of C# garbage collection in this simple test case.

To Dos

• Optimize the Rust implementation (I'm new to it!). Would expect runs with "--release" flag to run as fast as C#, but not quite there, any feedback appreicated!
• The Rust implementation is missing swagger spec rendering.
• Upgrade Python swagger spec implementation to be based on docstrings/comments rather than a static swagger.yaml file.

Roadmap

This project is just an initial foray into the world of custom databases, and is not intended for production use without significant deltas. Therefore, this repository will not be activley maintained to support production use, but issues and emails will be monitored to best support community developers.

Contributing

Contributions are welcome! Please submit a pull request or open an issue for discussions. As with any runtime comparisons I expect there will be many differing opinions and suggestions - please raise issues or email me at bryanbed@g.ucla.edu.

---

🚀 Kick off your vector search workflows with high-performance, multi-language API implementations!

License

This project is licensed under the MIT License.

Disclosures

This code is provided as-is and without any warranty or guarantee of fitness for a particular purpose.

At least some of this project was carried out within the scope of the Author's work at Sony Pictures. Express written consent from leadership and legal teams was provided prior to the public release of the code in this repository. This project contains NO intellectual property related to or reserved by Sony Pictures. Neither the Author nor the company assume liability over the adaptation and use of this code. It is being made public by the author for the betterment of humankind with a shared interest in furthering technology and progress in the space of machine learning and AI.