Skip to content

Commit f57128e

Browse files
zhiyuan8zhiyuan8
committed
update README
1 parent 0969f3c commit f57128e

2 files changed

Lines changed: 300 additions & 81 deletions

File tree

‎README.md‎

Lines changed: 171 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -1,129 +1,219 @@
1-
# GenieX
1+
<div align="center" style="text-decoration: none;">
2+
<p>
3+
<a href="https://www.producthunt.com/products/nexasdk-for-mobile?embed=true&utm_source=badge-top-post-badge&utm_medium=badge&utm_campaign=badge-nexasdk-for-mobile" target="_blank" rel="noopener noreferrer">
4+
<img alt="NexaSDK for Mobile - #1 Product of the Day" width="180" height="39" src="https://api.producthunt.com/widgets/embed-image/v1/top-post-badge.svg?post_id=1049998&theme=dark&period=daily&t=1765991451976">
5+
</a>
6+
</p>
7+
<p>
8+
<a href="https://docs.nexa.ai">
9+
<img src="https://img.shields.io/badge/docs-website-brightgreen?logo=readthedocs" alt="Documentation">
10+
</a>
11+
</p>
12+
</div>
213

3-
Multi-platform AI inference runtime for Snapdragon / Hexagon — runs LLMs on NPU, GPU, or CPU through a pluggable C SDK with Go (CLI), Python, and Java (Android) bindings.
14+
# NexaSDK
415

5-
> Status: pre-1.0, under active development. Public API and tags may change; see [notes/release.md](notes/release.md).
16+
**NexaSDK lets you build the smartest and fastest on-device AI with minimum energy.** It is a highly performant local inference framework that runs the latest multimodal AI models locally on NPU, GPU, and CPU - across Android, Windows, and Linux devices with a few lines of code.
617

7-
## Runtimes & compute units
18+
NexaSDK supported latest models **weeks or months before anyone else** — Qwen3-VL, DeepSeek-OCR, Gemma3n (Vision), and more.
819

9-
| Runtime / GGML backend | Compute unit | Model format | Enabled by |
10-
| ---------------------- | -------------- | ------------ | ----------------------------------------- |
11-
| llama.cpp / Hexagon | Snapdragon NPU | GGUF | `llama_cpp` + `-DGGML_HEXAGON=ON` |
12-
| llama.cpp / OpenCL | Adreno GPU | GGUF | `llama_cpp` + `-DGGML_OPENCL=ON` |
13-
| QAIRT / QNN | Snapdragon NPU | QAIRT `.bin` | `qairt` + `-DGENIEX_PLUGIN_QAIRT=ON` |
14-
| llama.cpp / CPU | Any | GGUF | `llama_cpp` (default; disable both flags) |
20+
> ⭐ **Star this repo** to keep up with exciting updates and new releases about latest on-device AI capabilities.
1521
16-
The `llama_cpp` and `qairt` runtimes both target the NPU but through **separate user-space stacks** (ggml-hexagon DSP skels vs. Qualcomm QNN) that consume **different model formats**. They are not substitutes. QAIRT libs are bundled under `third-party/geniex-qairt/`; Hexagon and OpenCL SDKs are external installs.
22+
## 🏆 Recognized Milestones
1723

18-
## Install
24+
- **Qualcomm** featured us **3 times** in official blogs.
25+
- [Innovating Multimodal AI on Qualcomm Hexagon NPU](https://www.qualcomm.com/developer/blog/2025/09/omnineural-4b-nexaml-qualcomm-hexagon-npu).
26+
- [First-ever Day-0 model support on Qualcomm Hexagon NPU for compute and mobile platforms, Auto and IoT](https://www.qualcomm.com/developer/blog/2025/10/granite-4-0-to-the-edge-on-device-ai-for-real-world-performance).
27+
- [A simple way to bring on-device AI to smartphones with Snapdragon](https://www.qualcomm.com/developer/blog/2025/11/nexa-ai-for-android-simple-way-to-bring-on-device-ai-to-smartphones-with-snapdragon)
1928

20-
Release assets live on the [Releases page](https://github.com/qcom-ai-hub/geniex/releases). `<TAG>` below is the release tag (e.g. `v0.4.0`).
29+
## 🚀 Quick Start
2130

22-
### Windows (installer)
31+
| Platform | Links |
32+
| --------------- | ----------------------------------------------------------------------------------------- |
33+
| 🖥️ CLI | [Quick Start](#-cli) ｜ [Docs](https://docs.nexa.ai/en/nexa-sdk-go/NexaCLI) |
34+
| 🐍 Python | [Quick Start](#-python-sdk) ｜ [Docs](https://docs.nexa.ai/en/nexa-sdk-python/overview) |
35+
| 🤖 Android | [Quick Start](#-android-sdk) ｜ [Docs](https://docs.nexa.ai/en/nexa-sdk-android/overview) |
36+
| 🐳 Linux Docker | [Quick Start](#-linux-docker) ｜ [Docs](https://docs.nexa.ai/en/nexa-sdk-docker/overview) |
2337

24-
Download `geniex-cli-setup-windows-arm64-<TAG>.exe` and the matching `geniex-sdk-windows-arm64-<TAG>.zip`, then run the installer. For the latest stable installer without picking a tag, fetch [`https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/geniex-cli.exe`](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/geniex-cli.exe) — this S3 object always mirrors the newest stable release.
38+
---
2539

26-
If the SDK name ends in `-selfsigned`, first follow [notes/run.md § Self-signed fallback](notes/run.md#self-signed-fallback) to import `ggml-htp-v1.cer` and enable test-signing. Full walkthrough: [notes/run.md § Running a prebuilt CI release](notes/run.md#running-a-prebuilt-ci-release-windows-on-snapdragon).
40+
### 🖥️ CLI
2741

28-
### Linux ARM64
42+
**Download:**
2943

30-
Install on a Snapdragon device (EVK, container, or any ARM64 Linux with a Qualcomm BSP):
44+
| Windows | Linux |
45+
| -------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
46+
| [arm64 (Qualcomm NPU)](https://public-storage.nexa4ai.com/nexa_sdk/downloads/nexa-cli_windows_arm64.exe) | [arm64](https://github.com/NexaAI/nexa-sdk/releases/latest/download/nexa-cli_linux_arm64.sh) |
47+
| [x64](https://public-storage.nexa4ai.com/nexa_sdk/downloads/nexa-cli_windows_x86_64.exe) | [x64](https://github.com/NexaAI/nexa-sdk/releases/latest/download/nexa-cli_linux_x86_64.sh) |
3148

32-
```bash
33-
# Optional: verify QCOM driver and system-library prerequisites first.
34-
curl -fsSL https://raw.githubusercontent.com/qcom-ai-hub/geniex/main/cli/release/linux/check.sh | sh
3549

36-
curl -fsSL https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/install.sh | sh
37-
```
3850

39-
If the launcher's directory isn't on your `PATH` yet, the installer prints the exact line to add — typically:
51+
**NPU Access Token (required for NPU models):**
4052

41-
```bash
42-
echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.bashrc # or ~/.zshrc / ~/.profile
53+
> **Note:** Our previous token validation service has been deprecated. For any NPU usage, simply set the access token below — no additional registration or validation is needed.
54+
55+
For Windows:
56+
```shell
57+
$env:NEXA_TOKEN="key/eyJhY2NvdW50Ijp7ImlkIjoiNDI1Y2JiNWQtNjk1NC00NDYxLWJiOWMtYzhlZjBiY2JlYzA2In0sInByb2R1Y3QiOnsiaWQiOiJkYjI4ZTNmYy1mMjU4LTQ4ZTctYmNkYi0wZmE4YjRkYTJhNWYifSwicG9saWN5Ijp7ImlkIjoiMmYyOWQyMjctNDVkZS00MzQ3LTg0YTItMjUwNTYwMmEzYzMyIiwiZHVyYXRpb24iOjMxMTA0MDAwMH0sInVzZXIiOnsiaWQiOiI3MGE2YzA4NS1jYjc3LTQ3YmEtOWUxNC1lNjFjYTA2ZThmZjUiLCJlbWFpbCI6ImFsYW40QG5leGE0YWkuY29tIn0sImxpY2Vuc2UiOnsiaWQiOiI4OTlhZGQ2NS1lOTI2LTQ2M2ItODllNi0xMjc0NzM3ZjA1MzYiLCJjcmVhdGVkIjoiMjAyNS0wOS0wNlQwMDo1MzozNi4yMDNaIiwiZXhwaXJ5IjoiMjAzNS0xMi0zMVQyMzo1OTo1OS4wMDBaIn19.BXoUHIEzFMuuZbBT7RvsKO9nTi5950C6kHO64blF7XBnfKvZ6ClA8a55tmszI1ZWdngzpNFTzMM5PV5euuzMCA=="
4358
```
4459

45-
Open a new shell or `source` that file, then use it:
60+
For Linux / Android adb shell:
61+
```shell
62+
export NEXA_TOKEN="key/eyJhY2NvdW50Ijp7ImlkIjoiNDI1Y2JiNWQtNjk1NC00NDYxLWJiOWMtYzhlZjBiY2JlYzA2In0sInByb2R1Y3QiOnsiaWQiOiJkYjI4ZTNmYy1mMjU4LTQ4ZTctYmNkYi0wZmE4YjRkYTJhNWYifSwicG9saWN5Ijp7ImlkIjoiMmYyOWQyMjctNDVkZS00MzQ3LTg0YTItMjUwNTYwMmEzYzMyIiwiZHVyYXRpb24iOjMxMTA0MDAwMH0sInVzZXIiOnsiaWQiOiI3MGE2YzA4NS1jYjc3LTQ3YmEtOWUxNC1lNjFjYTA2ZThmZjUiLCJlbWFpbCI6ImFsYW40QG5leGE4YWkuY29tIn0sImxpY2Vuc2UiOnsiaWQiOiI4OTlhZGQ2NS1lOTI2LTQ2M2ItODllNi0xMjc0NzM3ZjA1MzYiLCJjcmVhdGVkIjoiMjAyNS0wOS0wNlQwMDo1MzozNi4yMDNaIiwiZXhwaXJ5IjoiMjAzNS0xMi0zMVQyMzo1OTo1OS4wMDBaIn19.BXoUHIEzFMuuZbBT7RvsKO9nTi5950C6kHO64blF7XBnfKvZ6ClA8a55tmszI1ZWdngzpNFTzMM5PV5euuzMCA=="
63+
```
64+
65+
**Run your first model:**
4666

4767
```bash
48-
geniex pull Qwen/Qwen3-0.6B-GGUF
49-
geniex infer Qwen/Qwen3-0.6B-GGUF -p "Hello, in one short sentence please."
68+
# Chat with Qwen3
69+
nexa infer ggml-org/Qwen3-1.7B-GGUF
70+
71+
# Multimodal: drag images into the CLI
72+
nexa infer NexaAI/Qwen3-VL-4B-Instruct-GGUF
73+
74+
# NPU (Windows arm64 with Snapdragon X Elite)
75+
nexa infer NexaAI/OmniNeural-4B
5076
```
5177

52-
Pin a version: `... | sh -s -- --version v0.1.8`. Override the install location: `... | sh -s -- --prefix /opt/geniex`. Other flags: `-q`, `--help`.
78+
- **Models:** LLM, Multimodal, ASR, OCR, Rerank, Object Detection, Image Generation, Embedding
79+
- **Formats:** GGUF, NEXA
80+
- 📖 [CLI Reference Docs](https://docs.nexa.ai/en/nexa-sdk-go/NexaCLI)
5381

5482
---
5583

56-
Prefer Docker (versioned image, repeatable, no host-side install):
84+
### 🐍 Python SDK
5785

5886
```bash
59-
docker pull ghcr.io/qcom-ai-hub/geniex-cli:<TAG>
60-
61-
# interactive mode
62-
docker run -it --rm --privileged \
63-
-v "$PWD/data:/data" \
64-
-v /usr/lib:/opt/qcom-lib:ro \
65-
ghcr.io/qcom-ai-hub/geniex-cli:<TAG> \
66-
infer Qwen/Qwen3-0.6B-GGUF
67-
68-
# server mode
69-
docker run -it --rm --privileged \
70-
-v "$PWD/data:/data" \
71-
-v /usr/lib:/opt/qcom-lib:ro \
72-
--network=host \
73-
ghcr.io/qcom-ai-hub/geniex-cli:<TAG> \
74-
serve
75-
# interactive shell connect to server
76-
docker run -it --rm --privileged \
77-
-v "$PWD/data:/data" \
78-
-v /usr/lib:/opt/qcom-lib:ro \
79-
--network=host \
80-
ghcr.io/qcom-ai-hub/geniex-cli:<TAG> \
81-
run <model>
87+
pip install nexaai
8288
```
8389

84-
`--privileged` exposes the NPU/GPU devices and `./data` persists the model cache. `:latest` tracks the most recent stable tag.
90+
```python
91+
from nexaai import LLM, GenerationConfig, ModelConfig, LlmChatMessage
8592

86-
### Python
93+
llm = LLM.from_(model="NexaAI/Qwen3-0.6B-GGUF", config=ModelConfig())
8794

88-
```bash
89-
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple geniex
95+
conversation = [
96+
LlmChatMessage(role="user", content="Hello, tell me a joke")
97+
]
98+
prompt = llm.apply_chat_template(conversation)
99+
for token in llm.generate_stream(prompt, GenerationConfig(max_tokens=100)):
100+
print(token, end="", flush=True)
90101
```
91102

92-
The sdist auto-downloads the matching SDK zip per host at install time. API, CLI (`geniex-py`), and env vars: [bindings/python/README.md](bindings/python/README.md). Install sources (GitHub Release URL, offline mirror): [bindings/python/BUILD.md § Install sources](bindings/python/BUILD.md#install-sources).
103+
- **Models:** LLM, Multimodal, ASR, OCR, Rerank, Object Detection, Image Generation, Embedding
104+
- **Formats:** GGUF, NEXA
105+
- 📖 [Python SDK Docs](https://docs.nexa.ai/en/nexa-sdk-python/quickstart)
106+
107+
---
93108

94-
### Android (AAR)
109+
### 🤖 Android SDK
95110

96-
Download `geniex-android-aar-<TAG>.aar` from the Releases page and reference it as a local dependency:
111+
Add to your `app/AndroidManifest.xml`
112+
113+
```xml
114+
<application android:extractNativeLibs="true">
115+
```
116+
117+
Add to your `build.gradle.kts`:
97118

98119
```kotlin
99-
// settings.gradle.kts
100-
dependencyResolutionManagement {
101-
repositories { flatDir { dirs("libs") } }
120+
dependencies {
121+
implementation("ai.nexa:core:0.0.19")
102122
}
103-
// app/build.gradle.kts
104-
dependencies { implementation(files("libs/geniex-android-aar-<TAG>.aar")) }
105123
```
106124

107-
API and architecture: [bindings/android/README.md](bindings/android/README.md).
125+
```kotlin
126+
// Initialize SDK
127+
NexaSdk.getInstance().init(this)
128+
129+
// Load and run model
130+
VlmWrapper.builder()
131+
.vlmCreateInput(VlmCreateInput(
132+
model_name = "omni-neural",
133+
model_path = "/data/data/your.app/files/models/OmniNeural-4B/files-1-1.nexa",
134+
plugin_id = "npu",
135+
config = ModelConfig()
136+
))
137+
.build()
138+
.onSuccess { vlm ->
139+
vlm.generateStreamFlow("Hello!", GenerationConfig()).collect { print(it) }
140+
}
141+
```
142+
143+
- **Requirements:** Android minSdk 27, Qualcomm Snapdragon 8 Gen 4 Chip
144+
- **Models:** LLM, Multimodal, ASR, OCR, Rerank, Embedding
145+
- **NPU Models:** [Supported Models](https://docs.nexa.ai/en/nexa-sdk-android/overview#supported-models)
146+
- 📖 [Android SDK Docs](https://docs.nexa.ai/en/nexa-sdk-android/quickstart)
147+
148+
---
149+
150+
### 🐳 Linux Docker
151+
152+
```bash
153+
docker pull nexa4ai/nexasdk:latest
154+
155+
export NEXA_TOKEN="your_token_here"
156+
docker run --rm -it --privileged \
157+
-e NEXA_TOKEN \
158+
nexa4ai/nexasdk:latest infer NexaAI/Granite-4.0-h-350M-NPU
159+
```
160+
161+
- **Requirements:** Qualcomm Dragonwing IQ9, ARM64 systems
162+
- **Models:** LLM, VLM, ASR, CV, Rerank, Embedding
163+
- **NPU Models:** [Supported Models](https://docs.nexa.ai/en/nexa-sdk-docker/overview#supported-models)
164+
- 📖 [Linux Docker Docs](https://docs.nexa.ai/en/nexa-sdk-docker/quickstart)
165+
166+
---
167+
168+
## ⚙️ Features & Comparisons
169+
170+
<div align="center">
171+
172+
| Features | **NexaSDK** | **Ollama** | **llama.cpp** | **LM Studio** |
173+
| ---------------------------------------- | ---------------------------------------------------------- | ---------- | ------------- | ------------- |
174+
| NPU support | ✅ NPU-first | ❌ | ❌ | ❌ |
175+
| Android SDK support | ✅ NPU/GPU/CPU support | ⚠️ | ⚠️ | ❌ |
176+
| Linux support (Docker image) | ✅ | ✅ | ✅ | ❌ |
177+
| Day-0 model support | ✅ | ❌ | ⚠️ | ❌ |
178+
| Full multimodality support | ✅ Image, Audio, Text, Embedding, Rerank, ASR, TTS | ⚠️ | ⚠️ | ⚠️ |
179+
| Cross-platform support | ✅ Desktop, Mobile (Android), Automotive, IoT (Linux) | ⚠️ | ⚠️ | ⚠️ |
180+
| One line of code to run | ✅ | ✅ | ⚠️ | ✅ |
181+
| OpenAI-compatible API + Function calling | ✅ | ✅ | ✅ | ✅ |
182+
183+
<p align="center" style="margin-top:14px">
184+
<i>
185+
<b>Legend:</b>
186+
<span title="Full support">✅ Supported</span> &nbsp; | &nbsp;
187+
<span title="Partial or limited support">⚠️ Partial or limited support </span> &nbsp; | &nbsp;
188+
<span title="Not Supported">❌ No</span>
189+
</i>
190+
</p>
191+
</div>
192+
193+
## 🙏 Acknowledgements
194+
195+
We would like to thank the following projects:
196+
197+
- [ggml](https://github.com/ggml-org/ggml)
198+
- [mlx-lm](https://github.com/ml-explore/mlx-lm)
199+
- [mlx-vlm](https://github.com/Blaizzy/mlx-vlm)
200+
- [mlx-audio](https://github.com/Blaizzy/mlx-audio)
201+
202+
## 📄 License
108203

109-
### SDK zip (integrators)
204+
NexaSDK uses a dual licensing model:
110205

111-
Extract `geniex-sdk-<os>-arm64-<TAG>.zip` and point your build at its `include/` and `lib/` directories. To build the SDK in-tree instead, see [notes/build.md § Build the SDK](notes/build.md#build-the-sdk).
206+
### CPU/GPU Components
112207

113-
## Documentation
114-
To use `geniex`, please refer to [docs](docs/README.md) for detailed guides and API references.
208+
Licensed under [Apache License 2.0](LICENSE).
115209

210+
### NPU Components
116211

117-
For contribution to this project, see docs below to build and test your changes.
212+
- **Personal Use**: Free license key available from [Nexa AI Model Hub](https://sdk.nexa.ai/model). Each key activates 1 device for NPU usage.
213+
- **Commercial Use**: Contact [hello@nexa.ai](mailto:hello@nexa.ai) for licensing.
118214

119-
| File | Topic |
120-
| ---------------------------------- | --------------------------------------------------------------------- |
121-
| [notes/build.md](notes/build.md) | Build CLI, SDK, and Python bindings (Linux / Windows ARM64 / Android) |
122-
| [notes/run.md](notes/run.md) | Runtime / compute-unit selection, model pull, Windows self-signed HTP fallback |
123-
| [notes/release.md](notes/release.md) | SemVer tag procedure, channels, Hexagon HTP signing pipeline |
124-
| [notes/AI.md](notes/AI.md) | Claude Code integration (slash commands, skills) |
125-
| [CONTRIBUTING.md](CONTRIBUTING.md) | Commits, branches, PR format, FFI-update rule |
215+
## 🤝 Contact & Community Support
126216

127-
## License
217+
Want more model support, backend support, device support or other features? We'd love to hear from you!
128218

129-
Apache 2.0 — see [LICENSE](LICENSE) and [NOTICE](NOTICE).
219+
Feel free to [submit an issue](https://github.com/NexaAI/nexa-sdk/issues) on our GitHub repository with your requests, suggestions, or feedback. Your input helps us prioritize what to build next.

0 commit comments

Comments
 (0)