Raqeebah is a self-hosted platform made for the collective monitoring and management of your application logs, scheduled job logs, crash logs, etc. in a structured manner. It provides one centralized dashboard to view and structure live broadcasts of data from all across your services.
-
Raqeebah CLI: Put
raqeebah runin front of any command and it broadcasts that command's output to Raqeebah:raqeebah run [project name] [process name] -- python app.py raqeebah run [project name] [process name] -- node index.js raqeebah run [project name] [process name] -- ./process.sh raqeebah run [project name] [process name] -- process.exe
It is a transparent prefix, not a daemon: it stays in the foreground, prints the command's output to your terminal as it arrives, and exits with the command's own exit code. So it can be dropped into a cron entry or a systemd
ExecStartwithout changing how that thing behaves, and backgrounding stays yours to decide — as withtimeornohup.The
--is only needed when the command itself begins with a hyphen. -
Raqeebah file change tracking: Follows a text file and reports new lines as they are written:
raqeebah watch [project name] [process name] file.txt
Also foreground, and long-lived by nature — run it under systemd or a supervisor if you want it to survive a logout. It follows the file through rotation and truncation, starts from the end of the file rather than replaying its history (
--from-startif you want that), and remembers where it had reached, so lines written while it was stopped are still picked up when it starts again. -
Raqeebah SDK: For when you want to send from inside the program rather than watch it from outside. The SDKs do two things:
- Trackers around specified functions, recording initiation, execution, and success or failure. Unlike the CLI, an SDK knows what it is measuring, so it can assert
successorfailureoutright. - Logging hooks attached once at startup to your existing setup (Rust's
tracingorlog, Python'slogging, Node'swinston/pino). After that every call through that logger is captured and tagged with its source, with nothing else to change.
All three exist — Rust in
crates/sdk, Node.js insdks/node, Python insdks/python. They have the same surface, the same guarantees and the same bytes on the wire, down to a signing test vector all three pin; the buffer file is the same format too, so a service can move between them without losing what was waiting. See Rust SDK, Node SDK and Python SDK below for how to add each one, and theexamples/directory beside each for programs you can run. - Trackers around specified functions, recording initiation, execution, and success or failure. Unlike the CLI, an SDK knows what it is measuring, so it can assert
- Remote file changes
- Changes over the network
- Async/Buffered logging may sometimes delay in appearance, but won't be lost.
All three collection points send events to the central server over HTTP POST, authenticated per service with an API key, an HMAC signature and an expiry window. Point them at a server with the raqeebah init command or through SDK configuration.
The dashboard is where the data is used: projects, users and permissions, live event and run views, comparison mode, advanced search and similarity search.
Each execution is a run. Its events carry a status: log for ordinary output, init when the run starts, and success, failure or finished when it ends. The CLI reports finished for any non-zero exit code and puts the code in the event's metadata, because an exit code means different things to different programs. An SDK knows what it is measuring and asserts success or failure outright.
Events are buffered to local SQLite before they are sent and cleared only once the server confirms them, so an outage or a crash delays delivery rather than losing it. The server meters ingestion per service — 1000 events a second sustained with a 20,000-event burst, INGEST_EVENTS_PER_SECOND and INGEST_BURST_EVENTS to change it, 0 to disable — and answers 429 with a Retry-After past that. Shutdown on SIGTERM is graceful, bounded by SHUTDOWN_TIMEOUT_SECONDS (default 30).
Signing in is metered too, and for a different reason. Password verification is deliberately expensive — that is what makes a stolen password database hard to use — so an unmetered login endpoint is a way to make the server burn its own CPU on request. Three bounds apply: LOGIN_ATTEMPTS_PER_MINUTE per account (default 10), LOGIN_GLOBAL_ATTEMPTS_PER_SECOND across all accounts (default 20, which catches someone cycling through addresses), and LOGIN_MAX_CONCURRENT verifications at once (default 4). Past any of them the answer is 429 with a Retry-After, decided before the password is checked and before anything is written to the audit log. DATABASE_MAX_CONNECTIONS (default 20) sizes the pool the last of those is measured against.
Request bodies are capped per route: 2 MiB for a batch of events, 4 KiB for a sign-in, 64 KiB for the dashboard's own calls.
Off by default, and experimental. When enabled, the server embeds every stored log line and finds events by meaning rather than by words, either from an existing event or from free text. Embedding happens in-process; nothing is sent anywhere.
It is expensive. Every message is embedded on the same CPU that serves ingest. Leave it off on a busy server.
To enable it:
- Create the pgvector extension once as a superuser:
CREATE EXTENSION vector;. The server does not do this itself, for the same reason it does not run migrations. - Point
EMBEDDING_MODEL_PATHat a directory containingconfig.json,tokenizer.jsonandmodel.safetensors— potion-base-8M is a good default at about 30 MB — and setENABLE_VECTOR=true. Without a model path the feature stays off whateverENABLE_VECTORsays.
Only events recorded after it is switched on are searchable. There is no backfill. From that point coverage is guaranteed: every event carrying a message is queued for embedding in the same transaction that stores the event, so a crash, a restart or an outage delays embedding rather than skipping it. Coverage is eventual, and the search page reports what is still pending and what has been quarantined after repeated failures.
Long messages are embedded from roughly their first 2 KB. Stop the old server before starting a new one on upgrade — a previous binary still serving ingest writes no queue rows. Changing model means dropping event_embeddings and embedding_epoch together; dropping the table alone leaves a stale enabled-from date and the server treats uncovered history as covered. A redacted event has no embedding and cannot acquire one.
For sending from inside the program rather than watching it from outside. It is not published to crates.io yet, so depend on it from git and pin the tag:
[dependencies]
raqeebah-sdk = { git = "https://github.com/SannanOfficial/Raqeebah", tag = "v0.1.0" }Two optional features, both off by default: tracing gives you a tracing_subscriber::Layer, and log gives you a log::Log implementation. Neither is needed for track and run.
use raqeebah_sdk::{Buffering, Config, Raqeebah};
let raqeebah = Raqeebah::connect(
Config::new(server, service_id, api_key)
.buffer(Buffering::Durable("/var/lib/myapp/raqeebah".into())),
).await?;
// A run you drive yourself.
let run = raqeebah.run("nightly-report").await?;
run.log("gathering yesterday's rows").await?;
run.success().await?;
// Or the same thing around a future.
raqeebah.track("nightly-report", async { generate_report().await }).await?;
raqeebah.shutdown(Duration::from_secs(5)).await?;connect is a real handshake, not just construction — it calls POST /services/whoami, so a wrong key fails in front of you rather than surfacing later as a rejected signature on a background thread. It also opens an ambient run that stays open for the life of the process; bare raqeebah.log(…) calls and both logging hooks attach to that one.
track returns your future's own Result untouched, and re-raises a panic after recording it with {"panicked": true} — Raqeebah watches the work rather than taking part in it, so a recording problem never becomes your failure. Those problems go to a Reporter instead, which you can supply to route them wherever your other operational messages go.
log(…).await returns once the event is committed to the buffer, not once the server has it. The shutdown is not optional: Drop cannot flush, because there is no async in Drop, so without it whatever is still buffered waits for the next start.
Try it. crates/sdk/examples/ holds five runnable programs. They send real events, so they need a running server and a service registered in the dashboard:
export RAQEEBAH_SERVER=https://logs.internal:3000
export RAQEEBAH_SERVICE_ID=[id from the dashboard]
export RAQEEBAH_API_KEY=[key shown once at registration]
cargo run -p raqeebah-sdk --example quickstart # connect, log, shut down
cargo run -p raqeebah-sdk --example tracked_job # runs, track, failures, panics
cargo run -p raqeebah-sdk --example delivery_health # buffering and a custom Reporter
cargo run -p raqeebah-sdk --example tracing_hook --features tracing
cargo run -p raqeebah-sdk --example log_hook --features logStart with quickstart; it is about thirty lines. delivery_health is the one to read if you want to know what happens when the server is down — stop the server while it runs and watch the reporter.
The same SDK for Node.js and TypeScript, in sdks/node. Not on npm yet, so build it from this repository and depend on the directory:
cd sdks/node && npm ci && npm run buildNode 22.5 or later, because the durable buffer is node:sqlite — the same SQLite outbox the Rust SDK writes, in the same {service_id}.db format. On 22.5 to 23.3 that builtin needs --experimental-sqlite; from 23.4 it does not. There are no runtime dependencies: node:crypto, node:sqlite and the global fetch are the whole of it.
import { Raqeebah } from "@raqeebah/sdk";
const raqeebah = await Raqeebah.connect({
server, serviceId, apiKey,
buffer: { kind: "durable", dir: "/var/lib/myapp/raqeebah" },
});
// A run you drive yourself.
const run = await raqeebah.run("nightly-report");
await run.log("gathering yesterday's rows");
await run.success();
// Or the same thing around some work.
await raqeebah.track("nightly-report", () => generateReport());
await raqeebah.shutdown(5_000);Everything the Rust section above says still holds: connect is a real handshake and opens the ambient run, log resolves once the event is committed to the buffer rather than once the server has it, delivery trouble goes to a Reporter, and the shutdown is not optional — Node's exit event is synchronous and cannot drain, so without it whatever is still buffered waits for the next start.
The logging hooks are winston and pino here, in place of Rust's tracing and log, and they are subpath imports — @raqeebah/sdk/pino and @raqeebah/sdk/winston — so the core pulls in nothing:
import { pinoStream } from "@raqeebah/sdk/pino";
const logger = pino(pino.multistream([
{ stream: process.stdout },
{ stream: pinoStream(raqeebah) },
]));Try it. sdks/node/examples/ holds the same five programs as the Rust SDK's, against the same three environment variables:
node examples/quickstart.ts # connect, log, shut down
node examples/tracked-job.ts # runs, track, failures, throws
node examples/delivery-health.ts # buffering and a custom Reporter
node examples/pino-hook.ts
node examples/winston-hook.tssdks/node/README.md has the full surface, and a short list of the places Node forced a choice the Rust SDK did not have to make.
The same SDK for Python, in sdks/python. Not on PyPI yet, so install it from this repository:
pip install "raqeebah-sdk @ git+https://github.com/SannanOfficial/Raqeebah@v0.1.0#subdirectory=sdks/python"Python 3.10 or later, and no runtime dependencies at all — sqlite3, hmac, hashlib, uuid, http.client and logging are the whole of it. The buffer is the same SQLite outbox, in the same {service_id}.db format, as the other two write.
The API is synchronous. crates/core is already blocking and thread-based, so this is that design in Python rather than a simplification of it — and it is what logging.Handler.emit, cron scripts, Celery tasks and Django commands all want.
from raqeebah import Buffering, Config, Raqeebah
raqeebah = Raqeebah.connect(Config(
server=server, service_id=service_id, api_key=api_key,
buffer=Buffering.durable("/var/lib/myapp/raqeebah"),
))
# A run you drive yourself.
run = raqeebah.run("nightly-report")
run.log("gathering yesterday's rows")
run.success()
# Or the same thing around some work, in either spelling.
@raqeebah.track("nightly-report")
def generate_report(): ...
with raqeebah.track("nightly-report") as run:
run.log("gathering yesterday's rows")
raqeebah.shutdown(5.0)Everything the Rust section above says still holds: connect is a real handshake and opens the ambient run, log returns once the event is committed to the buffer rather than once the server has it, delivery trouble goes to a Reporter, and the shutdown is not optional — an atexit handler cannot drain, because draining is a network wait and by then there is no time left to spend on one. Timeouts are seconds, as a float, since that is what time.sleep and Thread.join take.
The logging hook is the standard library's logging, in place of Rust's tracing and log:
from raqeebah.logging import RaqeebahHandler
logging.getLogger("billing").addHandler(RaqeebahHandler(raqeebah))It drops records from the raqeebah logger, and that is load-bearing rather than tidiness: the default reporter writes delivery trouble through that same facade, so without it a server being down would produce a warning, the warning would become an event, and the event would be queued for the server that is down.
Try it. sdks/python/examples/ holds four programs, against the same three environment variables:
python examples/quickstart.py # connect, log, shut down
python examples/tracked_job.py # runs, track in both spellings, failures
python examples/delivery_health.py # buffering and a custom Reporter
python examples/logging_hook.pyFour rather than five: the other two SDKs each have two hook demos because they have two logging facades, and Python has one.
sdks/python/README.md has the full surface, and a short list of the places Python forced a choice.
1. Run the server. Take a release binary — raqeebah-server-[version]-[target].tar.gz, published for x86-64 and ARM64 Linux, with the schema inside it — or build it with cargo build --release, which puts raqeebah-server in target/release/. Give it a DATABASE_URL, and apply the schema from crates/server/migrations/ — the server does not run migrations itself, deliberately: applying them needs privileges the running server should not hold. It binds 0.0.0.0:3000.
2. Set up the dashboard and use it to create your projects and users, and to register a service for each process you want watched. Registering shows the API key once, and beside it the exact raqeebah init service … command to run on the machine that process lives on.
3. Install the CLI wherever you want to send from:
curl -fsSL https://raqeebah.syedsannan.com/install.sh | shThe script verifies a SHA-256 checksum before installing anything and never calls sudo. If piping a script into a shell is not to your taste, every release publishes binaries and a SHA256SUMS file to check them against, so download, verify and move it onto your PATH by hand.
Then point it at your server and paste the key:
raqeebah init server https://your-raqeebah-server:3000
raqeebah init service [project name] [process name] --service-id [id from the dashboard]The key is read from a prompt rather than an argument, so it stays out of your shell history, and the config file it is written to is created 0600. The CLI only ever holds a service key, which can do nothing but send events for that one service — it never sees a dashboard password.
4. Start sending, using the commands above or an SDK — Rust, Node or Python. The dashboard is realtime: it holds an authenticated event stream open, so new events, runs and services appear as they arrive.
For detailed documentation, visit https://raqeebah.syedsannan.com. Suggestions and bug reports go in the repository's Issues section, or through the contact form on the website.
Contributions are welcome. Two rules, both about making changes reviewable:
Keep a pull request to one thing. One or two commits, one context. A PR carrying a broad list of features is not accepted however good each item is — it cannot be reviewed as a unit, and it cannot be reverted as one either. Send the features separately.
Using an LLM is fine, as long as you have reviewed what it wrote. There is no disclosure ritual and no penalty for the tool. The condition is ownership: you have read every line, you understand why each one is there, and you can answer for it in review. Code the author cannot explain is what gets turned away, whoever or whatever wrote it.
CONTRIBUTING.md has the longer form, including the exact checks CI runs so you can run them first.
Raqeebah is developed in the open and supported on Patreon. Every tier is listed there, along with what each one includes.
Gold Sponsors — none yet.
Silver Sponsors — none yet.
Bronze Sponsors — none yet.
Supporters — none yet.
This project is licensed under the MIT license as included in the LICENSE file.