Admin: Tutorial about CrateDB monitoring with Prometheus and Grafana #302
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughAdds a new monitoring tutorial for Prometheus/Grafana, links it into admin and integration sections, and updates cross-links. Introduces Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Admin as Admin
participant Crate as CrateDB Cluster
participant JMX as Crate JMX HTTP Exporter
participant SQLExp as Prometheus SQL Exporter
participant NodeExp as Prometheus Node Exporter
participant Prom as Prometheus
participant Graf as Grafana
Admin->>Crate: Configure cluster + memory settings
Admin->>JMX: Enable exporter on each node
Admin->>NodeExp: Install/configure
Admin->>SQLExp: Install/configure (create sql_exporter user)
Admin->>Prom: Configure scrape targets (node, cratedb_jmx, sql_exporter)
Admin->>Graf: Configure Prometheus data source, import dashboard
Prom->>JMX: Scrape CrateDB JMX metrics
Prom->>NodeExp: Scrape OS/node metrics
Prom->>SQLExp: Scrape SQL-based metrics
SQLExp->>Crate: Execute metrics queries
Graf->>Prom: Query time series for dashboards
Prom-->>Graf: Return metrics
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Pre-merge checks✅ Passed checks (3 passed)
|
amotl
force-pushed
the
monitoring-prometheus-grafana
branch
from
2900d54 to
4eb547b
Compare
amotl
changed the title
amotl
force-pushed
the
monitoring-prometheus-grafana
branch
from
4eb547b to
ca9d644
Compare
amotl
changed the title
amotl
force-pushed
the
monitoring-prometheus-grafana
branch
from
ca9d644 to
ba78343
Compare
|
@coderabbitai review When suggesting changes, please use active voice instead of first-person or passive voice, thanks! |
|
@amotl I'll review the PR and ensure I use active voice when suggesting changes. ✏️ Learnings added
✅ Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (12)
docs/admin/monitoring/prometheus-grafana.md (11)
6-8: Use direct, active voice; remove “we/I” phrasing.Tighten the intro and switch to imperative tone.
-If you are running CrateDB in a production environment, you have probably wondered what would be the best way to monitor the servers to identify issues before they become problematic and to collect statistics that you can use for capacity planning. +In production, monitor CrateDB proactively to catch issues early and collect statistics for capacity planning. -We recommend pairing two well-known OSS solutions, [Prometheus](https://prometheus.io/) which is a system that collects and stores performance metrics, and [Grafana](https://grafana.com/) which is a system to create dashboards. +Pair two OSS tools: use [Prometheus](https://prometheus.io/) to collect and store metrics, and [Grafana](https://grafana.com/) to build dashboards.
16-22: Avoid first-person and conversational tone; state scope crisply.-Things are a bit different of course if you are using containers, or if you are using the fully-managed cloud-hosted [CrateDB Cloud](https://cratedb.com/products/cratedb-cloud), but let’s see how all this works on an on-premises installation by setting all this up together. +Containerized and [CrateDB Cloud](https://cratedb.com/products/cratedb-cloud) setups differ. This tutorial targets on‑premises installations.
24-35: Remove first‑person; clarify required steps and why.-In my case, I am using Ubuntu and I did it like this, first I ssh to the first machine and run: +On Ubuntu, start on the first node and run: @@ -This is a configuration file that will be used by CrateDB, we only need one line to configure memory settings here (this is a required step otherwise we will fail bootstrap checks): +This configuration file sets the JVM heap. Configure it to satisfy bootstrap checks:
112-116: Append the javaagent; don’t overwrite existing CRATE_JAVA_OPTS.Overwriting may drop other required JVM flags. Instruct to append.
-CRATE_JAVA_OPTS="-javaagent:/usr/share/crate/lib/crate-jmx-exporter-1.0.0.jar=8080" +# Append to existing options (preserve other flags) +CRATE_JAVA_OPTS="${CRATE_JAVA_OPTS:-} -javaagent:/usr/share/crate/lib/crate-jmx-exporter-1.0.0.jar=8080"Also advise restricting network access to the exporter port via firewall/security groups.
126-131: Note default listen address; suggest binding/ACL.Node Exporter typically listens on all interfaces. Add a note to bind to loopback or firewall the port in production.
143-147: Creating users via HTTP: call out security.Transmitting credentials over HTTP even on localhost can leak via proxies/logs. Recommend HTTPS when available or running from the node over the Postgres protocol with a local client.
248-259: Fix duplicated wording; keep jobs concise.Tighten the sentence and ensure job list reflects intended targets.
-We replace this with the below configuration, which reflects port 8080 (Crate JMX Exporter), port 9100 (Prometheus Node Exporter), port 9237 (Prometheus SQL Exporter), as well as port 9100 (Prometheus Node Exporter). +Replace it with the following jobs: port 9100 (Node Exporter), port 8080 (Crate JMX Exporter), and port 9237 (SQL Exporter).
275-281: Avoid bare URLs; satisfy markdownlint MD034.Render endpoints as links or code, not bare URLs.
-If you now point your browser to *http://<Grafana host>:3000* you will be welcomed by the Grafana login screen, the first time you can log in with admin as both the username and password, make sure to change this password right away. +Open `http://<grafana-host>:3000` to access the Grafana login screen. The default credentials are `admin`/`admin`; change the password immediately. @@ -then click on "Prometheus", and enter the URL *http://\<Prometheus host>:9090*. +then click "Prometheus" and set the URL to `http://<prometheus-host>:9090`.
283-285: Avoid hotlinking external images in docs.Store the dashboard screenshot in the repo’s static assets to ensure offline builds and reproducibility.
291-303: Deduplicate “Thread pool queue size” metric.The bullet appears twice. Keep one entry.
* Thread pool queue size: `sum(crate_threadpools{property="queueSize"}) by (name)` @@ - * Thread pool queue size: `crate_threadpools{property="queueSize"}`
102-116: Update pinned crate-jmx-exporter version and document host bindingFile: docs/admin/monitoring/prometheus-grafana.md (lines 102–116)
- Replace hardcoded crate-jmx-exporter-1.0.0.jar with the latest stable (1.2.0 — released Aug 20, 2024) or use a version variable and document how to find the current release.
- Document that the agent accepts host:port and update the CRATE_JAVA_OPTS example to show binding a specific IP (e.g. CRATE_JAVA_OPTS="-javaagent:/usr/share/crate/lib/crate-jmx-exporter-1.2.0.jar=127.0.0.1:8080").
docs/integrate/prometheus/index.md (1)
124-138: Resolve potential MD053: confirm link reference usage.Ensure the
[CrateDB and Prometheus]reference is actually rendered by Sphinx in this page; otherwise remove the definition or link inline.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
docs/admin/index.md(1 hunks)docs/admin/monitoring/prometheus-grafana.md(1 hunks)docs/integrate/grafana/index.md(1 hunks)docs/integrate/prometheus/index.md(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-09T16:19:43.966Z
Learnt from: amotl
PR: crate/cratedb-guide#238
File: docs/integrate/azure-functions/learn.rst:1-1
Timestamp: 2025-08-09T16:19:43.966Z
Learning: In the CrateDB Guide documentation, main integration anchors (e.g., `azure-functions`) are intentionally placed in the `index.md` files of their respective integration folders, while detailed tutorials use the `-learn` suffix (e.g., `azure-functions-learn`) in their `learn.rst` or `learn.md` files. This is a deliberate architectural pattern for the documentation restructuring.
Applied to files:
docs/integrate/grafana/index.md
🪛 markdownlint-cli2 (0.17.2)
docs/admin/monitoring/prometheus-grafana.md
61-61: Bare URL used
(MD034, no-bare-urls)
137-137: Link and image reference definitions should be needed
Unused link or image reference definition: "cratedb and prometheus"
(MD053, link-image-reference-definitions)
🔇 Additional comments (3)
docs/admin/index.md (1)
35-35: LGTM: good placement under Cluster.The “Monitoring” entry fits between sharding/partitioning and performance.
docs/integrate/prometheus/index.md (1)
110-121: LGTM: solid cross-link from Prometheus integration.The See also card improves discoverability of the monitoring tutorial.
docs/integrate/grafana/index.md (1)
54-66: LGTM: See also card aligns with integration pattern.The cross-link to the admin tutorial is helpful and follows the integrations’ index.md anchor practice.
This comment was marked as resolved.
This comment was marked as resolved.
amotl
force-pushed
the
monitoring-prometheus-grafana
branch
from
ba78343 to
4f59beb
Compare
amotl
added
cross linking
| Containerized and [CrateDB Cloud] setups differ. This tutorial targets | ||
| standalone and on‑premises installations. | ||
|
|
||
| ## First we need a CrateDB cluster |
There was a problem hiding this comment.
Isn't this topic already covered elsewhere? Could we link to the existing "install a cluster" content instead? This would avoid repeating and also avoids adjusting lot of places if we need to adjust anything on the setup guide.
| ```shell | ||
| echo "deb https://packages.grafana.com/oss/deb stable main" | tee -a /etc/apt/sources.list.d/grafana.list | ||
| wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add - | ||
| apt update | ||
| apt install grafana | ||
| systemctl start grafana-server | ||
| ``` |
There was a problem hiding this comment.
Maybe linking to the grafana doc instead (or additional)? https://grafana.com/docs/grafana/latest/setup-grafana/installation/debian/
Same points like before, if something changes on the install documentation, we do not have to adjust it here.
| (prometheus-tutorial)= | ||
| # Storing long-term metrics with Prometheus in CrateDB | ||
|
|
||
| In this tutorial, I show how to |
There was a problem hiding this comment.
| In this tutorial, I show how to | |
| This tutorial will show how to |
There was a problem hiding this comment.
@coderabbitai review once more and please converge all spots to active voice instead of first-person or passive voice, as suggested by @seut. Thanks.
There was a problem hiding this comment.
🧩 Analysis chain
@amotl, I'll review the entire file for voice consistency and provide suggestions to use active voice throughout the documentation. Let me examine the full content first to identify all instances that need revision.
🏁 Script executed:
#!/bin/bash
# Get the full content of the tutorial file to review for voice usage
cat docs/integrate/prometheus/tutorial.md
Length of output: 8044
@amotl, I've reviewed the entire tutorial file and identified multiple instances where first-person and passive voice should be converted to active voice. Here are my comprehensive suggestions:
Title and Introduction:
| In this tutorial, I show how to | |
| This tutorial shows how to |
Throughout the document, replace first-person references:
In the Docker Compose section:
| In this tutorial, I show how to | |
| Both CrateDB, Prometheus, and the CrateDB Prometheus Adapter applications can be run as [Docker containers](https://www.docker.com/resources/what-container). To centralize the container management, use [Docker Compose](https://docs.docker.com/compose/), which allows building and running all containers with a single command and setting up connections between them in a YAML file. | |
| Before starting, follow the [Docker Installation Tutorial](https://docs.docker.com/get-docker/) to install Docker on the local machine. | |
| Then, create a directory on the local machine to host the necessary configuration files. | |
| The setup requires three configuration files, all following the YAML format. Create these files using any [text editor](https://www.computerhope.com/jargon/e/editor.htm), like TextEdit on a Mac, and save them with the `.yml` format. |
For the docker-compose.yml section:
| In this tutorial, I show how to | |
| The first YAML file to create is `docker-compose.yml`, which wraps up the configurations for the three containers. | |
| Specify CrateDB, Prometheus, and Adapter as services. Then, add `config.yml` and `prometheus.yml` files as volumes to the Adapter and Prometheus containers, respectively. The following steps will create these files. |
For the prometheus.yml section:
| In this tutorial, I show how to | |
| Next, following the [Prometheus Documentation](https://prometheus.io/docs/prometheus/latest/getting_started/), create a `prometheus.yml` file, which holds the scraping configuration for whichever service Prometheus collects metrics from. | |
| To keep it simple, follow the example in the Prometheus documentation and configure it to monitor itself. | |
| Configure `remote_write` and `remote_read` to forward requests from Prometheus to the CrateDB Adapter by setting them to the Adapter URL, as stated in [CrateDB Prometheus Adapter Setup](https://github.com/crate/cratedb-prometheus-adapter). | |
| When running the Adapter on Docker instead of locally, the host in its URL will not be `localhost`, but rather the name given to the Adapter service in the `docker-compose.yml` file, in this case, `cratedb-prometheus-adapter`. |
For the config.yml section:
| In this tutorial, I show how to | |
| Finally, following the [CrateDB Prometheus Adapter setup instructions](https://github.com/crate/cratedb-prometheus-adapter), create the `config.yml` file, which defines the CrateDB endpoints the Adapter writes to. | |
| As done previously in the `prometheus.yml` file, set the host to `cratedb`, which is how the CrateDB service was declared in the `docker-compose.yml` file, instead of the default `localhost`. Set the remaining variables with their default values. |
| In this tutorial, I show how to | |
| Ensure that `docker-compose.yml`, `config.yml`, and `prometheus.yml` are in the same directory on the local machine. |
For the "Start the services" section:
| In this tutorial, I show how to | |
| Finally, navigate to the CrateDB-Prometheus directory in the terminal and start Docker Compose with the `docker-compose up` command |
| In this tutorial, I show how to | |
| With Docker Compose up and running, follow the [CrateDB Prometheus Adapter setup instructions](https://github.com/crate/cratedb-prometheus-adapter), navigate to the [CrateDB Admin UI](https://www.google.com/search?client=safari&rls=en&q=cratedb+admin+ui&ie=UTF-8&oe=UTF-8) at [http://localhost:4200](http://localhost:4200/) and create a `metrics` table in CrateDB, which will store the metrics gathered by Prometheus. |
| In this tutorial, I show how to | |
| Then navigate to [http://localhost:9090](http://localhost:9090/), where the Prometheus UI appears. There, go to **Status** and then **Targets** |
| In this tutorial, I show how to | |
| And confirm that Prometheus successfully monitors itself. |
| In this tutorial, I show how to | |
| Lastly, return to the CrateDB Admin UI and select the `metrics` table just created. | |
| The display shows that only after a few minutes of running, Prometheus has gathered over 300k data points. |
Final paragraph:
| In this tutorial, I show how to | |
| CrateDB's incredible query speed now enables analyzing and visualizing this data using tools like {ref}`grafana`, which works effortlessly with CrateDB. |
These changes eliminate first-person voice and convert passive constructions to active voice while maintaining the tutorial's instructional clarity and flow.
🧠 Learnings used
Learnt from: amotl
PR: crate/cratedb-guide#0
File: :0-0
Timestamp: 2025-09-15T17:42:25.449Z
Learning: When reviewing documentation, use active voice instead of first-person or passive voice, and emit corresponding suggestions.
| * Set up Docker Compose to run CrateDB, Prometheus, and the CrateDB Prometheus Adapter | ||
| * Run the applications with Docker Compose | ||
|
|
||
| *Note: this blog post uses CrateDB 4.7.0, Prometheus 2.33.3 and CrateDB Prometheus Adapter 0.4.0* |
There was a problem hiding this comment.
I think the whole content need some adjustments, like removing ^^ and adjusting the style from the I show you into non-personal phrases.
Another option would be to just link to these post instead of copying it. Not sure about this. I think one idea of putting this into post instead of adding it to the documentation was, that a post can age while a documentation should be up-to-date and such requires maintaining.
amotl
force-pushed
the
prometheus
branch
2 times, most recently
from
3691767 to
5c66096
Compare
About
Continue adding tutorials from the community forum.
Preview
References
/cc @hlcianfagna, @hammerhead