Skip to content

Redis database backend #206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bshifter wants to merge 34 commits into
base: main
Choose a base branch
from
Open

Redis database backend #206

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
1dfbaad
add Serialize/Deserialize derives to the Subscription struct
bshifter Dec 5, 2024
eed2246
redis-db: add skeleton for redis database trait
bshifter Dec 5, 2024
cf21b46
redis-db: add implementation for subscription
bshifter Dec 5, 2024
c2b1321
redis-db: add implementation for bookmarks
bshifter Dec 5, 2024
3ac20ef
redis-db: add MachineStatusFilter
bshifter Dec 5, 2024
dd55337
redis-db: add implementation for heartbeats
bshifter Dec 5, 2024
d1eca2c
redis-db: add implementation for stats
bshifter Dec 5, 2024
67302af
redis-db: add settings/config for redis
bshifter Dec 5, 2024
b2cab33
WIP migrations
bshifter Dec 5, 2024
373aaef
redis-db: Database trait test + tests
bshifter Dec 5, 2024
8f29898
fu: remove unnecessary migrations
bshifter Jan 27, 2025
05636bc
fu: cleanup unused imports
bshifter Jan 27, 2025
ddd8e2e
fu: redisdomain fmt::Diplay generated with strum macro
bshifter Jan 27, 2025
0546c1a
fu: use redis scan_match instead of keys
bshifter Jan 27, 2025
56bf4cf
fu:get bookmark
bshifter Jan 27, 2025
37fcae0
fu: get_value_or_default is now reference based
bshifter Jan 27, 2025
bd87f95
fu: use get method instead of direct access in array
bshifter Jan 27, 2025
632a163
fu: replace unnecessary pipe to hset_multiple in store_bookmark.
bshifter Jan 27, 2025
287dc54
fu: remove unnecessary pipe from delete_bookmarks
bshifter Jan 27, 2025
eb0dfca
fu: redesign Heartbeatfilter
bshifter Jan 27, 2025
0cbfd09
fu: use get method instead of direct access on heartbeats vec
bshifter Jan 27, 2025
b3c5c8a
fu: get_heartbeats_by_field skips heartbeat without subscription
bshifter Jan 27, 2025
cfa20a6
fu: use get for vector instead of direct access in get_machines
bshifter Jan 27, 2025
f527c15
fu: set_heartbeat_inner takes the ownership of value
bshifter Jan 28, 2025
931e09a
fu: use hset_multiple instead of pipeline in set_heartbeat_inner
bshifter Jan 28, 2025
bfe503e
fu: remove of unnecessary ip evaluation in set heartbeats
bshifter Jan 28, 2025
f390da2
fu: remove pointless 2nd pass on delete_subscription
bshifter Jan 28, 2025
dccdbed
fu: use with_context for non static context strings
bshifter Jan 28, 2025
04ad5ce
fu: remove unwanted local caching of subscription data from get_heart…
bshifter Jan 29, 2025
3aeba11
fu: remove key fallback logic at get_subscription_by_identifier, iter…
bshifter Jan 29, 2025
3cb98b0
fu: moved redisdomain implementation next to redis db into the same file
bshifter Jan 29, 2025
d5d921c
fu: clippy fixes
bshifter Feb 1, 2025
e272864
fu: add database schema v1 for subscription to support future migrations
bshifter Feb 1, 2025
60dadd8
fu: add From trait implementaion for SubscriptionData and Subscriptio…
bshifter Feb 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/rust-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ jobs:
ports:
# Maps tcp port 5432 on service container to the host
- 5432:5432
redis:
image: redis
ports:
- 6379:6379

steps:
- uses: actions/checkout@v4
Expand All @@ -57,4 +61,5 @@ jobs:
export POSTGRES_USER="postgres"
export POSTGRES_PASSWORD="postgres"
export POSTGRES_DBNAME="test"
export REDIS_URL="redis://redis:6379"
cargo test --verbose
35 changes: 34 additions & 1 deletion Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion common/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,11 @@ chrono = { version = "0.4.26", default-features = false, features = ["clock"] }
encoding_rs = "0.8.32"
deadpool-postgres = "0.14.0"
deadpool-sqlite = "0.5.0"
deadpool-redis = "0.18.0"
openssl = "0.10.70"
postgres-openssl = "0.5.0"
strum = { version = "0.26.1", features = ["derive"] }
futures-util = "0.3.28"

[dev-dependencies]
tempfile = "3.16.0"
Expand Down Expand Up @@ -68,4 +70,4 @@ assets = [
{ source = "../openwec.conf.sample.toml", dest = "/usr/share/doc/openwec/", mode = "0644", doc = true },
{ source = "../README.md", dest = "/usr/share/doc/openwec/", mode = "0644", doc = true },
{ source = "../doc/*", dest = "/usr/share/doc/openwec/doc/", mode = "0644", doc = true },
]
]
9 changes: 9 additions & 0 deletions common/src/database/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ use crate::{
bookmark::BookmarkData,
database::postgres::PostgresDatabase,
database::sqlite::SQLiteDatabase,
database::redis::RedisDatabase,
heartbeat::{HeartbeatData, HeartbeatsCache},
settings::Settings,
subscription::{
Expand All @@ -21,6 +22,7 @@ use self::schema::{Migration, Version};
pub mod postgres;
pub mod schema;
pub mod sqlite;
pub mod redis;

pub type Db = Arc<dyn Database + Send + Sync>;

Expand All @@ -40,6 +42,13 @@ pub async fn db_from_settings(settings: &Settings) -> Result<Db> {
schema::postgres::register_migrations(&mut db);
Ok(Arc::new(db))
}
crate::settings::Database::Redis(redis) => {
let mut db = RedisDatabase::new(redis.connection_url())
.await
.context("Failed to initialize Redis client")?;
schema::redis::register_migrations(&mut db);
Ok(Arc::new(db))
}
}
}

Expand Down
Loading