Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mongodb with 3 replicated nodes #9

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion MANIFEST
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
REPO monk-mongodb
DIRS mongodb mongodb-sharded
DIRS mongodb mongodb-replicated mongodb-sharded
3 changes: 3 additions & 0 deletions mongodb-replicated/MANIFEST
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
REPO monk-mongodbsharded
LOAD mongo.yaml stack.yaml
RESOURCES files/rs-init.sh
185 changes: 185 additions & 0 deletions mongodb-replicated/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
# Monk & MongoDB

This repository contains Monk.io template to deploy MongoDB cluster in system either locally or on cloud of your choice (AWS, GCP, Azure, Digital Ocean).

## Start

[Set up Monk](https://docs.monk.io/docs/monk-in-10/)

Start `monkd` and login.

```bash
monk login --email=<email> --password=<password>
```

## Clone Monk MongoDB repository

In order to load templates and change configuration simply use below commands:

```bash
git clone https://github.com/monk-io/monk-mongodb

# and change directory to the monk-elk template folder
cd monk-mongodb/mongodb-replicated

```

## Configuration

You can add/remove configuration of the template.

The current variables can be found in `mongodb/variables` section

```yaml
variables:
mongo-image-tag: "latest"
mongodb-init-username: "mongo"
mongodb-init-password: "password"
mongodb-init-database: "mongo"
```

## Template variables

| Variable | Description | Type | Default |
| ------------------------- | ------------------------------ | ------ | -------- |
| **mongo-image-tag** | MongoDB image version. | string | latest |
| **mongodb-init-username** | MongoDB Initial root username. | string | mongo |
| **mongodb-init-password** | MongoDB Initial root password. | string | password |
| **mongodb-init-database** | MongoDB Initial root database. | string | mongo |

## Local Deployment

| First clone the repository simply run below command after launching `monkd`: |
| :--------------------------------------------------------------------------: |

```bash
➜ monk load MANIFEST

✨ Loaded:
├─🔩 Runnables:
│ ├─🧩 mongodb-replicated/mongodb-common
│ ├─🧩 mongodb-replicated/mongodb-1
│ ├─🧩 mongodb-replicated/mongodb-2
│ └─🧩 mongodb-replicated/mongodb-3
├─🔗 Process groups:
│ └─🧩 mongodb-replicated/stack
└─⚙️ Entity instances:
└─🧩 mongodb-replicated/mongodb-common/metadata
✔ All templates loaded successfully

➜ monk list -l mongodb-replicated

✔ Got the list
Type Template Repository Version Tags
runnable mongodb-replicated/mongodb-1 local - -
runnable mongodb-replicated/mongodb-2 local latest database, nosql
runnable mongodb-replicated/mongodb-3 local latest database, nosql
runnable mongodb-replicated/mongodb-common local latest database, nosql
runnable mongodb-replicated/mongodb-express local latest -
group mongodb-replicated/stack local - -


➜ monk run mongodb-replicated/stack

✔ Started local/mongodb-replicated/stack

```

This will start the entire mongodb-replicated/stack.

## Cloud Deployment

To deploy the above system to your cloud provider, create a new Monk cluster and provision your instances.

```bash
➜ monk cluster new -n monkha
✔ Cluster created
Your cluster has been created successfully.

➜ monk cluster provider add -p aws -f <path/to/your-credentials-file>
✔ Provider added successfully

➜ monk cluster grow -p aws --name monkha --tag monkha -i t3.large --region eu-north-1 -m 3 -d 50 --disk-type SSD
✔ Start creation of new instance(s) on aws... DONE
✔ Creating node: monkha-1 DONE
✔ Creating node: monkha-2 DONE
✔ Initializing node: monkha-1 DONE
✔ Initializing node: monkha-2 DONE
✔ Creating node: monkha-3 DONE
✔ Initializing node: monkha-3 DONE
✔ Connecting: monkha-1 DONE
✔ Syncing peer: monkha-1 DONE
✔ Connecting: monkha-2 DONE
✔ Syncing peer: monkha-2 DONE
✔ Connecting: monkha-3 DONE
✔ Syncing peer: monkha-3 DONE
✔ Syncing nodes DONE
✔ Cluster grown successfully
```

Once cluster is ready execute the same command as for local and select your cluster (the option will appear automatically).

```bash
➜ monk load MANIFEST

✨ Loaded:
├─🔩 Runnables:
│ ├─🧩 mongodb-replicated/mongodb-common
│ ├─🧩 mongodb-replicated/mongodb-1
│ ├─🧩 mongodb-replicated/mongodb-2
│ └─🧩 mongodb-replicated/mongodb-3
├─🔗 Process groups:
│ └─🧩 mongodb-replicated/stack
└─⚙️ Entity instances:
└─🧩 mongodb-replicated/mongodb-common/metadata
✔ All templates loaded successfully

➜ monk list -l mongodb-replicated

✔ Got the list
Type Template Repository Version Tags
runnable mongodb-replicated/mongodb-1 local - -
runnable mongodb-replicated/mongodb-2 local latest database, nosql
runnable mongodb-replicated/mongodb-3 local latest database, nosql
runnable mongodb-replicated/mongodb-common local latest database, nosql
runnable mongodb-replicated/mongodb-express local latest -
group mongodb-replicated/stack local - -


➜ monk run mongodb-replicated/stack

✔ Started local/mongodb-replicated/stack


```

## Logs & Shell

```bash
# show MongoDB logs
➜ monk logs -l 1000 -f mongodb-replicated/mongodb-1

# show MongoDB logs
➜ monk logs -l 1000 -f mongodb-replicated/mongodb-2

# show MongoDB logs
➜ monk logs -l 1000 -f mongodb-replicated/mongodb-3

# access shell in the container running MongoDB
➜ monk shell mongodb-replicated/mongodb-1

# access shell in the container running MongoDB
➜ monk shell mongodb-replicated/mongodb-2

# access shell in the container running MongoDB
➜ monk shell mongodb-replicated/mongodb-3
```

## Stop, remove and clean up workloads and templates

```bash
➜ monk purge --ii --rv --rs --no-confirm --rv --rs local/mongodb-replicated/stack

✔ mongodb/mongodb purged
✔ mongodb/stack purged
```
31 changes: 31 additions & 0 deletions mongodb-replicated/files/rs-init.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/bin/bash

DELAY=25

mongo1="{{ v "mongo1-url" }}"
mongo2="{{ v "mongo2-url" }}"
mongo3="{{ v "mongo3-url" }}"
admin_user="{{ v "mongo-init-username" }}"
admin_pass="{{ v "mongo-init-password" }}"

mongosh <<EOF
var config = {
"_id": "dbrs",
"configsvr": true,
"members": [
{ "_id": 1, "host": "$mongo1" },
{ "_id": 2, "host": "$mongo2" },
{ "_id": 3, "host": "$mongo3" }
]
};

rs.initiate(config, { force: true });
rs.status();
EOF

echo "****** Waiting for ${DELAY} seconds for replicaset configuration to be applied ******"
sleep $DELAY

mongosh <<EOF
db.createUser({user: "$admin_user", pwd: "$admin_pass", roles:[{role: "userAdminAnyDatabase" , db:"admin"}]})
EOF
112 changes: 112 additions & 0 deletions mongodb-replicated/mongo.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
---
namespace: mongodb-replicated

mongodb-common:
defines: runnable
metadata:
name: Mongodb
version: latest
description: |
MongoDB is a popular open-source NoSQL database management system. It is designed to be scalable and flexible, making it suitable for handling large amounts of unstructured and semi-structured data.
MongoDB uses a document-based data model, where data is stored in flexible JSON-like documents that can have varying structures and fields.
This allows for greater flexibility and ease of data management, as data can be added, modified, or deleted without the need for predefined schema or migrations.
Additionally, MongoDB provides various features such as dynamic queries, indexing, replication, and sharding, which make it suitable for a wide range of applications and use cases.
tags: document-oriented database, indexing, json, nosql, clustering, querying, data analysis, sharding, data modeling
website: https://github.com/mongodb/mongo
source: https://www.mongodb.com/
publisher: monk.io
icon: https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRA8VbFgxH-i78AZmNqD93mVRkTRd30POqLeCmg9T05ug&s
private: true
services:
mongo:
container: mongodb
port: 27017
protocol: tcp
connections:
mongo-1:
runnable: mongodb-replicated/mongodb-1
service: mongo
mongo-2:
runnable: mongodb-replicated/mongodb-2
service: mongo
mongo-3:
runnable: mongodb-replicated/mongodb-3
service: mongo
containers:
mongodb:
image: mongo
image-tag: <- $mongo-image
entrypoint: /usr/bin/mongod --bind_ip_all --configsvr --replSet dbrs --port 27017 --dbpath /data/db
paths:
- <- `${monk-volume-path}/mongodb:/data/db`
variables:
mongo-image:
value: <- $mongo-image-tag default("latest")
type: string
mongo-init-username:
env: MONGO_INITDB_ROOT_USERNAME
value: <- $mongodb-init-username default("mongo")
type: string
mongo-init-password:
env: MONGO_INITDB_ROOT_PASSWORD
value: <- $mongodb-init-password default("password")
type: string

mongodb-1:
defines: runnable
inherits: mongodb-replicated/mongodb-common
readiness:
code: |
exec("mongodb", "/usr/bin/bash", "-c", "echo "rs.status().ok" | mongosh -u ${mongo-init-username} -p ${mongo-init-password} --quiet | grep 1") "1" contains?
period: 10
initialDelay: 5
containers:
mongodb:
hooks:
container-started: create-cluster
paths:
- <- `${monk-volume-path}/mongodb-1:/data/db`
depends:
wait-for:
runnables:
- mongodb-replicated/mongodb-2
- mongodb-replicated/mongodb-3
variables:
mongo1-url:
type: string
value: <- connection("mongo-1")
mongo2-url:
type: string
value: <- connection("mongo-2")
mongo3-url:
type: string
value: <- connection("mongo-3")
files:
init-script:
container: mongodb
mode: 645
path: /scripts/rs-init.sh
contents: <<< files/rs-init.sh
actions:
create-cluster:
description: Create cluster
code: exec("mongodb", "/bin/bash", "-c", `/scripts/rs-init.sh`)
create-status:
description: Cluster status
code: exec("mongodb", "/bin/bash", "-c", `echo "rs.status()" | mongosh -u ${mongo-init-username} -p ${mongo-init-password}`)

mongodb-2:
defines: runnable
inherits: mongodb-replicated/mongodb-common
containers:
mongodb:
paths:
- <- `${monk-volume-path}/mongodb-2:/data/db`

mongodb-3:
defines: runnable
inherits: mongodb-replicated/mongodb-common
containers:
mongodb:
paths:
- <- `${monk-volume-path}/mongodb-3:/data/db`
27 changes: 27 additions & 0 deletions mongodb-replicated/stack.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
namespace: mongodb-replicated

stack:
defines: process-group
metadata:
name: Mongodb
version: latest
description: |
MongoDB is a popular open-source NoSQL database management system. It is designed to be scalable and flexible, making it suitable for handling large amounts of unstructured and semi-structured data.
MongoDB uses a document-based data model, where data is stored in flexible JSON-like documents that can have varying structures and fields.
This allows for greater flexibility and ease of data management, as data can be added, modified, or deleted without the need for predefined schema or migrations.
Additionally, MongoDB provides various features such as dynamic queries, indexing, replication, and sharding, which make it suitable for a wide range of applications and use cases.
tags: document-oriented database, indexing, json, nosql, clustering, querying, data analysis, sharding, data modeling
website: https://github.com/mongodb/mongo
source: https://www.mongodb.com/
publisher: monk.io
icon: https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRA8VbFgxH-i78AZmNqD93mVRkTRd30POqLeCmg9T05ug&s
runnable-list:
- mongodb-replicated/mongodb-1
- mongodb-replicated/mongodb-2
- mongodb-replicated/mongodb-3
variables:
mongo-image-tag: latest
mongodb-init-username: mongo
mongodb-init-password: password
mongodb-init-database: mongo
2 changes: 1 addition & 1 deletion mongodb-sharded/MANIFEST
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
REPO monk-mongodbsharded
LOAD mongo.yaml stack.yaml
RESOURCES files/rs-init.sh
RESOURCES files/rs-init.sh files/sh-init.sh
Loading