Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ jobs:
DB_TYPE: 'elasticsearch'
MAX_REQ_PER_MINUTE: 320
MAX_CONNECTIONS_PER_MINUTE: 320
DOCKER_COMPUTE_ENVIRONMENTS: '[{"socketPath":"/var/run/docker.sock","resources":[{"id":"disk","total":1000000000}],"storageExpiry":604800,"maxJobDuration":3600,"fees":{"8996":[{"prices":[{"id":"cpu","price":1}]}]},"free":{"maxJobDuration":60,"maxJobs":3,"resources":[{"id":"cpu","max":1},{"id":"ram","max":1000000000},{"id":"disk","max":1000000000}]}}]'
DOCKER_COMPUTE_ENVIRONMENTS: '[{"socketPath":"/var/run/docker.sock","resources":[{"id":"disk","total":10}],"storageExpiry":604800,"maxJobDuration":3600,"fees":{"8996":[{"prices":[{"id":"cpu","price":1}]}]},"free":{"maxJobDuration":60,"maxJobs":3,"resources":[{"id":"cpu","max":1},{"id":"ram","max":1},{"id":"disk","max":1}]}}]'
- name: Check Ocean Node is running
run: |
for i in $(seq 1 90); do
Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,14 @@ export PRIVATE_KEY="0x_your_private_key_here"

The `PRIVATE_KEY` is the only mandatory environmental variable, you must include the `0x` at the front of your private key. Additional configurations can be set as needed. For all available configurations, refer to the [Environment Variables](docs/env.md) documentation.

2.1. If the config is a JSON file, run:

```
export CONFIG_PATH='<absolute_path_to_json_file>'
```

Config file should be absolute path.

3. Quick start the Ocean Node with PM2

```bash
Expand Down
7 changes: 4 additions & 3 deletions docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -1242,6 +1242,7 @@ It can include information about the file object, document ID, service ID, trans
- **transferTxId**: Optional. A string representing the transaction ID for the transfer of the compute algorithm.
- **algocustomdata**: Optional. An object containing additional custom data related to the compute algorithm.
- **userdata**: Optional. An object containing additional user-defined data related to the compute algorithm.
- **envs**: Optional. Array of keys:values to be used as environment variables for algo.

```typescript
export interface ComputeAlgorithm {
Expand Down Expand Up @@ -1337,7 +1338,7 @@ starts a free compute job and returns jobId if succesfull
"algorithm": {
"meta": { "container": { "image": "ubuntu", "entrypoint": "/bin/bash'" } }
},
"consumerAddress": "0xC7EC1970B09224B317c52d92f37F5e1E4fF6B687",
"consumerAddress": "0x00",
"signature": "123",
"nonce": 1,
"environment": "0x7d187e4c751367be694497ead35e2937ece3c7f3b325dcb4f7571e5972d092bd-0xbeaf12703d708f39ef98c3d8939ce458553254176dbb69fe83d535883c4cee38",
Expand All @@ -1351,7 +1352,7 @@ starts a free compute job and returns jobId if succesfull
```json
[
{
"owner": "0xC7EC1970B09224B317c52d92f37F5e1E4fF6B687",
"owner": "0x00",
"jobId": "0x7d187e4c751367be694497ead35e2937ece3c7f3b325dcb4f7571e5972d092bd-a4ad237d-dfd8-404c-a5d6-b8fc3a1f66d3",
"dateCreated": "1742291065.119",
"dateFinished": null,
Expand Down Expand Up @@ -1395,7 +1396,7 @@ Required at least one of the following parameters:
```json
[
{
"owner": "0xC7EC1970B09224B317c52d92f37F5e1E4fF6B687",
"owner": "0x00",
"did": null,
"jobId": "a4ad237d-dfd8-404c-a5d6-b8fc3a1f66d3",
"dateCreated": "1742291065.119",
Expand Down
48 changes: 24 additions & 24 deletions docs/GPU.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Here is the full definition of DOCKER_COMPUTE_ENVIRONMENTS:
}
}
},
{ "id": "disk", "total": 1000000000 }
{ "id": "disk", "total": 1 }
],
"storageExpiry": 604800,
"maxJobDuration": 3600,
Expand All @@ -102,8 +102,8 @@ Here is the full definition of DOCKER_COMPUTE_ENVIRONMENTS:
"maxJobs": 3,
"resources": [
{ "id": "cpu", "max": 1 },
{ "id": "ram", "max": 1000000000 },
{ "id": "disk", "max": 1000000000 },
{ "id": "ram", "max": 1 },
{ "id": "disk", "max": 1 },
{ "id": "myGPU", "max": 1 }
]
}
Expand All @@ -122,7 +122,7 @@ root@gpu-1:/repos/ocean/ocean-node# curl http://localhost:8000/api/services/comp
{
"id": "0xd6b10b27aab01a72070a5164c07d0517755838b9cb9857e2d5649287ec3aaaa2-0x66073c81f833deaa2f8e2a508f69cf78f8a99b17ba1a64f369af921750f93914",
"runningJobs": 0,
"consumerAddress": "0x4fb80776C8eb4cAbe7730dcBCdb1fa6ecD3c460E",
"consumerAddress": "0x00",
"platform": { "architecture": "x86_64", "os": "Ubuntu 22.04.3 LTS" },
"fees": {
"1": [
Expand All @@ -141,9 +141,9 @@ root@gpu-1:/repos/ocean/ocean-node# curl http://localhost:8000/api/services/comp
{ "id": "cpu", "total": 8, "max": 8, "min": 1, "inUse": 0 },
{
"id": "ram",
"total": 24888963072,
"max": 24888963072,
"min": 1000000000,
"total": 23,
"max": 23,
"min": 1,
"inUse": 0
},
{
Expand All @@ -162,15 +162,15 @@ root@gpu-1:/repos/ocean/ocean-node# curl http://localhost:8000/api/services/comp
"min": 0,
"inUse": 0
},
{ "id": "disk", "total": 1000000000, "max": 1000000000, "min": 0, "inUse": 0 }
{ "id": "disk", "total": 1, "max": 1, "min": 0, "inUse": 0 }
],
"free": {
"maxJobDuration": 60,
"maxJobs": 3,
"resources": [
{ "id": "cpu", "max": 1, "inUse": 0 },
{ "id": "ram", "max": 1000000000, "inUse": 0 },
{ "id": "disk", "max": 1000000000, "inUse": 0 },
{ "id": "ram", "max": 1, "inUse": 0 },
{ "id": "disk", "max": 1, "inUse": 0 },
{ "id": "myGPU", "max": 1, "inUse": 0 }
]
},
Expand All @@ -194,7 +194,7 @@ Start a free job using:
"rawcode": "import tensorflow as tf\nsess = tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(log_device_placement=True))\nprint(\"Num GPUs Available: \", len(tf.config.list_physical_devices('GPU')))\ngpus = tf.config.list_physical_devices('GPU')\nfor gpu in gpus:\n\tprint('Name:', gpu.name, ' Type:', gpu.device_type)"
}
},
"consumerAddress": "0xC7EC1970B09224B317c52d92f37F5e1E4fF6B687",
"consumerAddress": "0x00",
"signature": "123",
"nonce": 1,
"environment": "0xd6b10b27aab01a72070a5164c07d0517755838b9cb9857e2d5649287ec3aaaa2-0x66073c81f833deaa2f8e2a508f69cf78f8a99b17ba1a64f369af921750f93914",
Expand Down Expand Up @@ -259,7 +259,7 @@ Then define DOCKER_COMPUTE_ENVIRONMENTS with
},
{
"id": "disk",
"total": 1000000000
"total": 1
}
],
"storageExpiry": 604800,
Expand Down Expand Up @@ -291,11 +291,11 @@ Then define DOCKER_COMPUTE_ENVIRONMENTS with
},
{
"id": "ram",
"max": 1000000000
"max": 1
},
{
"id": "disk",
"max": 1000000000
"max": 1
},
{
"id": "myGPU",
Expand All @@ -311,7 +311,7 @@ aka

```bash
export DOCKER_COMPUTE_ENVIRONMENTS="[{\"socketPath\":\"/var/run/docker.sock\",\"resources\":[{\"id\":\"myGPU\",\"description\":\"AMD Radeon RX 9070 XT\",\"type\":\"gpu\",\"total\":1,\"init\":{\"advanced\":{
\"IpcMode\":\"host\",\"CapAdd\":[\"CAP_SYS_PTRACE\"],\"Devices\":[\"/dev/dxg\",\"/dev/dri/card0\"],\"Binds\":[\"/usr/lib/wsl/lib/libdxcore.so:/usr/lib/libdxcore.so\",\"/opt/rocm/lib/libhsa-runtime64.so.1:/opt/rocm/lib/libhsa-runtime64.so.1\"],\"SecurityOpt\":{\"seccomp\":\"unconfined\"}}}},{\"id\":\"disk\",\"total\":1000000000}],\"storageExpiry\":604800,\"maxJobDuration\":3600,\"fees\":{\"1\":[{\"feeToken\":\"0x123\",\"prices\":[{\"id\":\"cpu\",\"price\":1},{\"id\":\"nyGPU\",\"price\":3}]}]},\"free\":{\"maxJobDuration\":60,\"maxJobs\":3,\"resources\":[{\"id\":\"cpu\",\"max\":1},{\"id\":\"ram\",\"max\":1000000000},{\"id\":\"disk\",\"max\":1000000000},{\"id\":\"myGPU\",\"max\":1}]}}]"
\"IpcMode\":\"host\",\"CapAdd\":[\"CAP_SYS_PTRACE\"],\"Devices\":[\"/dev/dxg\",\"/dev/dri/card0\"],\"Binds\":[\"/usr/lib/wsl/lib/libdxcore.so:/usr/lib/libdxcore.so\",\"/opt/rocm/lib/libhsa-runtime64.so.1:/opt/rocm/lib/libhsa-runtime64.so.1\"],\"SecurityOpt\":{\"seccomp\":\"unconfined\"}}}},{\"id\":\"disk\",\"total\":10}],\"storageExpiry\":604800,\"maxJobDuration\":3600,\"fees\":{\"1\":[{\"feeToken\":\"0x123\",\"prices\":[{\"id\":\"cpu\",\"price\":1},{\"id\":\"nyGPU\",\"price\":3}]}]},\"free\":{\"maxJobDuration\":60,\"maxJobs\":3,\"resources\":[{\"id\":\"cpu\",\"max\":1},{\"id\":\"ram\",\"max\":1},{\"id\":\"disk\",\"max\":1},{\"id\":\"myGPU\",\"max\":1}]}}]"
```

you should have it in your compute envs:
Expand All @@ -325,7 +325,7 @@ root@gpu-1:/repos/ocean/ocean-node# curl http://localhost:8000/api/services/comp
{
"id": "0xbb5773e734e1b188165dac88d9a3dc8ac28bc9f5624b45fa8bbd8fca043de7c1-0x2c2761f938cf186eeb81f71dee06ad7edb299493e39c316c390d0c0691e6585c",
"runningJobs": 0,
"consumerAddress": "0x4fb80776C8eb4cAbe7730dcBCdb1fa6ecD3c460E",
"consumerAddress": "0x00",
"platform": {
"architecture": "x86_64",
"os": "Ubuntu 24.04.2 LTS"
Expand Down Expand Up @@ -359,9 +359,9 @@ root@gpu-1:/repos/ocean/ocean-node# curl http://localhost:8000/api/services/comp
},
{
"id": "ram",
"total": 33617674240,
"max": 33617674240,
"min": 1000000000,
"total": 31,
"max": 31,
"min": 1,
"inUse": 0
},
{
Expand Down Expand Up @@ -389,8 +389,8 @@ root@gpu-1:/repos/ocean/ocean-node# curl http://localhost:8000/api/services/comp
},
{
"id": "disk",
"total": 1000000000,
"max": 1000000000,
"total": 10,
"max": 10,
"min": 0,
"inUse": 0
}
Expand All @@ -406,12 +406,12 @@ root@gpu-1:/repos/ocean/ocean-node# curl http://localhost:8000/api/services/comp
},
{
"id": "ram",
"max": 1000000000,
"max": 1,
"inUse": 0
},
{
"id": "disk",
"max": 1000000000,
"max": 1,
"inUse": 0
},
{
Expand Down Expand Up @@ -450,7 +450,7 @@ Start a free job with
"rawcode": "import tensorflow as tf\nsess = tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(log_device_placement=True))\nprint(\"Num GPUs Available: \", len(tf.config.list_physical_devices('GPU')))\ngpus = tf.config.list_physical_devices('GPU')\nfor gpu in gpus:\n\tprint('Name:', gpu.name, ' Type:', gpu.device_type)"
}
},
"consumerAddress": "0xC7EC1970B09224B317c52d92f37F5e1E4fF6B687",
"consumerAddress": "0x00",
"signature": "123",
"nonce": 1,
"environment": "0xbb5773e734e1b188165dac88d9a3dc8ac28bc9f5624b45fa8bbd8fca043de7c1-0x2c2761f938cf186eeb81f71dee06ad7edb299493e39c316c390d0c0691e6585c",
Expand Down
9 changes: 6 additions & 3 deletions docs/env.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Environmental variables are also tracked in `ENVIRONMENT_VARIABLES` within `src/
## Core

- `PRIVATE_KEY` (Required): The private key for the node, required for node operations. Example: `"0x1d751ded5a32226054cd2e71261039b65afb9ee1c746d055dd699b1150a5befc"`
- `CONFIG_PATH`: Absolute path to JSON config file
- `RPCS`: JSON object defining RPC endpoints for various networks. Example: `"{ \"11155420\":{ \"rpc\":\"https://sepolia.optimism.io\", \"fallbackRPCs\": [\"https://public.stackup.sh/api/v1/node/optimism-sepolia\"], \"chainId\": 11155420, \"network\": \"optimism-sepolia\", \"chunkSize\": 1000 }}"`
- `DB_URL`: URL for connecting to the database. Required for running a database with the node. Example: `"http://localhost:8108/?apiKey=xyz"`
- `IPFS_GATEWAY`: The gateway URL for IPFS, used for downloading files from IPFS. Example: `"https://ipfs.io/"`
Expand Down Expand Up @@ -123,14 +124,16 @@ The `DOCKER_COMPUTE_ENVIRONMENTS` environment variable is used to configure Dock
Example Configuration
The `DOCKER_COMPUTE_ENVIRONMENTS` environment variable should be a JSON array of objects, where each object represents a Docker compute environment configuration. Below is an example configuration:

`Disk` and `Ram` resources are always expressed in GB.

```json
[
{
"socketPath": "/var/run/docker.sock",
"resources": [
{
"id": "disk",
"total": 1000000000
"total": 10
}
],
"storageExpiry": 604800,
Expand Down Expand Up @@ -158,11 +161,11 @@ The `DOCKER_COMPUTE_ENVIRONMENTS` environment variable should be a JSON array of
},
{
"id": "ram",
"max": 1000000000
"max": 1
},
{
"id": "disk",
"max": 1000000000
"max": 1
}
]
}
Expand Down
17 changes: 17 additions & 0 deletions elasticsearch-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
services:
elasticsearch:
image: elasticsearch:8.5.1
ports:
- 9200:9200
- 9300:9300
volumes:
- esdata:/usr/share/elasticsearch/data
environment:
ES_JAVA_OPTS: "-Xms512m -Xmx512m"
MAX_MAP_COUNT: "64000"
discovery.type: "single-node"
ELASTIC_PASSWORD: "changeme"
xpack.security.enabled: "false"
xpack.security.http.ssl.enabled: "false"
volumes:
esdata:
Loading
Loading