Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 155 additions & 0 deletions docs-gb/cli/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# CLI

Seldon provides a CLI to allow easy management and testing of Model, Experiment, and Pipeline resources.

At present this needs to be built by hand from the operator folder.

```
make build-seldon # for linux/macOS amd64
make build-seldon-arm # for macOS ARM
```

Then place the `bin/seldon` executable in your path.

* [cli docs](./docs/seldon.md)


## Environment Variables and Services

The CLI talks to 3 backend services on default endpoints:

1. The Seldon Core 2 Scheduler: default 0.0.0.0:9004
2. The Seldon Core inference endpoint: default 0.0.0.0:9000
3. The Seldon Kafka broker: default: 0.0.0.0:9092

These defaults will be correct when Seldon Core 2 is installed locally as per the docs. For Kubernetes, you will need to change these by defining environment variables.

```{literalinclude} ../../../../operator/cmd/seldon/cli/flags.go
:language: golang
:start-after: // Defaults
:end-before: // Help statements
```

## Kubernetes Usage

### Inference Service

For a default install into the `seldon-mesh` namespace if you have exposed the inference `svc` as a loadbalancer you will find it at:

```
kubectl get svc seldon-mesh -n seldon-mesh -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
```

Use above IP at port 80:

```
export SELDON_INFER_HOST=<ip>:80
```

### Scheduler Service

For a default install into the `seldon-mesh` namespace if you have exposed the scheduler svc as a loadbalancer you will find it at:

```
kubectl get svc seldon-scheduler -n seldon-mesh -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
```

Use above IP at port 9004:

```
export SELDON_SCHEDULE_HOST=<ip>:9004
```

### Kafka Broker

The Kafka broker will depend on how you have installed Kafka into your Kubernetes cluster. Find the broker IP and use:

```
export SELDON_KAFKA_BROKER=<ip>:<port>
```

## Config file

You can create a config file to manage connections to running Seldon Core 2 installs. The settings will override any environment variable settings.

The definition is shown below:

```{literalinclude} ../../../../operator/pkg/cli/config.go
:language: golang
:start-after: // start config struct
:end-before: // end config struct
```

An example below shows an example where we connect via TLS to the Seldon scheduler using our scheduler client certificate:

```
{
"controlplane":{
"schedulerHost": "seldon-scheduler.svc:9044",
"tls"; true,
"keyPath": "/home/certs/seldon-scheduler-client/tls.key",
"crtPath": "/home/certs/seldon-scheduler-client/tls.crt",
"caPath": "/home/certs/seldon-scheduler-client/ca.crt"
}
}

```

To manage config files and activate them you can use the CLI command `seldon config` which has subcommands to list, add, remove, activate and decative configs.

For example:

```
$ seldon config list
config path active
------ ---- ------
kind-sasl /home/work/seldon/cli/config-sasl.json *

$ seldon config deactivate kind-sasl

$ seldon config list
config path active
------ ---- ------
kind-sasl /home/work/seldon/cli/config-sasl.json

$ seldon config add gcp-scv2 ~/seldon/cli/gcp.json

$ seldon config list
config path active
------ ---- ------
gcp-scv2 /home/work/seldon/cli/gcp.json
kind-sasl /home/work/seldon/cli/config-sasl.json

$ seldon config activate gcp-scv2

$ seldon config list
config path active
------ ---- ------
gcp-scv2 /home/work/seldon/cli/gcp.json *
kind-sasl /home/work/seldon/cli/config-sasl.json

$ seldon config list kind-sasl
{
"controlplane": {
"schedulerHost": "172.19.255.2:9004"
},
"kafka": {
"bootstrap": "172.19.255.3:9093",
"caPath": "/home/work/gcp/scv2/certs/seldon-cluster-ca-cert/ca.crt"
}
}
```

## TLS Certificates for Local Use

For running with Kubernetes TLS connections on the control and/or data plane, certificates will need to be downloaded locally. We provide an example script which will download certificates from a Kubernetes secret and store them in a folder. It can be found in `hack/download-k8s-certs.sh` and takes 2 or 3 arguments:

```
./download-k8s-certs.sh <namespace> <secret> [<folder>]
```

e.g.:

```
./download-k8s-certs.sh seldon-mesh seldon-scheduler-client
```
2 changes: 1 addition & 1 deletion docs-gb/examples/batch-examples-k8s.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ pip install mlserver
```

{% hint style="info" %}
**Note**: The Seldon CLI allows you to view information about underlying Seldon resources and make changes to them through the scheduler in non-Kubernetes environments. However, it cannot modify underlying manifests within a Kubernetes cluster. Therefore, using the Seldon CLI for control plane operations in a Kubernetes environment is not recommended. For more details, see [Seldon CLI](../cli/).
**Note**: The Seldon CLI allows you to view information about underlying Seldon resources and make changes to them through the scheduler in non-Kubernetes environments. However, it cannot modify underlying manifests within a Kubernetes cluster. Therefore, using the Seldon CLI for control plane operations in a Kubernetes environment is not recommended. For more details, see [Seldon CLI](../getting-started/cli.md).
{% endhint %}

### Seldon V2 Batch Examples
Expand Down
2 changes: 1 addition & 1 deletion docs-gb/examples/custom-servers.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# Custom Servers

{% hint style="info" %}
**Note**: The Seldon CLI allows you to view information about underlying Seldon resources and make changes to them through the scheduler in non-Kubernetes environments. However, it cannot modify underlying manifests within a Kubernetes cluster. Therefore, using the Seldon CLI for control plane operations in a Kubernetes environment is not recommended. For more details, see [Seldon CLI](../cli/).
**Note**: The Seldon CLI allows you to view information about underlying Seldon resources and make changes to them through the scheduler in non-Kubernetes environments. However, it cannot modify underlying manifests within a Kubernetes cluster. Therefore, using the Seldon CLI for control plane operations in a Kubernetes environment is not recommended. For more details, see [Seldon CLI](../getting-started/cli.md).
{% endhint %}

```python
Expand Down
2 changes: 1 addition & 1 deletion docs-gb/examples/k8s-clusterwide.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ MESH_IP_NS2
### Run Models in Different Namespaces

{% hint style="info" %}
**Note**: The Seldon CLI allows you to view information about underlying Seldon resources and make changes to them through the scheduler in non-Kubernetes environments. However, it cannot modify underlying manifests within a Kubernetes cluster. Therefore, using the Seldon CLI for control plane operations in a Kubernetes environment is not recommended. For more details, see [Seldon CLI](../cli/).
**Note**: The Seldon CLI allows you to view information about underlying Seldon resources and make changes to them through the scheduler in non-Kubernetes environments. However, it cannot modify underlying manifests within a Kubernetes cluster. Therefore, using the Seldon CLI for control plane operations in a Kubernetes environment is not recommended. For more details, see [Seldon CLI](../getting-started/cli.md).
{% endhint %}

```bash
Expand Down
Loading
Loading