From 867b43ea6cc55eeba1bd897e34f569be268262a7 Mon Sep 17 00:00:00 2001 From: Angel Luu Date: Wed, 30 Aug 2023 16:48:02 -0600 Subject: [PATCH 1/3] Add dev quick start guide Signed-off-by: Angel Luu --- docs/quickstart.md | 115 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 docs/quickstart.md diff --git a/docs/quickstart.md b/docs/quickstart.md new file mode 100644 index 0000000..63699fe --- /dev/null +++ b/docs/quickstart.md @@ -0,0 +1,115 @@ +--- +layout: default +title: Dev Quick Start Guide +nav_order: 7 +--- + +# Dev Quick Start Guide +{: .no_toc } + +This document describes how to quickly get a runtime server built with `sample_lib` library, train a model with gRPC and with that trained model, send an inference call to the server with either HTTP or gRPC call. + +## Pre-reqs +To set up development environment and tooling, the [Development Guide]({{ site.baseurl }}{% link docs/developing.md %}) page is a good place to start. + +## Build and start a runtime server with sample_lib + +Run the `start_runtime_with_sample_lib` python script: + +```shell +python3 -m examples.sample_lib.start_runtime_with_sample_lib +``` + +This will setup a config with both `grpc` and `http` servers enabled for inference service, as well as training service. The library used with some sample modules is `sample_lib`. The script then starts the caikit runtime server. While the server is running, you can see the generated protobufs in a directory called `protos`. + +We generate 3 services total: +- A train service. The proto for this service is `protos/samplelibtrainingservice.proto` +- An inference service. The proto for this service is `protos/samplelibservice.proto` +- A training management service. The proto for this service is `protos/trainingmanagement.proto` + +You can now proceed to leave the server running and open a new terminal to proceed with next steps to train a model, check its training status and send an inference request to your model. + +(To kill the server, press Ctrl + C. This will remove the `protos` directory to clean up.) + +## To train a model + +As of caikit v`0.18.0`, we do not yet support training model via REST. +In order to train a model via gRPC, we will use `grpcurl` and point the import-path to `protos` dir, then call one of the Train rpc's available in the `SampleLibTrainingService` (see `protos/samplelibtrainingservice.proto` file generated above for all Train rpcs): + +```shell +grpcurl -plaintext -import-path protos/ -proto samplelibtrainingservice.proto -d '{"model_name": "my_model", "training_data": {"file": {"filename": "protos/sample.json"}}}' localhost:8085 caikit.runtime.SampleLib.SampleLibTrainingService/SampleTaskSampleModuleTrain +``` + +You should receive a response similar to the below: + +```shell +{ + "trainingId": "wTHxlsu:5bdb5949-4efa-4512-bbac-709cbf37c00e", + "modelName": "my_model" +} +``` + +Copy the `trainingId` to use in next step. + +## To check on training status of a trainingId + +With a `trainingId`, you can get a training status via gRPC. Replace the command below with your `trainingId`. + +```shell +grpcurl -plaintext -import-path protos/ -proto trainingmanagement.proto -d '{"training_id": " Date: Wed, 30 Aug 2023 16:52:45 -0600 Subject: [PATCH 2/3] Remove repeated 'proceed' Signed-off-by: Angel Luu --- docs/quickstart.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/quickstart.md b/docs/quickstart.md index 63699fe..f11cfa1 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -27,7 +27,7 @@ We generate 3 services total: - An inference service. The proto for this service is `protos/samplelibservice.proto` - A training management service. The proto for this service is `protos/trainingmanagement.proto` -You can now proceed to leave the server running and open a new terminal to proceed with next steps to train a model, check its training status and send an inference request to your model. +You can now leave the server running and open a new terminal to proceed with next steps to train a model, check its training status and send an inference request to your model. (To kill the server, press Ctrl + C. This will remove the `protos` directory to clean up.) From e5cc7522900d9e23cf855e5f98ba2542286a4e77 Mon Sep 17 00:00:00 2001 From: Angel Luu Date: Thu, 31 Aug 2023 11:46:03 -0600 Subject: [PATCH 3/3] Update docs and glossary Signed-off-by: Angel Luu --- docs/glossary.md | 3 ++ docs/quickstart.md | 102 +-------------------------------------------- 2 files changed, 5 insertions(+), 100 deletions(-) diff --git a/docs/glossary.md b/docs/glossary.md index baf03c5..61955b1 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -73,6 +73,9 @@ self-supervised learning supervised learning : A machine learning training method in which a model is trained on a labeled dataset to make predictions on new data. +task +: A machine learning task that includes one or multiple implementations to perform a prediction or inference. For example: a `sample` task can have a `sample implementation` and a `list implementation`. All the implementations within a task will have a common protobuf definition for their request and response, ex: `SampleTaskRequest` and `SampleOutputType`. We generate inference service rcps per task, not per implementation. + token : A discrete unit of meaning or analysis in a text, such as a word or subword (part of a word). diff --git a/docs/quickstart.md b/docs/quickstart.md index f11cfa1..c9d8530 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -12,104 +12,6 @@ This document describes how to quickly get a runtime server built with `sample_l ## Pre-reqs To set up development environment and tooling, the [Development Guide]({{ site.baseurl }}{% link docs/developing.md %}) page is a good place to start. -## Build and start a runtime server with sample_lib +## Start Guide -Run the `start_runtime_with_sample_lib` python script: - -```shell -python3 -m examples.sample_lib.start_runtime_with_sample_lib -``` - -This will setup a config with both `grpc` and `http` servers enabled for inference service, as well as training service. The library used with some sample modules is `sample_lib`. The script then starts the caikit runtime server. While the server is running, you can see the generated protobufs in a directory called `protos`. - -We generate 3 services total: -- A train service. The proto for this service is `protos/samplelibtrainingservice.proto` -- An inference service. The proto for this service is `protos/samplelibservice.proto` -- A training management service. The proto for this service is `protos/trainingmanagement.proto` - -You can now leave the server running and open a new terminal to proceed with next steps to train a model, check its training status and send an inference request to your model. - -(To kill the server, press Ctrl + C. This will remove the `protos` directory to clean up.) - -## To train a model - -As of caikit v`0.18.0`, we do not yet support training model via REST. -In order to train a model via gRPC, we will use `grpcurl` and point the import-path to `protos` dir, then call one of the Train rpc's available in the `SampleLibTrainingService` (see `protos/samplelibtrainingservice.proto` file generated above for all Train rpcs): - -```shell -grpcurl -plaintext -import-path protos/ -proto samplelibtrainingservice.proto -d '{"model_name": "my_model", "training_data": {"file": {"filename": "protos/sample.json"}}}' localhost:8085 caikit.runtime.SampleLib.SampleLibTrainingService/SampleTaskSampleModuleTrain -``` - -You should receive a response similar to the below: - -```shell -{ - "trainingId": "wTHxlsu:5bdb5949-4efa-4512-bbac-709cbf37c00e", - "modelName": "my_model" -} -``` - -Copy the `trainingId` to use in next step. - -## To check on training status of a trainingId - -With a `trainingId`, you can get a training status via gRPC. Replace the command below with your `trainingId`. - -```shell -grpcurl -plaintext -import-path protos/ -proto trainingmanagement.proto -d '{"training_id": "