Skip to content

Commit 60e412e

Browse files
authored
Update MII Example (deepspeedai#798)
1 parent 0e1bb1f commit 60e412e

File tree

3 files changed

+5
-1
lines changed

3 files changed

+5
-1
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ This folder contains end-to-end applications that use DeepSpeed to train and use
88
There are several training and finetuning examples so please see the individual folders for specific instructions.
99

1010
## 3. Inference
11-
The DeepSpeed Huggingface inference [README](./inference/huggingface/README.md) explains how to get started with running DeepSpeed Huggingface inference examples.
11+
- The DeepSpeed-MII inference [README](./inference/mii/README.md) explains how to get started with running model inference with [DeepSpeed-MII](https://github.com/Microsoft/DeepSpeed-MII) and [DeepSpeed-FastGen](https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen).
12+
- The DeepSpeed Huggingface inference [README](./inference/huggingface/README.md) explains how to get started with running DeepSpeed Huggingface inference examples.
1213

1314
## 4. Compression
1415
Model compression examples.

inference/mii/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
11
# DeepSpeed MII Examples
22

3+
Install the requirements by running `pip install -r requirements.txt`.
4+
35
Once [DeepSpeed-MII](https://github.com/microsoft/deepspeed-mii) is installed you have two options for deployment: an interactive non-persistent pipeline or a persistent serving deployment. For details on these files please refer to the [Getting Started guide for MII](https://github.com/microsoft/deepspeed-mii#getting-started-with-mii).

inference/mii/requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
mii>=0.1.0

0 commit comments

Comments
 (0)