You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `codegen.py` Python script. Build the MegaService Docker image via the command below:
Then run the command `docker images`, you will have the following Docker images:
177
+
178
+
-`opea/llm-textgen:latest`
179
+
-`opea/retriever:latest`
180
+
-`opea/dataprep:latest`
181
+
-`opea/codegen:latest`
182
+
-`opea/codegen-gradio-ui:latest` (Recommended)
183
+
-`opea/codegen-ui:latest` (Optional)
184
+
-`opea/codegen-react-ui:latest` (Optional)
185
+
186
+
113
187
### Start the Docker Containers for All Services
114
188
115
189
CodeGen support TGI service and vLLM service, you can choose start either one of them.
@@ -139,37 +213,69 @@ docker compose --profile codegen-xeon-vllm up -d
139
213
```bash
140
214
curl http://${host_ip}:8028/v1/chat/completions \
141
215
-X POST \
142
-
-d '{"model": "Qwen/Qwen2.5-Coder-7B-Instruct", "messages": [{"role": "user", "content": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}], "max_tokens":32}' \
143
-
-H 'Content-Type: application/json'
216
+
-H 'Content-Type: application/json' \
217
+
-d '{"model": "Qwen/Qwen2.5-Coder-7B-Instruct", "messages": [{"role": "user", "content": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}], "max_tokens":32}'
218
+
144
219
```
145
220
146
221
2. LLM Microservices
147
222
148
223
```bash
149
224
curl http://${host_ip}:9000/v1/chat/completions\
150
225
-X POST \
151
-
-d '{"query":"Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception.","max_tokens":256,"top_k":10,"top_p":0.95,"typical_p":0.95,"temperature":0.01,"repetition_penalty":1.03,"stream":true}' \
152
-
-H 'Content-Type: application/json'
226
+
-H 'Content-Type: application/json' \
227
+
-d '{"query":"Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception.","max_tokens":256,"top_k":10,"top_p":0.95,"typical_p":0.95,"temperature":0.01,"repetition_penalty":1.03,"stream":true}'
153
228
```
154
229
155
-
3. MegaService
230
+
3. Dataprep Microservice
231
+
232
+
Make sure to replace the file name placeholders with your correct file name
"messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."
160
-
}'
246
+
curl http://${host_ip}:7778/v1/codegen \
247
+
-H "Content-Type: application/json" \
248
+
-d '{"messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
161
249
```
162
250
163
-
If the user wants a CodeGen service with RAG and Agents based on dedicated documentation.
251
+
CodeGen service with RAG and Agents activated based on an index.
164
252
165
253
```bash
166
-
curl http://localhost:7778/v1/codegen \
254
+
curl http://${host_ip}:7778/v1/codegen \
167
255
-H "Content-Type: application/json" \
168
256
-d '{"agents_flag": "True", "index_name": "my_API_document", "messages": "Implement a high-level API for a TODO list application. The API takes as input an operation request and updates the TODO list in place. If the request is invalid, raise an exception."}'
169
257
```
170
258
171
259
172
-
## 🚀 Launch the UI
260
+
## 🚀 Launch the Gradio Based UI (Recommended)
261
+
To access the Gradio frontend URL, follow the steps in [this README](../../../../ui/gradio/README.md)
To access the frontend, open the following URL in your browser: `http://{host_ip}:5173`. By default, the UI runs on port 5173 internally. If you prefer to use a different host port to access the frontend, you can modify the port mapping in the `compose.yaml` file as shown below:
175
281
@@ -282,54 +388,3 @@ For example:
282
388
- Ask question and get answer
283
389
284
390

285
-
286
-
## 🚀 Download or Build Docker Images
287
-
288
-
Should the Docker image you seek not yet be available on Docker Hub, you can build the Docker image locally.
To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `codegen.py` Python script. Build MegaService Docker image via the command below:
0 commit comments