Commit 655b69c
authored
[TRT RTX EP] Don't register set device function when we use existing stream (microsoft#26542)
### Description
- Don't register set device function when we use existing stream
- Fix bug nv_execution_provider.cc : set device only if user did not
provide existing stream
### Motivation and Context
In some use cases, we push a user generated CUDA context, create streams
using this context, and then provide these streams to TRT-RTX.
However, we noticed that after calling Run(), the custom context is
replaced by another CUDA context created by ORT. This means that TRT-RTX
is no longer using the original CUDA context.
After investigating further, we found that the new context is being
created in onnxruntime/core/framework/stream_execution_context.cc.
The solution we propose is to not register set device function if we
provide the stream.
Also there is a bug in
onnxruntime\core\providers\nv_tensorrt_rtx\nv_execution_provider.cc.
We should set the device only if the user has not provided any stream.
(coherent with the original comment)1 parent 935affb commit 655b69c
File tree
2 files changed
+5
-4
lines changed- onnxruntime/core/providers
- cuda
- nv_tensorrt_rtx
2 files changed
+5
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
250 | 250 | | |
251 | 251 | | |
252 | 252 | | |
253 | | - | |
| 253 | + | |
254 | 254 | | |
255 | 255 | | |
256 | 256 | | |
257 | 257 | | |
258 | 258 | | |
259 | 259 | | |
260 | 260 | | |
261 | | - | |
| 261 | + | |
| 262 | + | |
262 | 263 | | |
263 | 264 | | |
264 | 265 | | |
| |||
267 | 268 | | |
268 | 269 | | |
269 | 270 | | |
270 | | - | |
| 271 | + | |
271 | 272 | | |
272 | 273 | | |
273 | 274 | | |
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
766 | 766 | | |
767 | 767 | | |
768 | 768 | | |
769 | | - | |
| 769 | + | |
770 | 770 | | |
771 | 771 | | |
772 | 772 | | |
| |||
0 commit comments