Skip to content

Update threading configuration for NVL and PTL#36367

Draft
sunxiaoxia2022 wants to merge 4 commits into
openvinotoolkit:masterfrom
sunxiaoxia2022:xiaoxia/nvl_support
Draft

Update threading configuration for NVL and PTL#36367
sunxiaoxia2022 wants to merge 4 commits into
openvinotoolkit:masterfrom
sunxiaoxia2022:xiaoxia/nvl_support

Conversation

@sunxiaoxia2022

Copy link
Copy Markdown
Contributor

Details:

  • Update threading configuration for NVL and PTL
  1. Improve hybrid CPU latency heuristics to enable more ALL/AUTO configurations on high LP E-core-share platforms.
  2. Add new model-profile-based AUTO rules using lp_ecore_share, memory tolerance, convolution pressure, and GEMM ratio.
  3. Refine latency stream sizing so model_prefer_threads is honored when building single-stream ALL core configurations.
  4. Extend CPUStreamsExecutor internal stream metadata from a single core type to multiple core types.
  5. Update task arena creation logic to bind streams by the actual set of eligible core types instead of forcing a single-type binding.
  6. Add unit coverage for mixed-core stream metadata and stream type selection behavior.

Tickets:

AI Assistance:

  • AI assistance used: yes
  • If yes, summarize how AI was used and what human validation was performed (build/tests/manual checks).

@github-actions github-actions Bot added category: inference OpenVINO Runtime library - Inference category: CPU OpenVINO CPU plugin labels Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: CPU OpenVINO CPU plugin category: inference OpenVINO Runtime library - Inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant