vfio_assigned_device: dynamic IOMMU mapping for P2P DMA#3367
Open
jstarks wants to merge 9 commits intomicrosoft:mainfrom
Open
vfio_assigned_device: dynamic IOMMU mapping for P2P DMA#3367jstarks wants to merge 9 commits intomicrosoft:mainfrom
jstarks wants to merge 9 commits intomicrosoft:mainfrom
Conversation
|
This PR modifies files containing For more on why we check whole files, instead of just diffs, check out the Rustonomicon |
Contributor
There was a problem hiding this comment.
Pull request overview
Enables peer-to-peer DMA for VFIO-assigned devices by keeping VFIO Type1 IOMMU mappings synchronized with the evolving guest memory map (including dynamically-mapped device BARs) via a new DMA-mapping consumer interface in membacking.
Changes:
- Add
membacking::DmaTarget+DmaMapperClientto publish per-submapping map/unmap events (with optional host VA support viaVaMapper). - Update
vfio_assigned_deviceto register each VFIO container as a DMA-mapper consumer instead of snapshotting guest RAM mappings at container creation. - Make
vfio_sys::Container::map_dmaexplicitlyunsafeand take a typed pointer (*const u8) for the mapped host VA.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| vm/devices/user_driver/vfio_sys/src/lib.rs | Make DMA mapping API unsafe and pointer-typed to reflect memory safety constraints. |
| vm/devices/pci/vfio_assigned_device/src/resolver.rs | Switch resolver construction to accept a DmaMapperClient rather than GuestMemory. |
| vm/devices/pci/vfio_assigned_device/src/manager.rs | Register VFIO containers as DmaTarget consumers for dynamic IOMMU updates. |
| vm/devices/pci/vfio_assigned_device/src/lib.rs | Allow unsafe usage at crate level (needed for DMA mapping integration). |
| vm/devices/pci/vfio_assigned_device/Cargo.toml | Replace guestmem dependency with membacking. |
| openvmm/openvmm_core/src/worker/dispatch.rs | Wire GuestMemoryManager::dma_mapper_client() into VFIO resolver creation. |
| openvmm/membacking/src/region_manager.rs | Introduce DMA mapper registration + replay + live update plumbing in the region manager. |
| openvmm/membacking/src/memory_manager/mod.rs | Expose dma_mapper_client() from GuestMemoryManager. |
| openvmm/membacking/src/memory_manager/device_memory.rs | Clarify that dma_target=false affects sharing, not IOMMU DMA notifications. |
| openvmm/membacking/src/mapping_manager/va_mapper.rs | Add small public accessors/doc comments used by DMA mapping logic. |
| openvmm/membacking/src/lib.rs | Re-export DmaTarget/DmaMapperClient/DmaMapperHandle and Mappable for consumers. |
| Cargo.lock | Update dependency graph for the new membacking usage. |
VFIO device assignment needs peer-to-peer DMA between assigned devices — one device must be able to DMA into another device's BAR. The old VFIO container manager snapshot guest RAM at creation time via GuestMemory::full_mapping() and programmed a static set of IOMMU identity mappings, so device BARs (which are mapped dynamically as the guest configures them) were never visible to the IOMMU. Introduce a DmaTarget trait in the region manager that receives incremental sub-mapping events (map_dma / unmap_dma) as regions are enabled, disabled, or modified. VFIO containers register themselves as DMA mapper consumers via a new DmaMapperClient, and the region manager replays all existing active sub-mappings on registration, then sends live updates as the memory map evolves. This means device BAR mappings are now programmed into every VFIO container's IOMMU as they appear, enabling P2P DMA between assigned devices. The region manager maintains the VaMapper lifecycle for backends that need host VAs (VFIO type1's pin_user_pages), controlled by a needs_va flag at registration. This also sets up the abstraction for future iommufd support, which maps from fd+offset directly and won't need a VaMapper at all. On the vfio_sys side, Container::map_dma now takes *const u8 instead of a raw u64 and is marked unsafe, properly reflecting the memory safety requirements that were previously hidden in the API.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
VFIO device assignment needs peer-to-peer DMA between assigned devices — one device must be able to DMA into another device's BAR. The old VFIO container manager snapshot guest RAM at creation time via GuestMemory::full_mapping() and programmed a static set of IOMMU identity mappings, so device BARs (which are mapped dynamically as the guest configures them) were never visible to the IOMMU.
Introduce a DmaTarget trait in the region manager that receives incremental sub-mapping events (map_dma / unmap_dma) as regions are enabled, disabled, or modified. VFIO containers register themselves as DMA mapper consumers via a new DmaMapperClient, and the region manager replays all existing active sub-mappings on registration, then sends live updates as the memory map evolves. This means device BAR mappings are now programmed into every VFIO container's IOMMU as they appear, enabling P2P DMA between assigned devices.
The region manager maintains the VaMapper lifecycle for backends that need host VAs (VFIO type1's pin_user_pages), controlled by a needs_va flag at registration. This also sets up the abstraction for future iommufd support, which maps from fd+offset directly and won't need a VaMapper at all.