Align sources with upstream #122
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ git rev-parse cc-msft-prototypes 328e440 $ git checkout cc-msft-prototypes src/tools/genpolicy/ $ git checkout cc-msft-prototypes src/agent/samples/policy/
$ git rev-parse cc-msft-prototypes 328e440 $ git checkout cc-msft-prototypes src/utarfs $ git checkout cc-msft-prototypes src/tarfs $ git checkout cc-msft-prototypes src/overlay $ git checkout cc-msft-prototypes src/tardev-snapshotter $ git checkout cc-msft-prototypes Makefile
Add option to use an IGVM image in the UVM Signed-off-by: Dallas Delaney <[email protected]>
Add configuration file configuration-clh-snp.toml with SNP related settings enabled Signed-off-by: Dallas Delaney <[email protected]>
* Remove unused param from CLH SNP config * Remove extra package definition from cbl-mariner rootfs config
* Enable SNP=on with CH conf-guest enabled and add dummy host_data value * Add IGVM, HostData, Snp to config markdown doc * sanitize clh-snp.toml.in and clh.toml.in * Further changes required for SEV SNP enablement * Remove unnecessary debug output * Update outdated comment in config
Signed-off-by: Dallas Delaney <[email protected]>
* Set PCI segments in all cases * Clean-up
This makes it so that any container has access to /dev/sev-guest out of
the box with no privileges required.
Since /dev/sev-guest isn't available yet, I've validated this change
using /dev/cpu_dma_latency (original chmod 600) by:
1. Verifying that the device is present in the container.
2. Verifying that reading from the device from a container yields
the same result as from the VM context.
Signed-off-by: Aurélien Bombo <[email protected]>
Add image build macro to change partition format for kernel's "dm-mod.create" command, and allow for igvm + image usecase in kata shim Signed-off-by: Dallas Delaney <[email protected]>
This is a workaround for kata-containers#7993. Signed-off-by: Dan Mihai <[email protected]>
* rootfs: delete some of the mariner packages Delete some of the mariner packages from the Guest image, for faster TEE memory measurement. Signed-off-by: Dan Mihai <[email protected]>
The shell is useful for debugging. Signed-off-by: Dan Mihai <[email protected]>
Useful for debugging. Signed-off-by: Dan Mihai <[email protected]>
Useful for debugging. Signed-off-by: Dan Mihai <[email protected]>
Make sure the hash of an incoming Policy matches the value of the SNP Host Data field. The value of Host Data will be validated through Remote Attestation, outside of this patch. Signed-off-by: Dan Mihai <[email protected]>
There are 10 segments in the ACPI tables, and CLH works better when it uses all of them. Signed-off-by: Dan Mihai <[email protected]>
When a request cannot be evaluated to true, OPA can return an empty response. It doesn't respond with "response = false" unless a default value of false has been defined. Handle empry responses the same way as "response = false", thus allowing users to bypass those responses by using AllowRequestsFailingPolicy := true. Signed-off-by: Dan Mihai <[email protected]>
This is needed when enabling dm-verity. `udevd` reads kernel uevents that announce the creation of `/dev/dm-XXX` devices, and then creates devices with the actual names under `/dev/mapper/`. Signed-off-by: Wedson Almeida Filho <[email protected]>
This allows us to avoid repeating paths when they're the same. Signed-off-by: Wedson Almeida Filho <[email protected]>
This replicates Wedson's changes in 0935263 in a way that is aligned with the upstream implementation introduced in kata-containers#7200. NOTE: This will require compiling the runtime with DEFSHAREDFS_CLH_SNP_VIRTIOFS=none.
Upstream now uses the new DEFSTATICRESOURCEMGMT_TEE variable to set static resource management for TEEs so we align on that. It's true by default so we don't have to update our build script for this. NOTE: For non-tee CH, upstream now uses DEFSTATICRESOURCEMGMT_CLH (already in our codebase) instead of DEFSTATICRESOURCEMGMT. It's still false by default so we WILL have to update our build script for this one.
The layer string is now base64-encoded, so decode it before inspecting the fields. Signed-off-by: Wedson Almeida Filho <[email protected]>
Newer versions of depmod are failing without the -a option. They get confused with the kernel version and expect it to start with a slash: depmod: FATAL: modules: not absolute path. Signed-off-by: Wedson Almeida Filho <[email protected]>
This allows the agent to ensure the integrity of the device. Signed-off-by: Wedson Almeida Filho <[email protected]>
This allows snapshotters to skip the path to layers. They can, naturally, still specify the full path to other locations when needed. Signed-off-by: Wedson Almeida Filho <[email protected]>
This allows us to tell, from the error, what the agent was attempting to do with the devicemapper. Signed-off-by: Wedson Almeida Filho <[email protected]>
This is so that dependents are removed first, so that depencies don't have references anymore when they're unmounted. Signed-off-by: Wedson Almeida Filho <[email protected]>
danmihai1
reviewed
Dec 7, 2023
src/agent/samples/policy/set-policy-allowed/set-policy-allowed-data.json
Outdated
Show resolved
Hide resolved
|
General comment. To assess which of the patch files from 'https://github.com/microsoft/CBL-Mariner/tree/2.0/SPECS/kata-containers' (resp. remaining files in https://github.com/microsoft/CBL-Mariner/tree/mfrw/kata-containers-3.2.0/SPECS/kata-containers) are still missing here. None of our SPECS should live with those patch files, if possible. One known missing one is: runtime-reduce-uvm-high-mem-footprint.patch. To assess. |
|
@ms-mahuber Good call out! From CBL-Mariner 2.0:
Note the patches in Falak's branch are a subset of the above. |
sprt
commented
Dec 8, 2023
After discussion, to include runtime-reduce-uvm-high-mem-footprint.patch |
Although not really required in general, this is expected by the local driver, so we just do this as a minimal change to get the local driver to work. Signed-off-by: Wedson Almeida Filho <[email protected]>
|
Only going to include the memory patch as discussed offline. |
Bug: https://microsoft.visualstudio.com/OS/_workitems/edit/43668151 Rationale: This is a temporary solution for optimizing memory usage for the current mechanism of requesting resources through pod Limit annotations: - if no Limits are specified and hence WorkloadMemMB is 0, set a default value 'StaticWorkloadDefaultMem' to allocate a default amount of memory for use for containers in the sandbox in addition to the base memory - if Limits are specified, the base memory and the sum of Limits are allocated. The end user needs to be aware of the minimum memory requirements for their pods, otherwise the pod will be stuck in the ContainerCreating state Testing: Manual testing, creating pods with Limits and without limits, and with two containers where each container has a limit, tested with integration in a SPEC file where the config variables were set via environment variables via the make command Adapted by @mfrw from 3.1.0 to apply to 3.2.0 Signed-off-by: Muhammad Falak R Wani <[email protected]> Signed-off-by: Manuel Huber <[email protected]>
* genpolicy: allow empty env variables that may be forgotten to specify Allow empty AZURE_CLIENT_ID and AZURE_TENANT_ID Signed-off-by: Saul Paredes <[email protected]> * genpolivy: update sample Signed-off-by: Saul Paredes <[email protected]> * genpolicy: update sample Remove empty env variable from sample Signed-off-by: Saul Paredes <[email protected]> --------- Signed-off-by: Saul Paredes <[email protected]>
Previously the tool would use the layers_cache folder for all instances and hence delete the cache when it was done, interfereing with other instances. This change makes it so that each instance of the tool will have its own temp folder to use. Co-authored-by: Khalil Sayid <[email protected]> Signed-off-by: Seth Hollandsworth <[email protected]>
Attach block devices to segments 1-9, and use segment 0 for other types of devices. Signed-off-by: Dan Mihai <[email protected]>
all-allowed.rego now lives under src/kata-opa/ and set-policy-allowed.rego will be reintroduced by #123 in the same location.
danmihai1
approved these changes
Dec 19, 2023
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Jan 24, 2024
* Requires the new Rust virtiosfd 1.8.0. * Removes all patches. See microsoft/kata-containers#122 (comment) for a discussion. Since then, the memory patch was included in msft-main and patch 0004 is included in microsoft/kata-containers#154. * Cleans up runtime make flags: * The following were redundant as we were setting the default: * DEFSHAREDFS * DEFVIRTIOFSCACHESIZE * DEFSANDBOXCGROUPONLY * DEFSTATICRESOURCEMGMT_CLH
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Jan 24, 2024
* Requires the new Rust virtiosfd 1.8.0. * Removes all patches. See microsoft/kata-containers#122 (comment) for a discussion. Since then, the memory patch was included in msft-main and patch 0004 is included in microsoft/kata-containers#154. * Cleans up runtime make flags: * Set DEFVIRTIOFSDAEMON to the Rust binary path. * The following were redundant as we were setting the default: * DEFSHAREDFS (now DEFSHAREDFS_CLH_VIRTIOFS) * DEFVIRTIOFSCACHESIZE * DEFSANDBOXCGROUPONLY * DEFSTATICRESOURCEMGMT_CLH * The following were referring to macros that do not have a value: * QEMUPATH * MACHINETYPE * FEATURE_SELINUX does not exist in the Kata source code.
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Jan 24, 2024
* Removes all patches. See microsoft/kata-containers#122 (comment) for a discussion. Since then, the memory patch was included in msft-main and patch 0004 is included in microsoft/kata-containers#154. * Requires the new Rust virtiosfd 1.8.0. * Adds BuildRequires for devmapper code paths in msft-main. * Cleans up runtime make flags: * Set DEFVIRTIOFSDAEMON to the Rust binary path. * The following were redundant as we were setting the default: * DEFSHAREDFS (now DEFSHAREDFS_CLH_VIRTIOFS) * DEFVIRTIOFSCACHESIZE * DEFSANDBOXCGROUPONLY * DEFSTATICRESOURCEMGMT_CLH * The following were referring to macros that do NOT expand to a value: * QEMUPATH * MACHINETYPE * FEATURE_SELINUX does not exist in the Kata source code. * DEFENABLEANNOTATIONS should not be set to ".*". * Ensures sed doesn't break symlinks. For configuration.toml, sed would convert that file from a symlink to a regular, so it'd become out of sync with configuration-clh.toml. rootfs.sh isn't a symlink but added the flag as well for good measure.
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Jan 24, 2024
* Cleans up runtime make flags:
* Set DEFVIRTIOFSDAEMON to the Rust binary path.
* The following were redundant as we were setting the default:
* DEFSHAREDFS (now DEFSHAREDFS_CLH_VIRTIOFS)
* DEFVIRTIOFSCACHESIZE
* DEFSANDBOXCGROUPONLY
* DEFSTATICRESOURCEMGMT_CLH
* The following were referring to macros that do NOT expand to a value:
* QEMUPATH
* MACHINETYPE
* FEATURE_SELINUX does not exist in the Kata source code.
* DEFENABLEANNOTATIONS should not be set to ".*".
* Removes all patches. See
microsoft/kata-containers#122 (comment)
for a discussion. Since then, the memory patch was included in msft-main
and patch 0004 is included in microsoft/kata-containers#154.
* Adds BuildRequires for devmapper code paths in msft-main.
* Requires the new Rust virtiosfd 1.8.0.
* Ensures sed doesn't break symlinks. For configuration.toml, sed would
convert that file from a symlink to a regular, so it'd become out of sync
with configuration-clh.toml. rootfs.sh isn't a symlink but added the flag as
well for good measure.
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Jan 24, 2024
* Cleans up runtime make flags:
* Set DEFVIRTIOFSDAEMON to the Rust binary path.
* The following were redundant as we were setting the default:
* DEFSHAREDFS (now DEFSHAREDFS_CLH_VIRTIOFS)
* DEFVIRTIOFSCACHESIZE
* DEFSANDBOXCGROUPONLY
* DEFSTATICRESOURCEMGMT_CLH
* The following were referring to macros that do NOT expand to a value:
* QEMUPATH
* MACHINETYPE
* FEATURE_SELINUX does not exist in the Kata source code.
* DEFENABLEANNOTATIONS should not be set to ".*".
* Removes all patches. See
microsoft/kata-containers#122 (comment)
for a discussion. Since then, the memory patch was included in msft-main
and patch 0004 is included in microsoft/kata-containers#154.
* Adds BuildRequires for devmapper code paths in msft-main.
* Requires the new Rust virtiosfd 1.8.0.
* Ensures sed doesn't break symlinks. For configuration.toml, sed would
convert that file from a symlink to a regular file, so it'd become out of
sync with configuration-clh.toml. rootfs.sh isn't a symlink but added the flag
as well for good measure.
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Jan 25, 2024
* Cleans up runtime make flags:
* Set DEFVIRTIOFSDAEMON to the Rust binary path.
* The following were redundant as we were setting the default:
* DEFSHAREDFS (now DEFSHAREDFS_CLH_VIRTIOFS)
* DEFVIRTIOFSCACHESIZE
* DEFSANDBOXCGROUPONLY
* DEFSTATICRESOURCEMGMT_CLH
* The following were referring to macros that do NOT expand to a value:
* QEMUPATH
* MACHINETYPE
* FEATURE_SELINUX does not exist in the Kata source code.
* DEFENABLEANNOTATIONS should not be set to ".*".
* Removes all patches. See
microsoft/kata-containers#122 (comment)
for a discussion. Since then, the memory patch was included in msft-main
and patch 0004 is included in microsoft/kata-containers#154.
* Adds BuildRequires for devmapper code paths in msft-main.
* Requires the new Rust virtiosfd 1.8.0.
* Ensures sed doesn't break symlinks. For configuration.toml, sed would
convert that file from a symlink to a regular file, so it'd become out of
sync with configuration-clh.toml. rootfs.sh isn't a symlink but added the flag
as well for good measure.
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Jan 25, 2024
* Cleans up runtime make flags:
* Set DEFVIRTIOFSDAEMON to the Rust binary path.
* The following were redundant as we were setting the default:
* DEFSHAREDFS (now DEFSHAREDFS_CLH_VIRTIOFS)
* DEFVIRTIOFSCACHESIZE
* DEFSANDBOXCGROUPONLY
* DEFSTATICRESOURCEMGMT_CLH
* The following were referring to macros that do NOT expand to a value:
* QEMUPATH
* MACHINETYPE
* FEATURE_SELINUX does not exist in the Kata source code.
* DEFENABLEANNOTATIONS should not be set to ".*".
* Removes all patches. See
microsoft/kata-containers#122 (comment)
for a discussion. Since then, the memory patch was included in msft-main
and patch 0004 is included in microsoft/kata-containers#154.
* Adds BuildRequires for devmapper code paths in msft-main.
* Requires the new Rust virtiosfd 1.8.0.
* Ensures sed doesn't break symlinks. For configuration.toml, sed would
convert that file from a symlink to a regular file, so it'd become out of
sync with configuration-clh.toml. rootfs.sh isn't a symlink but added the flag
as well for good measure.
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Jan 31, 2024
* Cleans up runtime make flags:
* Set DEFVIRTIOFSDAEMON to the Rust binary path.
* The following were redundant as we were setting the default:
* DEFSHAREDFS (now DEFSHAREDFS_CLH_VIRTIOFS)
* DEFVIRTIOFSCACHESIZE
* DEFSANDBOXCGROUPONLY
* DEFSTATICRESOURCEMGMT_CLH
* The following were referring to macros that do NOT expand to a value:
* QEMUPATH
* MACHINETYPE
* FEATURE_SELINUX does not exist in the Kata source code.
* DEFENABLEANNOTATIONS should not be set to ".*".
* Removes all patches. See
microsoft/kata-containers#122 (comment)
for a discussion. Since then, the memory patch was included in msft-main
and patch 0004 is included in microsoft/kata-containers#154.
* Adds BuildRequires for devmapper code paths in msft-main.
* Requires the new Rust virtiosfd 1.8.0.
* Ensures sed doesn't break symlinks. For configuration.toml, sed would
convert that file from a symlink to a regular file, so it'd become out of
sync with configuration-clh.toml. rootfs.sh isn't a symlink but added the flag
as well for good measure.
sprt
added a commit
to microsoft/azurelinux
that referenced
this pull request
Feb 12, 2024
* Cleans up runtime make flags:
* Set DEFVIRTIOFSDAEMON to the Rust binary path.
* The following were redundant as we were setting the default:
* DEFSHAREDFS (now DEFSHAREDFS_CLH_VIRTIOFS)
* DEFVIRTIOFSCACHESIZE
* DEFSANDBOXCGROUPONLY
* DEFSTATICRESOURCEMGMT_CLH
* The following were referring to macros that do NOT expand to a value:
* QEMUPATH
* MACHINETYPE
* FEATURE_SELINUX does not exist in the Kata source code.
* DEFENABLEANNOTATIONS should not be set to ".*".
* Removes all patches. See
microsoft/kata-containers#122 (comment)
for a discussion. Since then, the memory patch was included in msft-main
and patch 0004 is included in microsoft/kata-containers#154.
* Adds BuildRequires for devmapper code paths in msft-main.
* Requires the new Rust virtiosfd 1.8.0.
* Ensures sed doesn't break symlinks. For configuration.toml, sed would
convert that file from a symlink to a regular file, so it'd become out of
sync with configuration-clh.toml. rootfs.sh isn't a symlink but added the flag
as well for good measure.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related PR: microsoft/azurelinux#6942
This cherry picks our
cc-msft-prototypeschanges on top of upstream main. Vanilla sources are out of scope for this PR.Conformance run on 8aec434: https://dev.azure.com/mariner-org/mariner/_build/results?buildId=470892