Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate CAF Custom IAM Roles #223

Open
6 of 9 tasks
AErmie opened this issue Jan 9, 2025 · 14 comments
Open
6 of 9 tasks

Evaluate CAF Custom IAM Roles #223

AErmie opened this issue Jan 9, 2025 · 14 comments
Assignees

Comments

@AErmie
Copy link

AErmie commented Jan 9, 2025

Out-of-the-box, the Cloud Adoption Framework (CAF), has the following custom roles defined:

  • Job Function Roles:
    • Network-Management
    • Network-Subnet-Contributor
    • Security-Operations
  • Privileged Administrator Roles:
    • Subscription-Owner
    • Application-Owners

These (or a customization of these roles) will be applied to various Security Groups, which end-users will be added to. In the future, this will be the supported method for granting and controlling access in Azure.

We need to evaluate these roles/permissions, to assess if any further customizations are required, and what the end-users can/cannot do. We can use the Azure built-in roles as a reference point.

Scenarios to Test

  • Deploy an application (inclusive of Private Endpoints)
  • Create Service Principals
  • Create Managed Identities (System Assigned vs User Assigned)
  • Modify RBAC permissions
  • Changing networking
    • Add/Remove subnets
    • Create VNet outside of Landing Zone Vending
      • Peering VNets outside of vWAN

Preventative Scenarios

We don't want end users to...

  • Rename Management Groups and/or Subscriptions
  • Delete Management Groups and/or Subscriptions
  • Remove/modify the VNet peering to the hub
  • Changing the DNS server configuration
  • Create any Express Route circuits, VPN sites, Route tables, VPN/NAT/Local Gateways
  • Peer non-Vending VNets with the Hub, or with their Vending provided VNet
  • Create a new VNet, but be still be able to modify an existing one (ie. add subnets)
  • Modify the existing Diagnostics collection
  • Modify or delete the existing Network Watcher

Potential Customizations

In addition to what these custom roles provide out-of-the-box, we might consider the following customizations.

Identity

  • AllowActions:
    • Microsoft.ManagedIdentity/*

Network

  • NotActions:
    • Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete
    • Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write
    • Microsoft.Network/virtualNetworks.properties.dhcpOptions.dnsServers/write (unconfirmed path reference)
    • Microsoft.Insights/DiagnosticSettings/setbypolicy/write
    • Microsoft.Insights/DiagnosticSettings/setbypolicy/delete
    • Microsoft.Network/natGateways/join/action
    • Microsoft.Network/connections/write
    • Microsoft.Network/localnetworkgateways/write
    • Microsoft.Network/virtualNetworkGateways/write
@AErmie AErmie self-assigned this Jan 9, 2025
@AErmie AErmie changed the title Evaluate CAF Custom Roles Evaluate CAF Custom IAM Roles Jan 9, 2025
@AErmie
Copy link
Author

AErmie commented Jan 9, 2025

To evaluate these custom roles, we need an "example" user account (with an IDIR), to simulate a regular user in Azure, so that we can test/evaluate the custom permissions/roles that are being created (and linked with the security groups).

Catherine is following up on this need with the Service Desk.

Update

MySC ticket submitted: REQ0631621

Update 2

New testing account created.

Verified the account can log into Azure

@AErmie
Copy link
Author

AErmie commented Jan 13, 2025

Out-of-the-Box Permissions

Subscription Owner

"notActions": [
  "Microsoft.Authorization/*/write",
  "Microsoft.Network/vpnGateways/*",
  "Microsoft.Network/expressRouteCircuits/*",
  "Microsoft.Network/routeTables/write",
  "Microsoft.Network/vpnSites/*"
],

Application Owner

"notActions": [
  "Microsoft.Authorization/*/write",
  "Microsoft.Network/publicIPAddresses/write",
  "Microsoft.Network/virtualNetworks/write",
  "Microsoft.KeyVault/locations/deletedVaults/purge/action"
],

@AErmie
Copy link
Author

AErmie commented Jan 13, 2025

Subscription-Owner Notes

  • When attempted to assign the [BCGOV-MANAGED-LZ-FORGE] Subscription-Owner permission to Breboin, Thibault (TBREBTRN) CITZ:EX, encountered the following error:

Failed to add Breboin, Thibault (TBREBTRN) CITZ:EX as [BCGOV-MANAGED-LZ-FORGE] Subscription-Owner for abc123 : Action pattern Microsoft.Authorization/roleAssignments/write is not referenced in role c7358265-e8fc-5b81-a9de-676f2d5bdbde..

image.png

Update

The c7358265-e8fc-5b81-a9de-676f2d5bdbde is actually the custom RBAC Role "[BCGOV-MANAGED-LZ-FORGE] Subscription-Owner".

image.png

And, because we're trying to assign the "Allow user to assign all roles except privileged administrator roles Owner, UAA, RBAC (Recommended)" condition, which allows the User to assign roles; but the custom role has the notActions: "Microsoft.Authorization/*/write" it prevents us from actually assigning the role, because our conditions are conflicting with the raw permissions.

I cloned that Role, and removed that one notAction and it worked.

What I don't know/see, is how to assign a Role without any conditions (since all conditions actually allow the User to assign Roles). So all conditions conflict with the custom Role notActions.

So, both CAF custom roles have that same notAction, and will therefore prevent us from actually assigning it! LOL

I kind of don't see how any of the custom CAF roles will work then, since they all will prevent the IAM conditions.

@AErmie
Copy link
Author

AErmie commented Jan 14, 2025

Created the following GitHub Issue against the CAF: Bug Report: Custom CAF Roles Conflict with Entra ID Role Assignment Conditions.

@AErmie
Copy link
Author

AErmie commented Jan 14, 2025

Project Set Resources (Out-of-the-Box)

When we create a Project Set, we only create a VNet, peer that with the vWAN Hub, and config it's DNS to route through the firewall.

What We Don't Want

We don't want end users to...

  • Rename Management Groups and/or Subscriptions
    • I don't think this is possible, as it's a /write action, which would prevent any deployments within the MG/Sub
  • Delete Management Groups and/or Subscriptions
  • Remove/modify the VNet peering to the hub
  • Changing the DNS server configuration
  • Create any Express Route circuits, VPN sites, Route tables, VPN/NAT Gateways
  • Peer non-Vending VNets with the Hub, or with their Vending provided VNet
    • Avoid overlapping address spaces, routing issues, etc.
  • Modify the existing Diagnostics collection
    • They can create their own in addition to ours though
  • Modify or delete the existing Network Watcher
"actions": [
    "Microsoft.Network/virtualNetworks/subnets/*"
],
"notActions": [
    "Microsoft.Management/managementGroups/delete",
    "Microsoft.Management/managementGroups/write",
    "Microsoft.Management/managementGroups/subscriptions/delete",
    "Microsoft.Management/managementGroups/subscriptions/write",
    "Microsoft.Network/virtualNetworks/write", 
    "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete",
    "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write",
    "Microsoft.Network/virtualNetworks.properties.dhcpOptions.dnsServers/write",
    "Microsoft.Network/expressRouteCircuits/*", 
    "Microsoft.Network/vpnSites/*",
    "Microsoft.Network/vpnGateways/*", 
    "Microsoft.Network/virtualNetworkGateways/write",
    "Microsoft.Network/connections/write", // Creates or updates an existing VirtualNetworkGatewayConnection
    "Microsoft.Network/natGateways/*",
    "Microsoft.Network/localnetworkgateways/write", // Creates or updates an existing LocalNetworkGateway
    "Microsoft.Network/routeTables/write",
    "Microsoft.Insights/DiagnosticSettings/setbypolicy/write",
    "Microsoft.Insights/DiagnosticSettings/setbypolicy/delete",
    "Microsoft.Network/networkWatchers/NetworkWatcher_canadacentral/delete",
    "Microsoft.Network/networkWatchers/NetworkWatcher_canadacentral/write",
]

@AErmie
Copy link
Author

AErmie commented Jan 15, 2025

Scenario: Rename Management Groups and/or Subscriptions

Created a custom role with the following permissions:

"notActions": [
    "Microsoft.Management/managementGroups/delete",
    "Microsoft.Management/managementGroups/write",
    "Microsoft.Management/managementGroups/subscriptions/delete",
    "Microsoft.Management/managementGroups/subscriptions/write",
]

Attempted to rename a Management Group, and received the following error:

Group update failed. An error occurred. 
{"code":"BadRequest","message":"Permission to write on resources of type 'Microsoft.Management/managementGroups' is required on the management group or its ancestors.","details":"Management Group ID: '/providers/Microsoft.Management/managementGroups/abc123'"}

image.png

Unfortunately, was still able to rename the Subscription.

Update

Found the following permission that's referenced when re-naming a Subscription: Microsoft.Subscription/rename/action

Added this to the notActions list, and tested. Confirmed that re-naming a Subscription is blocked, however, the error message presented does not clearly inform the user that it's due to not having permissions.

Changing subscription name
Something went wrong when changing subscription name. Please try again later.

image.png

Permissions

The final set of custom permissions that work to cover this scenario is as follows:

"notActions": [
    "Microsoft.Management/managementGroups/delete",
    "Microsoft.Management/managementGroups/write",
    "Microsoft.Management/managementGroups/subscriptions/delete",
    "Microsoft.Subscription/cancel/action",
    "Microsoft.Subscription/rename/action"
]

@AErmie
Copy link
Author

AErmie commented Jan 15, 2025

Scenario: Create any Express Route circuits, VPN sites, Route tables, VPN/NAT/Local Gateways

Created a custom role with the following permissions:

"notActions": [
    "Microsoft.Network/expressRouteCircuits/*",
    "Microsoft.Network/vpnSites/*",
    "Microsoft.Network/vpnGateways/*",
    "Microsoft.Network/virtualNetworkGateways/*",
    "Microsoft.Network/natGateways/*",
    "Microsoft.Network/localnetworkgateways/*",
    "Microsoft.Network/connections/*",
    "Microsoft.Network/routeTables/write"
]

Express Route

Attempted to create an Express Route, and received the following error:

{"code":"DeploymentFailed","target":"/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/permissions-testing/providers/Microsoft.Resources/deployments/Microsoft.ExpressRoute-20250115135718","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"AuthorizationFailed","message":"The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have authorization to perform action 'Microsoft.Network/expressRouteCircuits/write' over scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourcegroups/permissions-testing/providers/Microsoft.Network/expressRouteCircuits/test' or the scope is invalid. If access was recently granted, please refresh your credentials."}]}

image.png

NAT Gateway

Attempted to create a NAT Gateway, and received the following error:

The template deployment failed with error: 'Authorization failed for template resource 'test' of type 'Microsoft.Network/natGateways'. The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have permission to perform action 'Microsoft.Network/natGateways/write' at scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/permissions-testing/providers/Microsoft.Network/natGateways/test'.'.

image.png

Local Gateway

Attempted to create a Local Gateway, and received the following error:

InvalidTemplateDeploymentThe template deployment failed with error: 'Authorization failed for template resource 'test' of type 'Microsoft.Network/localNetworkGateways'. The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have permission to perform action 'Microsoft.Network/localNetworkGateways/write' at scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/permissions-testing/providers/Microsoft.Network/localNetworkGateways/test'.'.

image.png

Route Table

Attempted to create a Route Table, and received the following error:

Deployment validation failed.
Additional details from the underlying API that might be helpful: The template deployment failed with error: 'Authorization failed for template resource 'test' of type 'Microsoft.Network/routeTables'. The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have permission to perform action 'Microsoft.Network/routeTables/write' at scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/permissions-testing/providers/Microsoft.Network/routeTables/test'.'.

image.png

@AErmie
Copy link
Author

AErmie commented Jan 15, 2025

Scenario: VNet Peering

Created a custom role with the following permissions:

"notActions": [
    "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete",
    "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write"
]

Create New VNet Peering

Attempted to add a new VNet Peering, and received the following error:

Failed to add virtual network peering 'test' to 'abc123-dev-vwan-spoke'. Error: The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have authorization to perform action 'Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write' over scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/abc123-dev-networking/providers/Microsoft.Network/virtualNetworks/abc123-dev-vwan-spoke/virtualNetworkPeerings/test' or the scope is invalid. If access was recently granted, please refresh your credentials.

image.png

Modify Existing vHub VNet Peering

Attempted to modify the peering settings of the vHub Peering, and the Save button is greyed out/disabled.

image.png

Delete Existing VNet Peering

Attempted to delete the existing vHub VNet Peering, and received the following error:

Failed to delete virtual network peering 'RemoteVnetToHubPeering_b927af47-81a2-4779-9ae1-3b829a89e7e6'. Error: The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have authorization to perform action 'Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete' over scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/abc123-dev-networking/providers/Microsoft.Network/virtualNetworks/abc123-dev-vwan-spoke/virtualNetworkPeerings/RemoteVnetToHubPeering_b927af47-81a2-4779-9ae1-3b829a89e7e6' or the scope is invalid. If access was recently granted, please refresh your credentials.

image.png

Peer Non-Vending VNet with the vHub

Attempted to peer a non-Vending VNet with the vHub, and received the following error:

Failed to add virtual network peering
Failed to add virtual network peering 'test' to 'test'. Error: The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have authorization to perform action 'Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write' over scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/permissions-testing/providers/Microsoft.Network/virtualNetworks/test/virtualNetworkPeerings/test' or the scope is invalid. If access was recently granted, please refresh your credentials.

image.png

Peer Non-Vending VNet with the Vending VNet

Attempted to peer a non-Vending VNet with the VNet that was created through the LZ Vending, and received the following error:

Failed to add virtual network peering
Failed to add virtual network peering 'test' to 'abc123-dev-vwan-spoke'. Error: The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have authorization to perform action 'Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write' over scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/abc123-dev-networking/providers/Microsoft.Network/virtualNetworks/abc123-dev-vwan-spoke/virtualNetworkPeerings/test' or the scope is invalid. If access was recently granted, please refresh your credentials.

image.png

@AErmie
Copy link
Author

AErmie commented Jan 15, 2025

Scenario: Changing the DNS server configuration

Attempted various permeations of the following permissions:

"notActions": [
    "Microsoft.Network/virtualNetworks.properties.dhcpOptions.dnsServers"
]

But it seems it is not possible to reference a property of an object for IAM permission control. However, discovered the following custom Community-based Azure Policies that control VNet DNS settings:

Update

Created and deploy a customized version of the "Deny changing VNet DNS settings from pre-defined value" policy. Attempted to change the VNet DNS configuration (both to a different IP address, and setting it to "Default (Azure-provided)"), and received the following error:

Failed to save DNS settings for virtual network 'abc123-dev-vwan-spoke'. Error: Resource 'abc123-dev-vwan-spoke' was disallowed by policy. Policy identifiers: '[{"policyAssignment":{"name":"Prevent VNet DNS Changes","id":"/providers/Microsoft.Management/managementGroups/abc123/providers/Microsoft.Authorization/policyAssignments/6678dc769fe94e01b1091068"},"policyDefinition":{"name":"Prevent VNet DNS Changes","id":"/providers/Microsoft.Management/managementGroups/bcgov-managed-lz-forge-landing-zones/providers/Microsoft.Authorization/policyDefinitions/ea6188f3-9fa4-4ff3-901d-3eba6d65d6e6"}}]'.

image.png

Implementation

To implement this across different environments (ie. FORGE vs LIVE), we will need to create a custom Policy Definition, Policy Assignment, and use the local.archetype_config_overrides to control the required values for the dnsSettings parameter.

@AErmie
Copy link
Author

AErmie commented Jan 15, 2025

Scenario: Not create a new VNet, but be still be able to modify an existing one

Created a custom role with the following permissions:

"actions": [
    "*"
],
"notActions": [
    "Microsoft.Network/virtualNetworks/write"
],

Attempted to create a new VNet, and received the following error:

The template deployment failed with error: 'Authorization failed for template resource 'test2' of type 'Microsoft.Network/virtualNetworks'. The client '[email protected]' with object id '5027fbbf-7ebd-4113-a697-60e4a8511117' does not have permission to perform action 'Microsoft.Network/virtualNetworks/write' at scope '/subscriptions/60e89f81-a15c-4d7a-9be3-c3795a33a277/resourceGroups/permissions-testing/providers/Microsoft.Network/virtualNetworks/test2'.'.

image.png

Then attempted to modify an existing VNet (ie. add a Subnet to the existing LZ Vending VNet), and was able to do so successfully.

image.png

image.png

@AErmie
Copy link
Author

AErmie commented Jan 20, 2025

Azure Policy Alternatives

To simplify RBAC, and remove the need to filter available Roles based, we want to identify if an Azure Policy (either built-in, or community created), can provide the same level of resource protection and restrictions.

Note

Tested the following Azure Policies with the User account being granted Contributor access at the Management Group level (ie. no custom RBAC role).

Azure Policies

  • Delete/Rename Management Groups and/or Subscriptions
    • Policy cannot protect the MG or Subscription for re-naming. It would have to be a custom RBAC role
  • Changing the DNS server configuration
  • Create any Express Route circuits, VPN sites, Route tables, VPN/NAT/Local Gateways
    • Created a custom policy "[TESTING] Deny Creating Protected Networking Resource"
  • Create/Modify/Remove VNet peering (ie. Edit the existing peer to the Hub, Peer non-Vending VNets with the Hub, or with their Vending provided VNet)
    • Policy Deny vNet peering
    • This policy is part of the ALZ Enterprise-Scale
    • Prevents creation and modification of peering, but not delete!
    • Will need customization, maybe through actionNames and denyAction
  • Create a new VNet, but still be able to modify an existing one (ie. add subnets)
    • Created a custom policy "[TESTING] Deny Creating VNets"
  • Modify the existing Diagnostics collection
    • Created a custom policy "[TESTING] Deny Modification of Diagnostics Settings"
    • The custom policy does not prevent modification; but the Contributor role does not have Provider Register permissions out-of-the-box
  • Modify or delete the existing Network Watcher

Custom Deny Networking Resource Policy

If there are existing resources defined in this policy, the user is able to still view them, and can also make edits/changes. Only affects net-new.

image.png

Deny VNet Peering Policy

Attempted to create a new VNet Peer. Prevents the creation of new peerings, and also blocks modification of existing peerings. However, it does not prevent deleting a VNet peering!

image.png

Deny Creating VNets

New VNet creation denied, but still able to add a subnet to an existing VNet. Also able to delete a subnet.

image.png

Deny Modification of Diagnostics Settings Policy

When attempting to update the existing setbypolicy Diagnostics Setting:
image.png

Note

The authorization to perform action 'Microsoft.Insights/register/action' error is not related to custom policy. This error occurs even without this policy assigned.

When attempting to delete the existing setbypolicy Diagnostics Setting:
image.png

@AErmie
Copy link
Author

AErmie commented Jan 20, 2025

Additional Testing

We need to test the following:

  • Ensure that the Microsoft.Insights Resource Provider is registered in the target Subscription, and then re-test the Diagnostics policy, to confirm modification is blocked
  • Validate if the VNet Peering policy shows the existing VNet's peering with the Hub as "out of compliance"

Updates

Diagnostics

Even with Owner permissions, the error "does not have authorization to perform action 'Microsoft.Insights/register/action' over scope" is thrown. This is not caused by the Policy assignment though (as this error occurs even without the Policy being assigned). So, it does prevent modification of the Diagnostic Settings. But it does not prevent adding a new Diagnostic Setting (and being able to update and/or delete that one).

VNet Peering

Confirmed that the Deny vNet peering policy does indicate the Hub peering as being out of compliance. Need to modify the policy to exclude this peering.

image.png

Added the following to the Policy Rule, which ignores the VNet peering created for the Hub:

{
    "field": "name",
    "like": "RemoteVnetToHubPeering*"
}

image.png

@AErmie
Copy link
Author

AErmie commented Jan 21, 2025

Policy successfully deployed and applied in FORGE.

Policies

Note: Policy Assignments have the same naming.

  • Deny Creating Protected Networking Resource
  • Deny Creating VNets
  • Deny Deleting Diagnostics Settings
  • Deny changing VNet DNS settings from pre-defined value
  • Deny New VNet Peering

Scenarios to test:

  • Modifying an existing Project Set
  • Creating a new Project Set

Scenario: New Project Set

Created request for new Project Set in FORGE (beb42d).

@AErmie
Copy link
Author

AErmie commented Jan 23, 2025

Polices added to the shared CAF module, and deployed in FORGE. PR created for LIVE to apply the policies (in Audit mode), which is pending the merging of PCS-35 Update CAF module

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant