AD RBAC Role Mining Toolchain

A two-stage toolchain for discovering natural role clusters in Active Directory environments of ~600 users. Stage 1 (PowerShell) exports group membership data read-only from AD. Stage 2 (Python) analyses that data to surface candidate RBAC roles.

No AD writes are performed at any point.

---

Contents

File	Purpose
Export-ADUserGroups.ps1	Stage 1 — PowerShell export from Active Directory
rbac_analyze.py	Stage 2 — Python 8-step analysis pipeline
config.yaml	All tunable thresholds (edit this, not the Python)
requirements.txt	Python package dependencies
README.md	This document
ALGORITHMS.md	In-depth algorithm reference (Jaccard, Louvain)

---

Prerequisites

Stage 1 (PowerShell)

• Windows machine joined to the target domain, or a machine with network access to a domain controller with RSAT installed.
• PowerShell 5.1 or later.
• ActiveDirectory PowerShell module:

  Get-WindowsCapability -Online -Name Rsat.ActiveDirectory* | Add-WindowsCapability -Online
  

• Read access to AD user and group objects (a standard domain user account is sufficient; no elevated privileges required).

Stage 2 (Python)

• Python 3.10 or later.
• Install dependencies:

  pip install -r requirements.txt

---

Pipeline Walkthrough

Step 1 — Export from Active Directory

Run on a domain-joined Windows machine:

.\Export-ADUserGroups.ps1 -Verbose

Optional parameters:

Parameter	Default	Description
-OutputDir	Current directory	Where to write the CSV files
-SearchBase	Entire domain	OU DN to restrict the search scope
-Server	PDC Emulator	Specific domain controller to target

Example with all options:

.\Export-ADUserGroups.ps1 `
    -OutputDir C:\RoleMining `
    -SearchBase "OU=Corp,DC=contoso,DC=com" `
    -Server dc01.contoso.com `
    -Verbose

Output files produced:

• users_groups_raw.csv — long format, one row per user–group pair: SamAccountName, DisplayName, Department, Title, GroupName, GroupDN
• users_meta.csv — one row per user: SamAccountName, DisplayName, Department, Title, OU, LastLogonDate, Enabled

Copy both CSV files to the machine where you will run Stage 2.

---

Step 2 — Python Analysis

python rbac_analyze.py

Or with a custom config path:

python rbac_analyze.py --config /path/to/config.yaml

The script runs 8 discrete steps and produces four output files:

Output	Description
preflight_report.txt	Data quality summary (Step 1)
dendrogram.png	Merge-structure visualisation (Step 5)
rbac_clusters.csv	Per-user cluster and community assignments
rbac_report.html	Full HTML report with shared groups, core groups, member tables, and cross-method comparison

Typical tuning loop:

1. Run with defaults.
2. Open dendrogram.png and preflight_report.txt.
3. Adjust distance_threshold (and optionally excluded_groups_*) in config.yaml.
4. Re-run until clusters align with organisational reality.
5. Rename ROLE_<DEPT>_<ID> placeholders to match your naming convention.

---

How to Read the Dendrogram

The dendrogram (dendrogram.png) is a tree diagram showing how users were merged into clusters step by step. The y-axis is Jaccard distance — a measure of how dissimilar two users' group memberships are.

Distance
  1.0 ┤                         ┌───────────────────────────┐
      │                         │                           │
  0.7 ┤               ┌─────────┤                   ┌───────┤
      │               │         │                   │       │
  0.5 ┤──threshold──  │    ┌────┘              ┌────┘  ┌───┘
      │               │    │                   │       │
  0.3 ┤          ┌────┘ ┌──┘         ┌─────┐  │  ┌────┘
      │          │      │            │     │  │  │
  0.0 ┤    [u1][u2]  [u3][u4]    [u5][u6][u7][u8][u9]

Reading guide:

• Tall merges (high y-axis) indicate very different users being forced into the same cluster — potential signal that distance_threshold is too high, or that two distinct roles exist in the same cluster.
• Short, flat merges (low y-axis) indicate users with nearly identical group membership — strong role candidates.
• The red dashed line shows the current distance_threshold. Everything below the line becomes a separate cluster; everything above is merged.
• Truncation: The plot shows only the last 50 merge operations (configurable via truncate_p). A node labelled (n) means n individual users were merged at that height.
• Moving the threshold left/right (lower/higher):
  • Lower threshold → more, smaller clusters (more granular roles)
  • Higher threshold → fewer, larger clusters (broader roles)

A good threshold sits just below a visible gap in the dendrogram — a height range where no merges occur, indicating a natural boundary between distinct user populations.

---

Configuration Reference

All parameters live in config.yaml. The file is fully commented; key settings to understand:

Parameter	Default	Effect
noise_removal.min_group_size	3	Drop groups with fewer members
noise_removal.excluded_groups_exact	Domain defaults	Exact names to remove
noise_removal.excluded_groups_regex	^Default.*	Regex patterns to remove
noise_removal.excluded_admin_groups	Privileged AD groups	IT/admin groups to skip
clustering.distance_threshold	0.5	Primary tuning knob
cluster_analysis.core_threshold	0.80	80% rule for core groups
cluster_analysis.outlier_multiplier	1.5	Outlier sensitivity
louvain.resolution	1.0	Louvain community granularity
preflight.service_account_max_groups	2	Service account detection

---

Known Failure Modes

Group Sprawl

Symptom: Every cluster has dozens of "shared" or "core" groups; the HTML report is overwhelming; preflight_report.txt shows hundreds of groups.

Cause: The AD environment has accumulated many low-membership groups over time — project groups, legacy distribution lists, one-off security groups.

Remedies:

• Raise min_group_size from 3 to 5 or 10.
• Add common noise groups to excluded_groups_regex (e.g. ^PRJ-.*, ^DL-.*).
• Review the Top 20 largest groups in preflight_report.txt and add any "everyone gets this" groups to excluded_groups_exact.

---

Legacy Groups

Symptom: Clusters mix users from completely different departments. The dendrogram shows no clear separation at any threshold.

Cause: Legacy groups were created for a past purpose (e.g. a retired application, an old OU structure) and were never cleaned up. Because these groups span departments, they artificially link users.

Remedies:

• Identify such groups in preflight_report.txt (look at the largest groups — they often represent decommissioned systems or old all-staff groups).
• Add them to excluded_admin_groups or excluded_groups_exact in config.yaml.
• In the longer term, consider a group lifecycle review.

---

Nested Group Inheritance

Symptom: Users appear in far more groups than expected. Service accounts or admin accounts appear to have hundreds of groups.

Cause: The PowerShell export resolves group membership recursively. A user in a leaf group that is itself nested three levels deep will appear as a member of all ancestor groups. In deeply nested environments this can multiply apparent membership significantly.

Remedies:

• This is usually correct behaviour — the inherited permissions are real.
• If nested groups are noise (e.g. a top-level "All Staff" group that contains every department group), add the top-level wrappers to excluded_groups_exact.
• Consider whether your AD design relies on nesting as a substitute for direct group assignment; the role mining output will reflect your actual permission effective set, which may differ from your intended design.

---

Service Accounts

Symptom: preflight_report.txt lists unexpected users as service account candidates; or known service accounts appear in clusters alongside human users.

Cause: Service/system accounts often have very few or very specialised group memberships that don't reflect job functions. Including them distorts clusters.

Remedies:

• Review the "Service Account Candidates" list in the HTML report. The threshold is service_account_max_groups (default: 2).
• Increase this value if your environment uses more groups for service accounts, or decrease it to be more permissive.
• For named service accounts you know in advance, handle them outside the toolchain (e.g. exclude them from the PowerShell export's search scope using a more specific -SearchBase).

---

Why Two Clustering Methods?

For a detailed mathematical treatment of both algorithms — including worked examples, complexity analysis, and the modularity formula — see ALGORITHMS.md.

The toolchain runs agglomerative (hierarchical) clustering and Louvain community detection independently, then compares their outputs. Here is why:

Agglomerative Clustering (primary)

Agglomerative clustering with Jaccard distance builds a hierarchy of similarity from the bottom up. It does not require you to specify the number of clusters in advance — you cut the hierarchy at a chosen distance threshold. This is ideal for role mining because you rarely know how many roles exist before looking at the data.

The dendrogram makes the method transparent and inspectable: you can see exactly where clusters form and why. The analyst retains full control over the granularity via distance_threshold.

Weakness: It is sensitive to the linkage method and threshold. A poorly chosen threshold can split a natural role into several fragments or merge distinct roles into one blob.

Louvain Community Detection (second opinion)

Louvain works on a different representation: a user–user graph where edges are weighted by the number of shared groups. It optimises a modularity function — seeking communities that have more internal connections than you would expect by chance. It is entirely independent of the Jaccard distance framework.

Weakness: Louvain is non-deterministic and resolution-sensitive. The resolution parameter is less intuitive than a distance threshold, and results can vary between runs.

Why compare both?

If both methods produce similar groupings, that agreement is strong evidence that the clusters reflect real structure in the data. If they disagree significantly, it signals one of three things:

1. The distance_threshold needs adjustment (inspect the dendrogram gap).
2. The resolution parameter for Louvain is misconfigured.
3. The data genuinely has ambiguous structure — users who straddle roles, interim assignments, or organic group accumulation over time.

The cross-method comparison table in the HTML report flags clusters where < 80% of members agree. These are the clusters worth the most scrutiny — they are where human judgement matters most.

The tools don't know your organisation. They show you patterns. You decide which patterns map to real roles.

---

Security Notes

• The PowerShell script is read-only. It calls only Get-ADUser and Get-ADPrincipalGroupMembership. No AD objects are created, modified, or deleted.
• The Python script reads CSV files and writes local output files only. It makes no network connections.
• The output files may contain sensitive HR data (usernames, departments, job titles). Handle and store them appropriately.
• Do not commit users_groups_raw.csv or users_meta.csv to source control.

---

Troubleshooting

Problem	Solution
community module not found	pip install python-louvain (the PyPI package is python-louvain, the import is community)
Get-ADPrincipalGroupMembership is slow	Normal for 600 users with deep nesting; expect 5–30 minutes
Fewer than 2 users after filtering	Noise filters are too aggressive — lower min_group_size or reduce excluded_groups_*
All users in one cluster	Raise distance_threshold; inspect dendrogram for a natural gap
600 separate single-user clusters	Lower distance_threshold; check whether meaningful groups survived filtering
HTML report is blank / missing sections	Check preflight_report.txt — likely all users were filtered as service accounts