Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions Documentation/TacitRed-Defender-ThreatIntelligence/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# TacitRed Defender Threat Intelligence - Solution Documentation

## Overview
**TacitRed Defender Threat Intelligence** provides a seamless integration between TacitRed's high-fidelity threat intelligence and **Microsoft Defender for Endpoint**. This solution enables automated ingestion of TacitRed findings as IOCs (Indicators of Compromise) into your Defender environment, enhancing your threat detection and response capabilities.

## Capabilities
- **Automated Ingestion**: Periodically fetches compromised credentials and malware indicators from TacitRed.
- **Customizable Filtering**: Allows filtering findings by specific domains or severity levels.
- **Microsoft Defender Integration**: Post indicators directly to the Defender API for immediate blocking and alerting.
- **Sentinel Monitoring**: Provides logs and monitoring within Azure Sentinel for operational visibility.

## Purpose
This package is designed for Security Operations Centers (SOCs) that utilize both TacitRed (by Data443) for external threat intelligence and Microsoft Defender for Endpoint for endpoint protection. It bridges the gap by automating the operationalization of threat intel.

## Support & Contact
- **Publisher**: Data443 Risk Mitigation, Inc.
- **Website**: [https://www.data443.com](https://www.data443.com)
- **Support Email**: [[email protected]](mailto:[email protected])
- **Product Page**: [TacitRed](https://www.data443.com/products/tacitred/)

## How to Run / Deploy Manually

### Prerequisites
1. **TacitRed API Key**: You must have a valid API key from your TacitRed account.
2. **Azure Subscription**: An active Azure subscription with permissions to deploy resources.
3. **Microsoft Sentinel**: A Log Analytics workspace with Sentinel enabled (optional, but recommended).

### Manual Deployment via Azure Portal (Custom Template)
1. Navigate to the [Azure Portal](https://portal.azure.com).
2. Search for **"Deploy a custom template"**.
3. Click **"Build your own template in the editor"**.
4. Copy the contents of `Package/mainTemplate.json` and paste it into the editor.
5. Click **Save**.
6. Fill in the required parameters:
- `TacitRed_ApiKey`: Your API Key.
- `Workspace`: The name of your Log Analytics Workspace.
- `Location`: The region of your workspace.
7. Click **Review + create** -> **Create**.

### Manual Deployment via PowerShell
```powershell
New-AzResourceGroupDeployment -ResourceGroupName "YourResourceGroup" `
-TemplateFile "Package/mainTemplate.json" `
-TacitRed_ApiKey "YOUR_API_KEY" `
-workspace "YourLogAnalyticsWorkspaceName"
```

## Pull Request
- **PR #13247**: [Azure-Sentinel Pull Request](https://github.com/Azure/Azure-Sentinel/pull/13247)
- **Status**: Submitted / Waiting for CI
165 changes: 165 additions & 0 deletions Project_Structure_and_Workflow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# Project Context, Structure, and Workflow Memory

## Critical Deployment Rules
**ALWAYS FOLLOW THESE RULES TO AVOID CONFLICTS:**
1. **One Solution Per PR**: Never bundle multiple solutions into a single Pull Request.
2. **Separate Branches**: Create a dedicated feature branch for each solution (e.g., `feature/solution-name`).
3. **Clean History**: Ensure your branch only contains commits relevant to that specific solution.

## Active Pull Requests & Status

### 1. Cyren Threat Intelligence
- **PR #13224**: [Azure-Sentinel Pull Request](https://github.com/Azure/Azure-Sentinel/pull/13224)
- **Status**: Active / In Review
- **Source Branch**: [`feature/cyren-threat-intelligence-clean`](https://github.com/mazamizo21/Azure-Sentinel/tree/feature/cyren-threat-intelligence-clean)

### 2. TacitRed CrowdStrike IOC
- **PR #13241**: [Azure-Sentinel Pull Request](https://github.com/Azure/Azure-Sentinel/pull/13241)
- **Status**: Active / In Review
- **Source Branch**: [`feature/tacitred-crowdstrike-ioc`](https://github.com/mazamizo21/Azure-Sentinel/tree/feature/tacitred-crowdstrike-ioc)

### 3. TacitRed Threat Intelligence (CCF)
- **PR #13242**: [Azure-Sentinel Pull Request](https://github.com/Azure/Azure-Sentinel/pull/13242)
- **Status**: Active / In Review
- **Source Branch**: [`feature/tacitred-ccf-hub-v2`](https://github.com/mazamizo21/Azure-Sentinel/tree/feature/tacitred-ccf-hub-v2)

### 4. TacitRed SentinelOne
- **PR #13243**: [Azure-Sentinel Pull Request](https://github.com/Azure/Azure-Sentinel/pull/13243)
- **Status**: Active / In Review
- **Source Branch**: [`feature/tacitred-sentinelone-v1`](https://github.com/mazamizo21/Azure-Sentinel/tree/feature/tacitred-sentinelone-v1)

### 5. TacitRed Defender Threat Intelligence
- **PR #13247**: [Azure-Sentinel Pull Request](https://github.com/Azure/Azure-Sentinel/pull/13247)
- **Status**: **Submitted / Waiting for CI**
- **Source Branch**: [`feature/tacitred-defender-ti`](https://github.com/mazamizo21/Azure-Sentinel/tree/feature/tacitred-defender-ti)

### Guidance
- Use **“Conversation”** tab on the PR to see reviewer comments.
- Use **“Checks”** tab to see latest SolutionValidations / arm‑ttk / KQL checks.

### Previous/Related PR
- **PR #13204**: [Azure-Sentinel Pull Request](https://github.com/Azure/Azure-Sentinel/pull/13204) (Superseded)

### Important Links

#### TacitRed CCF solution folders (in Azure repo)
- **TacitRedThreatIntelligence (Master)**: [Link](https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/TacitRedThreatIntelligence)
- **CyrenThreatIntelligence (Master)**: [Link](https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/CyrenThreatIntelligence)
- *Note: When the PR is merged, these folders in master will contain your final code.*

#### TacitRed CrowdStrike IOC solution
- **TacitRed-IOC-CrowdStrike (Master)**: [Link](https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/TacitRed-IOC-CrowdStrike)
- **TacitRed-IOC-CrowdStrike (Fork/PR Context)**: [Link](https://github.com/mazamizo21/Azure-Sentinel/tree/feature/tacitred-ccf-hub-v2threatintelligence/Solutions/TacitRed-IOC-CrowdStrike)

---

## Standard Operating Procedure (SOP)

### 1. Development (Staging)
- **Action**: Make all code changes, edits, and fixes in the **Staging** environment.
- **Locations**:
- **TacitRed CCF**: `sentinel-production/Tacitred-CCF-Hub-v2`
- **Cyren CCF**: `sentinel-production/Cyren-CCF-Hub`
- **TacitRed CrowdStrike**: `sentinel-production/TacitRed-IOC-CrowdStrike`
- **TacitRed SentinelOne**: `sentinel-production/TacitRed-SentinelOne`
- **Note**: These folders are the **Source of Truth**. Any changes made directly to the Production folder will be overwritten by the Deployment script.

### 2. Validation (Local)
- **Action**: Run validation tools locally against the **Staging** files to catch errors before uploading.
- **Location**: `sentinel-production/Project/Deployment-Workflows/`
- **Tools**:
- **API Version Check**: Verify all Azure resource API versions are up-to-date
- Check https://learn.microsoft.com/en-us/azure/templates/ for latest versions
- Common resources: Microsoft.Web/sites (2024-04-01), Microsoft.Logic/workflows (2019-05-01)
- **ARM-TTK**: Run `RUN-TTK-Validation.ps1 -SolutionName "Tacitred-CCF-Hub-v2"` (or other solution name).
- **TruffleHog**: Automatically run as part of the deployment script, or manually via `TruffleHog/run_safe_scan.sh`.


### 3. Promotion & Deployment (Unified)
- **Action**: Run the **Unified Deployment Script** to handle everything end-to-end.
- **Script**: `DEPLOY-UNIFIED.ps1`
- **Location**: `sentinel-production/Project/Deployment-Workflows/`
- **Usage**:
- **Live Deployment**: `pwsh -NoLogo -ExecutionPolicy Bypass -File ./Project/Deployment-Workflows/DEPLOY-UNIFIED.ps1`
- **Dry Run (Test)**: `pwsh -NoLogo -ExecutionPolicy Bypass -File ./Project/Deployment-Workflows/DEPLOY-UNIFIED.ps1 -DryRun`
- **What this SINGLE script does**:
1. **Security Scan**: Runs TruffleHog once for the whole project.
2. **Upstream Sync**: Syncs your repo with Microsoft's `master` branch once.
3. **Loop Through All Solutions**:
* **Auto-Versioning**: Increments version in `packageMetadata.json` (and `mainTemplate.json` if applicable).
* **Packaging**: Zips the appropriate folder (`Data Connectors` or `Playbooks`) into a versioned zip (e.g., `3.0.1.zip`).
* **Promote**: Copies all files to the Production folder.
* **Git Stage**: Adds changes to git staging area.
4. **Commit & Push**: Commits all changes for all solutions in one go and pushes to GitHub (Microsoft Fork).
5. **Sync to Data443**: Automatically pushes the same changes to the private Data443 repository (`data443` remote) as a backup.

### 4. CI/CD (Remote)
- **Action**: Monitor the Pull Request on GitHub.
- **Check**: Ensure "SolutionValidations", "TruffleHog", and other Microsoft CI checks pass.

---

## Environments & Structure

### Staging
- **TacitRed CCF**: `sentinel-production/Tacitred-CCF-Hub-v2`
- **Cyren CCF**: `sentinel-production/Cyren-CCF-Hub`
- **TacitRed CrowdStrike**: `sentinel-production/TacitRed-IOC-CrowdStrike`
- **TacitRed SentinelOne**: `sentinel-production/TacitRed-SentinelOne`

### Production
- **Location**: `sentinel-production/Project/Tools/Azure-Sentinel/Solutions/`
- **Purpose**: The official production version of the solutions, located within the Azure-Sentinel solutions repository structure.

## Tools

### ARM TTK (Template Test Kit)
- **Location**: `sentinel-production/Project/Tools/arm-ttk`
- **Runner Script**: `sentinel-production/Project/Deployment-Workflows/RUN-TTK-Validation.ps1`

### Sentinel CI
- **Location**: `sentinel-production/Project/Tools/SentinelCI`

## Workflows

### Unified Deployment
- **Directory**: `sentinel-production/Project/Deployment-Workflows`
- **Script**: `DEPLOY-UNIFIED.ps1`
- **Features**: Auto-versioning, Auto-zipping, TruffleHog Scan, Upstream Sync, Git Push.

## Pre-Submission Checklist (Critical Lessons Learned)
Before creating a Pull Request, you **MUST** verify the following to ensure it passes 'SolutionValidations' and 'SafeToRun' constraints:

### 1. File Hygiene
- [ ] **Allowed Extensions Only**: Ensure the solution folder contains **ONLY** `.json`, `.zip`, `.md`, `.txt`, `.png`, `.svg`.
- [ ] **Prohibited Files**: Remove ALL `.ps1`, `.py`, `.sh`, `.exe`, `.dll`, `.bin` files.
- [ ] **Clean Up**: Remove any temporary files (`.outofscope`, `.bak`) and **DELETE OLD ZIP VERSIONS** (only keep latest).
- [ ] **Common Tools**: Do NOT modify shared scripts like `Tools/Create-Azure-Sentinel-Solution/common/commonFunctions.ps1`.

### 2. Metadata Consistency
- [ ] **Resource Existence**: Ensure `mainTemplate.json` includes the `Microsoft.OperationalInsights/workspaces/providers/contentPackages` resource (kind: Solution).
- [ ] **Variables**: Ensure `mainTemplate.json` has `_solutionName`, `_solutionVersion`, `_solutionId` variables defined.
- [ ] **Version Match**: The `_solutionVersion` in `mainTemplate.json` **MUST MATCH** the version in `packageMetadata.json`.

### 3. JSON Validation
- [ ] **Syntax Check**: Run `jq empty mainTemplate.json` or use an IDE linter to ensure valid JSON syntax. Trailing commas are a common failure cause.

## Standardized Solution Mapping

| Solution Name | Staging Path | Prod Zip Folder | Notes |
| :--- | :--- | :--- | :--- |
| **TacitRedThreatIntelligence** | `Tacitred-CCF-Hub` | `Solutions/TacitRedThreatIntelligence/Package` | Uses `Data Connectors` zip source. |
| **CyrenThreatIntelligence** | `Cyren-CCF-Hub` | `Solutions/CyrenThreatIntelligence/Package` | Uses `Data Connectors` zip source. |
| **TacitRed-IOC-CrowdStrike** | `TacitRed-IOC-CrowdStrike` | `Solutions/TacitRed-IOC-CrowdStrike/Package` | Uses `Playbooks` zip source. |
| **TacitRed-SentinelOne** | `TacitRed-SentinelOne` | `Solutions/TacitRed-SentinelOne/Package` | Uses `Playbooks` zip source. |

## PR & Validation Procedure

1. **Clean Staging**: Always delete `*.zip` in Staging *before* running deployment script (`rm *.zip`).
2. **Run Script**: `pwsh ... -SolutionName "TacitRedThreatIntelligence"`.
3. **Clean Production**: The script creates a new zip but **does not delete old ones** in Prod. You **MUST** manually run `rm 1.0.X.zip` in `Tools/Azure-Sentinel/Solutions/.../Package` to leave *only* the new version.
4. **Push & Verify**: Push to branch. Check PR.
5. **Security Approval**: New PRs require "Security Approval" in GitHub. You cannot bypass this. Ask repository owner.

## Sync Documentation
When updating `Project_Structure_and_Workflow.md`, you **MUST** sync this single file to ALL active feature branches immediately to prevent outdated instructions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"Name": "TacitRed-Defender-ThreatIntelligence",
"Author": "Data443 Risk Mitigation, Inc. - [email protected]",
"Logo": "<img src=\"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Logos/Azure_Sentinel.svg\"width=\"75px\"height=\"75px\">",
"Description": "The TacitRed Defender Threat Intelligence solution integrates TacitRed's threat intelligence feed with Microsoft Sentinel. It automatically retrieves compromised credentials and other threat indicators from TacitRed and ingests them into Microsoft Sentinel using the Upload API for enhanced threat detection.",
"Data Connectors": [
"Package/mainTemplate.json"
],
"Playbooks": [
"Playbooks/TacitRedToDefenderTI_Playbook.json"
],
"Metadata": "SolutionMetadata.json",
"BasePath": "C:\\GitHub\\Azure-Sentinel\\Solutions\\TacitRed-Defender-ThreatIntelligence",
"Version": "1.0.0",
"TemplateSpec": true,
"Is1Pconnector": false
}
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
{
"$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#",
"handler": "Microsoft.Azure.CreateUIDef",
"version": "0.1.2-preview",
"parameters": {
"config": {
"isWizard": false,
"basics": {
"description": "**TacitRed Defender Threat Intelligence** automates the synchronization of threat intelligence from TacitRed to Microsoft Defender.\n\n**Playbooks:** 1\n\n[Learn more about TacitRed >](https://www.tacitred.com/)",
"subscription": {
"constraints": {
"validations": [
{
"permission": "Microsoft.OperationalInsights/workspaces/read",
"message": "Please ensure you have read permissions for the workspace"
}
]
},
"resourceProviders": [
"Microsoft.OperationalInsights/workspaces",
"Microsoft.Logic"
]
},
"location": {
"label": "Location",
"toolTip": "Location for all resources",
"resourceTypes": [
"Microsoft.OperationalInsights/workspaces"
]
},
"resourceGroup": {
"allowExisting": true
}
}
},
"basics": [
{
"name": "getLAWorkspace",
"type": "Microsoft.Solutions.ArmApiControl",
"toolTip": "This fetches all workspaces in the subscription",
"condition": "[greater(length(resourceGroup().name),0)]",
"request": {
"method": "GET",
"path": "[concat(subscription().id,'/providers/Microsoft.OperationalInsights/workspaces?api-version=2020-08-01')]"
}
},
{
"name": "workspace-guidance",
"type": "Microsoft.Common.TextBlock",
"options": {
"text": "Please select the Microsoft Sentinel Workspace where you want to install the solution. This must be in the same Resource Group selected above."
}
},
{
"name": "workspace",
"type": "Microsoft.Common.DropDown",
"label": "Sentinel Workspace",
"placeholder": "Select your Sentinel Workspace",
"toolTip": "Select the Microsoft Sentinel workspace where you want to deploy the solution. It must be in the current Resource Group.",
"constraints": {
"allowedValues": "[map(filter(basics('getLAWorkspace').value, (filter) => contains(toLower(filter.id), toLower(resourceGroup().name))), (item) => parse(concat('{\"label\":\"', item.name, '\",\"value\":\"', item.name, '\"}')))]",
"required": true
},
"visible": true
}
],
"steps": [
{
"name": "configuration",
"label": "Configuration",
"bladeTitle": "Configuration",
"elements": [
{
"name": "config-text",
"type": "Microsoft.Common.TextBlock",
"options": {
"text": "Please provide the configuration details for the Logic App."
}
},
{
"name": "TacitRed_ApiKey",
"type": "Microsoft.Common.PasswordBox",
"label": "TacitRed API Key",
"toolTip": "The API Key for TacitRed authentication.",
"constraints": {
"required": true,
"regex": "^[A-Za-z0-9]+$",
"validationMessage": "Please enter a valid API Key (alphanumeric)."
}
}
]
}
],
"outputs": {
"location": "[location()]",
"workspace": "[basics('workspace')]",
"TacitRed_ApiKey": "[steps('configuration').TacitRed_ApiKey]"
}
}
}
Binary file not shown.
Loading