Merge pull request #40 from mitre-attack/doc-cleanup

Remove extensible schemas and update documentation

Author: seansica

.gitignore

@@ -1,3 +1,6 @@
+# auto-generated schema documentation
+docusaurus/docs/reference/schemas/
+
 # TypeScript artifacts
 *.d.ts
 *.ts.map

README.md

@@ -1,14 +1,13 @@
 # MITRE ATT&CK® Data Model
 
-The ATT&CK Data Model (ADM) is a TypeScript library that provides a structured way to interact with MITRE ATT&CK datasets. It uses Zod schemas, TypeScript types, and ES6 classes to create a type-safe, object-oriented interface for navigating the ATT&CK data model. This library is designed to parse, validate, and serialize STIX 2.1 formatted content, making it easy to work with ATT&CK-related data in a programmatic and intuitive way.
+**A TypeScript library for working with MITRE ATT&CK data using STIX 2.1**
 
-You can browse the ATT&CK schemas in a user-friendly interface at:
+The ATT&CK Data Model (ADM) provides a type-safe, object-oriented interface for working with MITRE ATT&CK datasets.
+Built on STIX 2.1 compliance, it uses Zod schemas and TypeScript types to ensure data integrity while providing intuitive relationship navigation between ATT&CK objects.
 
-https://mitre-attack.github.io/attack-data-model/. 
+**[CLICK HERE](https://mitre-attack.github.io/attack-data-model) <sup>[1](#footnotes)</sup>** to browse the ATT&CK schemas in a user-friendly interface. 
 
-This site is dynamically generated from the contents of the `@latest` distribution channel / `main` branch. Please note that we do not currently maintain separate documentation for previous releases.
-
-## Features
+## Key Features
 
 - **Type-Safe Data Parsing**: ADM validates STIX 2.1 bundles using Zod schemas, ensuring data model compliance and type safety.
 - **Easy Relationship Navigation**: Each object instance contains pointers to related objects, simplifying the process of navigating between techniques, tactics, and other ATT&CK elements.
@@ -91,9 +90,9 @@ For most users, we recommend:
 
 Example of loading the latest ATT&CK data:
 ```javascript
-import { registerDataSource, loadDataModel, DataSource } from '@mitre-attack/attack-data-model';
+import { registerDataSource, loadDataModel, DataSourceRegistration } from '@mitre-attack/attack-data-model';
 
-const dataSource = new DataSource({
+const dataSource = new DataSourceRegistration({
     source: 'attack',
     domain: 'enterprise-attack',
     version: '17.1',
@@ -133,12 +132,12 @@ For additional context about the ATT&CK specification, please refer to the [ATT&
 Here's an example script that demonstrates how to use the ADM library to load ATT&CK data from the official MITRE ATT&CK GitHub repository:
 
 ```typescript
-import { registerDataSource, loadDataModel, DataSource } from '@mitre-attack/attack-data-model';
+import { registerDataSource, loadDataModel, DataSourceRegistration } from '@mitre-attack/attack-data-model';
 
 (async () => {
-    
-    // Instantiating a DataSource object will validate that the data source is accessible and readable
-    const dataSource = new DataSource({
+
+    // Instantiating a DataSourceRegistration object will validate that the data source is accessible and readable
+    const dataSource = new DataSourceRegistration({
         source: 'attack', // Built-in index to retrieve ATT&CK content from the official MITRE ATT&CK STIX 2.1 GitHub repository
         domain: 'enterprise-attack',
         version: '15.1', // Omitting 'version' will default to the latest version available in the repository
@@ -160,7 +159,7 @@ import { registerDataSource, loadDataModel, DataSource } from '@mitre-attack/att
 
             // Type hinting is supported for all object properties
             if (technique.x_mitre_is_subtechnique) {
-                
+
                 // Access related objects with helpful getter methods
                 console.log(technique.getParentTechnique());
             }
@@ -231,20 +230,24 @@ For more detailed examples, please refer to the [examples](./examples/README.md)
 
 ## Compatibility Matrix
 
-Our [COMPATIBILITY.md](./docs/COMPATIBILITY.md) document tracks the compatibility between versions of the ATT&CK Data Model (ADM) TypeScript API (`@mitre-attack/attack-data-model`) and versions of the MITRE ATT&CK® dataset (`mitre-attack/attack-stix-data`).
+Our [Compatibility documentation](https://mitre-attack.github.io/attack-data-model/principles/versioning-philosophy) tracks the compatibility between versions of the ATT&CK Data Model (ADM) TypeScript API (`@mitre-attack/attack-data-model`) and versions of the MITRE ATT&CK dataset (`mitre-attack/attack-stix-data`).
 
 ## Contributing
 
 We welcome contributions! Please see our [CONTRIBUTING.md](./docs/CONTRIBUTING.md) file for details on how to contribute to this project.
 
+## Footnotes
+
+<sup>1</sup> The [schemas site](https://mitre-attack.github.io/attack-data-model) is dynamically generated from the contents of the `@latest` distribution channel / `main` branch. We do not currently maintain separate documentation for previous releases, though we hope to in the future.
+
 ## License
 
 This project is licensed under the Apache 2.0 License.
 
-## Notice 
+## Notice
 
 Copyright 2020-2025 The MITRE Corporation.
 
-This project makes use of ATT&CK®
+This project makes use of ATT&CK
 
-[ATT&CK Terms of Use](https://attack.mitre.org/resources/terms-of-use/)
+[ATT&CK Terms of Use](https://attack.mitre.org/resources/terms-of-use/)

docs/CONTRIBUTING.md

@@ -61,9 +61,21 @@ When submitting a pull request:
 5. Increase the version numbers in any examples files and the README.md to the new version that this Pull Request would represent. The versioning scheme we use is [SemVer](http://semver.org/).
 6. Include a description of your changes and why they're necessary.
 
+## Developer Setup
+
+### Requirements
+
+- [Node.js](https://nodejs.org/) v18.20
+
+### Install dependences
+
+```bash
+npm install
+```
+
 ## Developer Workflow
 
-To maintain code quality and consistency, we use **ESLint**, **Prettier**, and **Husky** as part of the development workflow. Below is an overview of how each tool is configured and how it affects the contribution process:
+To maintain code quality and consistency, we use **ESLint**, **Prettier**, and **Husky** as part of the development workflow. These tools are installed and configured by the [Developer Setup](#developer-setup) steps above. Below is an overview of how each tool is configured and how it affects the contribution process:
 
 ### ESLint and Prettier Configuration
 

docs/SPEC.md

@@ -1,10 +1,10 @@
 > [!IMPORTANT]
 > **Documentation Notice**
-> 
+>
 > This document is **not the source of truth** for the ATT&CK specification. The authoritative source is the **ATT&CK Data Model (ADM) TypeScript library**.
-> 
-> 📖 **Browse ATT&CK schemas:** https://mitre-attack.github.io/attack-data-model/
-> 
+>
+> 📖 **Browse ATT&CK schemas:** <https://mitre-attack.github.io/attack-data-model/>
+>
 > While maintained to the best of our ability, this documentation may drift from the ADM library. If you find discrepancies, please [open a GitHub Issue](https://github.com/mitre-attack/attack-data-model/issues).
 
 # The ATT&CK Specification
@@ -60,8 +60,7 @@ There are three general ways that ATT&CK extends the STIX 2.1 specification:
 | `x_mitre_domains`                         | string[] | Identifies the domains the object is found in. See [domains](#domains) for more information.<br>Not found on objects of type `relationship`, `identity`, or `marking-definition`.                                                                                                                                                                           |
 | `x_mitre_attack_spec_version`<sup>1</sup> | string   | The version of the ATT&CK specification used by the object. Consuming software can use this field to determine if the data format is supported. Current version is 3.3.0. |
 
-
-- New relationship types. Unlike custom object types and extended fields, custom relationship types are **not** prefixed with `x_mitre_`. You can find a full list of relationship types in the [Relationships](#Relationships) section, which also mentions whether the type is a default STIX type.
+- New relationship types. Unlike custom object types and extended fields, custom relationship types are **not** prefixed with `x_mitre_`. You can find a full list of relationship types in the [Relationships](#relationships) section, which also mentions whether the type is a default STIX type.
 
 Please see also the STIX documentation on [customizing STIX](https://docs.oasis-open.org/cti/stix/v2.0/csprd01/part1-stix-core/stix-v2.0-csprd01-part1-stix-core.html#_Toc476227365).
 
@@ -93,7 +92,7 @@ The `x_mitre_domains` (string array) field identifies the "domain" to which the
 | `mobile-attack`     | Mobile         |
 | `ics-attack`        | ATT&CK for ICS |
 
-Most objects in ATT&CK belong in a single technology domain, but an object _may_ be included in multiple domains. 
+Most objects in ATT&CK belong in a single technology domain, but an object _may_ be included in multiple domains.
 
 `x_mitre_domains` is supported on all ATT&CK object types except the following:
 
@@ -127,6 +126,7 @@ ATT&CK IDs are human-readable identifiers commonly used for referencing objects
 | [Log Source](#log-sources)                | `LSxxxx`                                |
 
 **Important limitations:**
+
 - ATT&CK IDs are not guaranteed to be unique (see [Collisions with Technique ATT&CK IDs](#collisions-with-technique-attck-ids))
 - Matrices within the same domain share identical ATT&CK IDs
 - Relationship objects do not have ATT&CK IDs
@@ -213,6 +213,7 @@ Sub-techniques are specialized implementations of parent techniques, providing m
 Procedures describe specific instances of technique implementation by adversaries. Unlike other ATT&CK concepts, procedures are not represented by dedicated STIX objects. Instead, they are modeled as `uses` relationships where the `target_ref` points to a technique (`attack-pattern`).
 
 **Procedure relationships:**
+
 - **Source objects:** Groups (`intrusion-set`) or software (`malware`/`tool`)
 - **Target objects:** Techniques (`attack-pattern`)
 - **Content:** Procedure details are captured in the relationship's `description` field
@@ -250,6 +251,7 @@ Software represents tools and malicious code used by adversaries to accomplish t
 Data sources and data components define the telemetry and observational data that security teams can use to detect adversary techniques. This hierarchical model provides granular mapping between detection capabilities and techniques.
 
 **Structural relationships:**
+
 - Data components are nested within data sources but maintain separate STIX objects
 - Each data component has exactly one parent data source
 - Data sources can contain multiple data components
@@ -364,6 +366,7 @@ Detection strategies define high-level approaches for detecting specific adversa
 | `x_mitre_analytic_refs`     | string[] | Array of STIX IDs referencing `x-mitre-analytic` objects that implement this detection strategy.         |
 
 **Key characteristics:**
+
 - Each detection strategy has a one-to-one relationship with a specific ATT&CK technique
 - Detection strategies typically reference 1-3 analytics (one for each supported platform)
 - Uses soft relationships (STIX ID references) to analytics for flexibility
@@ -497,4 +500,4 @@ Relationship objects frequently include `description` fields that provide contex
 
 ### Collections
 
-See our [collections document](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md) for more information about the design and intention of collection objects.
+See our [collections document](https://github.com/center-for-threat-informed-defense/attack-workbench-frontend/blob/master/docs/collections.md) for more information about the design and intention of collection objects.

docs/USAGE.md

@@ -105,68 +105,44 @@ The library provides Zod schemas for all ATT&CK object types, enabling validatio
 
 ### Accessing Schemas
 
-Schemas are available under the `schemas` directory. You can import them directly from the package root:
+Schemas and their associated TypeScript types are available under the `schemas` directory. You can import them directly from the package root:
 
 ```typescript
-import { tacticSchema } from '@mitre-attack/attack-data-model';
+import { campaignSchema } from '@mitre-attack/attack-data-model';
+import type { Campaign } from '@mitre-attack/attack-data-model';
 ```
 
-Notably, there are two versions of each ATT&CK type available: 
-
-* One "extensible" schema (denoted by its `extensible` prefix) that returns a `ZodType`
-* One "standard" schema that returns a `ZodEffect` 
-
-The extensible schemas (e.g., `extensibleCampaignSchema`) return a `ZodType` by intentionally omitting all Zod refinements. You should use these schemas if you are looking to extend or modify the baseline schema behaviors. 
-
-For example, let's say you wish to augment ATT&CK campaigns with your own custom fields—you can do this with the `extensibleCampaignSchema` as follows:
-
-```typescript
-// Using the extensible schema for type definition or extension
-import { extensibleCampaignSchema } from '@mitre-attack/attack-data-model';
-const myCustomCampaignSchema = extensibleCampaignSchema.extend({ /* additional fields */ });
-```
+Many of the ATT&CK schemas use Zod refinements. We leverage refinements to execute advanced validation checks (e.g., validating that the first reference in `external_references` contains a valid ATT&CK ID).
 
-This would not work for the "standard" `campaignSchema`.
+Unfortunately, in current versions of Zod, if a schema is modified with one of the object methods (`pick`, `omit`, `extend`), the refinements will be discarded.
 
-`campaignSchema` returns a `ZodEffect` by nature of employing Zod refinements. We leverage refinements to execute advanced validation checks (e.g., validating that the first reference in `external_references` contains a valid ATT&CK ID). You can use the refined schemas like you would any other Zod schema, with the added disclaimer that they are less extensible than their aforementioned counterparts:
+For example, let's say you wish to augment ATT&CK campaigns with your own custom fields:
 
 ```typescript
-// Using the refined schema for validation
 import { campaignSchema } from '@mitre-attack/attack-data-model';
-const validCampaign = campaignSchema.parse(rawCampaignData);
+const myCustomCampaignSchema = campaignSchema.extend({ /* additional fields */ });
 ```
 
-And don't worry—you can still use these refinements with your custom schemas. Each ATT&CK refinement is decoupled so they can be used modularly. They are exported as factory functions in the `refinements` sub-package:
+`myCustomCampaignSchema` would not be valid, as it is missing the refinements that were originally present in `campaignSchema`.
 
-```typescript
-// Step 1 - import the refinements you want to use
-import { createFirstAliasRefinement, createCitationsRefinement } from '@mitre-attack/attack-data-model';
+You can still use the original refinements in your custom schemas, it will just take an extra step. Each ATT&CK refinement is decoupled so they can be used modularly. They are exported as factory functions in the `refinements` sub-package:
 
-// Step 2 - initialize the refinements
-const validateFirstAlias = createFirstAliasRefinement();
-const validateCitations = createCitationsRefinement();
+```typescript
+// Import the original schema, and the refinements you want to use
+import { campaignSchema, createFirstAliasRefinement, createCitationsRefinement } from '@mitre-attack/attack-data-model';
 
-// Step 3 - apply a single refinement that combines the imported refinements
-const myCustomCampaignSchema = extensibleCampaignSchema
+// Apply a single refinement that combines the imported refinements
+const myCustomCampaignSchema = campaignSchema
   .extend({ /* additional fields */ })
-  .superRefine((val, ctx) => {
-    validateFirstAlias(val, ctx);
-    validateCitations(val, ctx);
+  .check((ctx) => {
+    createFirstAliasRefinement()(ctx);
+    createCitationsRefinement()(ctx);
   });
 ```
 
-Notably, all ATT&CK schemas export only one TypeScript type, named in accordance with the refined schema, but inferred from the extensible schema. Since the refinements only add validation rules (rejected values) without changing the shape of valid data, a single type definition is sufficient:
-
-```typescript
-// An extensible schema for customizing or augmenting ATT&CK campaigns
-import { extensibleCampaignSchema } from '@mitre-attack/attack-data-model';
-
-// An inelastic but fully implemented schema for validating ATT&CK campaigns
-import { campaignSchema } from '@mitre-attack/attack-data-model';
+You will have to look in the original schema file, in this case [/src/schemas/sdo/campaign.schema.ts](/src/schemas/sdo/campaign.schema.ts) to see which refinements, if any, should be applied to the ATT&CK schema that you wish to extend.
 
-// One type definition for *all* ATT&CK campaigns (custom or otherwise)
-import type { Campaign } from '@mitre-attack/attack-data-model';
-```
+This [GitHub issue](https://github.com/colinhacks/zod/issues/4874) and [pull request](https://github.com/colinhacks/zod/pull/4865) describe the behavior and an upcoming `safeExtend` method that will allow you to extend the ATT&CK schemas without having to reapply the refinements.
 
 ### Validating Data
 

docusaurus/.gitignore

@@ -18,6 +18,3 @@
 npm-debug.log*
 yarn-debug.log*
 yarn-error.log*
-
-docs/*/*
-!docs/sdo/stix-bundle.schema.md

docusaurus/README.md

@@ -25,7 +25,7 @@ $ npm install
 The `generate-docs.sh` script is used to automate the process of converting Zod schemas into Markdown.
 
 ```bash
-$ npm run autodocs
+$ npm run gendocs
 ```
 
 ## Local Development

docusaurus/blog/known-issues.md

@@ -0,0 +1,46 @@
+# Coding Style & Linting
+
+ADM enforces a **single source-of-truth** for style:
+
+* **ESLint** – `@eslint/js` + custom rules
+* **Prettier** – code formatting
+* **commitlint** – conventional-commits
+
+## ESLint + Prettier
+
+```bash
+npm run lint        # read-only
+npm run format      # auto-fix + prettier
+```
+
+CI will fail if `npm run lint` reports errors or if Prettier formatting changes are uncommitted.
+
+## Typical workflow
+
+```bash
+git checkout -b feat/my-awesome-change
+# …code…
+npm run format
+git add .
+git commit -m "feat(core): add awesome change"
+```
+
+The commit message is checked by [commitlint](https://commitlint.js.org).
+Prefix your commit with one of the following:
+
+| Prefix   | Description                                    | Example Commit Command                                   |
+|----------|------------------------------------------------|----------------------------------------------------------|
+| feat     | A new feature                                  | `git commit -m "feat: add new schema"`                   |
+| fix      | A bug fix                                      | `git commit -m "fix: correct null pointer exception"`    |
+| docs     | Documentation only changes                     | `git commit -m "docs: update installation instructions"` |
+| chore    | Routine tasks, maintenance, or tooling         | `git commit -m "chore: update dependency versions"`      |
+| refactor | Code changes that neither fix nor add features | `git commit -m "refactor: simplify token validation"`    |
+| test     | Adding or updating tests                       | `git commit -m "test: add tests for date parser"`        |
+| perf     | Performance improvements                       | `git commit -m "perf: optimize query performance"`       |
+
+## TypeScript strictness
+
+The library is compiled with `"strict": true` and imports must be path-alias aware (`@/…`).
+Run `npm run build` to catch any type errors locally.
+
+---