Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 0 additions & 1 deletion .agents/skills/azure-ai-fine-tuning

This file was deleted.

1 change: 0 additions & 1 deletion .claude/skills/azure-ai-fine-tuning

This file was deleted.

50 changes: 0 additions & 50 deletions .github/ISSUE_TEMPLATE/bug_report.md

This file was deleted.

29 changes: 0 additions & 29 deletions .github/ISSUE_TEMPLATE/feature_request.md

This file was deleted.

6 changes: 6 additions & 0 deletions .github/requirements-docs.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
pytest>=9,<10
nbformat>=5.10,<6
PyYAML>=6,<7
jsonschema>=4,<5
python-dotenv>=1.1,<2
Pillow>=12,<13
1 change: 0 additions & 1 deletion .github/skills/azure-ai-fine-tuning

This file was deleted.

57 changes: 57 additions & 0 deletions .github/workflows/documentation-checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Root and managed documentation checks

on:
pull_request:
paths:
- '*.md'
- 'tests/**'
- 'managed_fine_tuning/**'
- 'interactive_training/docs/supported_models.md'
- '.github/requirements-docs.txt'
- '.github/workflows/documentation-checks.yml'
push:
branches: [main]
paths:
- '*.md'
- 'tests/**'
- 'managed_fine_tuning/**'
- 'interactive_training/docs/supported_models.md'
- '.github/requirements-docs.txt'
- '.github/workflows/documentation-checks.yml'

permissions:
contents: read

jobs:
offline-contracts:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
python: ['3.11', '3.13']
runs-on: ${{ matrix.os }}
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
cache: pip
cache-dependency-path: .github/requirements-docs.txt
- name: Check root navigation and current model eligibility
run: python -m unittest discover -s tests -p "test_*.py"
- name: Install offline contract test dependencies
run: python -m pip install -r .github/requirements-docs.txt
- name: Run managed offline contracts without cloud credentials
working-directory: managed_fine_tuning
run: >-
python -m pytest -q --tb=short
shared/tests/test_catalog.py
shared/tests/test_cookbook_contract.py
shared/tests/test_dataset_validation.py
shared/tests/test_config.py
shared/tests/test_logging_naming.py
shared/tests/test_retry.py
shared/tests/test_manifests_operations.py
shared/tests/test_notebook_execution.py
02-multimodal-sft/tests
43 changes: 43 additions & 0 deletions .github/workflows/interactive-training-checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Interactive training cookbook checks

on:
pull_request:
paths:
- 'interactive_training/**'
- '.github/workflows/interactive-training-checks.yml'
push:
branches: [main]
paths:
- 'interactive_training/**'
- '.github/workflows/interactive-training-checks.yml'

permissions:
contents: read

jobs:
tests:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
python: ['3.11', '3.13']
runs-on: ${{ matrix.os }}
timeout-minutes: 30
defaults:
run:
working-directory: interactive_training
env:
HF_HUB_DISABLE_TELEMETRY: '1'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
cache: pip
cache-dependency-path: interactive_training/pyproject.toml
- name: Install public SDK and test dependencies
run: python -m pip install -e ".[test]" --index-url https://pypi.org/simple --extra-index-url https://download.pytorch.org/whl/cpu
- name: Check dependencies
run: python -m pip check
- name: Run tests and documentation checks
run: python -m pytest -q --tb=short
42 changes: 42 additions & 0 deletions .github/workflows/smoke-testing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Smoke testing

on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:

permissions:
contents: read

concurrency:
group: smoke-testing
cancel-in-progress: false

jobs:
interactive-training:
if: vars.ENABLE_DAILY_TRAINING == 'true'
runs-on: ubuntu-latest
timeout-minutes: 360
environment: interactive-training-smoke
defaults:
run:
working-directory: interactive_training
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
cache-dependency-path: interactive_training/pyproject.toml
- name: Install cookbook
run: python -m pip install -e . --index-url https://pypi.org/simple --extra-index-url https://download.pytorch.org/whl/cpu
- name: Run recipes sequentially
env:
AZURE_AI_PROJECT_ENDPOINT: ${{ secrets.AZURE_AI_PROJECT_ENDPOINT }}
AZURE_AI_API_KEY: ${{ secrets.AZURE_AI_API_KEY }}
run: |
if [[ -z "$AZURE_AI_PROJECT_ENDPOINT" || -z "$AZURE_AI_API_KEY" ]]; then
echo "Configure AZURE_AI_PROJECT_ENDPOINT and AZURE_AI_API_KEY as environment secrets." >&2
exit 1
fi
python scripts/run_recipe_smokes.py --gap-minutes 10
73 changes: 73 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Coding-agent execution contract

These instructions apply to the repository. Use the public READMEs and maintained recipes; no separate fine-tuning skill installation is required.

## Route and resolve sources

1. Start with the root README, then the chosen track's README and setup guide. Use managed jobs for supported predefined methods; use interactive sessions for custom training-loop behavior. A custom managed grader alone does not require interactive training.
2. For managed work, read `managed_fine_tuning/README.md`, `catalog.yml`, the selected demo's `demo.yaml`, README, `.env.template`, and canonical `notebooks/demo.ipynb`. Use the learning path/task index to select the demo.
3. For interactive work, read `interactive_training/README.md`, `docs/quickstart.md`, `docs/auth.md`, and the selected recipe's README and configuration.
4. Interactive model/region eligibility comes from `interactive_training/docs/supported_models.md`. Managed eligibility comes from Foundry's model-specific documentation and the catalog prerequisites; notebook model roles are not a universal availability guarantee.
5. Track-specific setup guides define repository paths. Subtree metadata/templates define data and configuration; installed SDK signatures and implementation define operations. Historical changelog entries, benchmarks, and past runtime artifacts do not override these sources. If they conflict, report the conflict and stop before remote operations rather than guessing.

Before updating any README, check whether a more focused guide, recipe, or reference
document is a better home for the change. Keep READMEs useful as human-facing entry
points: link to detailed material rather than duplicating it or adding unrelated
implementation details.

## No-spend preflight

Read files and run offline tests, syntax/schema checks, dataset validation, and split/hash checks first. Do not run a whole notebook as a smoke test: later cells may upload data, allocate training, invoke tools, or submit/resume a paid job without another switch.

- Use the appropriate track's environment; managed uses Python 3.11-3.14 and its central lock, interactive uses its own package and SDK requirements.
- Check paths, kernel, working directory, exact model ID, schema, and prerequisites. Managed demos should run from their demo directory unless a tested alternative is documented.
- Preserve canonical/hash-pinned sample files. Put personal-data adaptations in separate, versioned inputs; never bypass validators to make them run.

Run root navigation and model-eligibility checks without cloud credentials:

```powershell
python -m unittest discover -s tests -p "test_*.py"
```

Use the repository workflow's selected managed tests for the managed contracts. Run interactive tests from `interactive_training` using its test environment. Tests validate local contracts, not cloud eligibility.

Dependency installation, network dataset downloads, credential prompts, or resource reads are not implied by an offline preflight. Explain them and obtain approval where needed.

## Explicit approval before spend or mutation

Approval is required before uploads, fine-tuning job creation/resume, deployment
creation, session allocation, interactive recipes, sampling/inference, external
tools, or cancellation/unload/deletion. Before the named operation, present:

- Task, target tenant/subscription/project, exact model and region, and verified
access/role/quota prerequisites or unresolved checks.
- Input provenance, license/privacy constraints, counts and hashes, and intended
upload destination. Do not expose populated endpoints or runtime identifiers.
- Method, steps/epochs/tokens, evaluation/tool-call scope, timeouts, stop
conditions, and budget. State known pricing or unresolved cost uncertainty;
time and step limits are not dollar caps.
- Expected artifacts, held-out evaluation and serving limitations, private
evidence retention, and cleanup responsibilities for every owned resource.

Wait for explicit approval covering the operation and budget. "Help me fine-tune"
is not blanket spend authorization. Do not provision or change permissions
automatically. Obtain separate deletion confirmation; stopping monitoring is not
cancellation, and closing a client is not remote unload.

Recurring runs require explicit approval of frequency, models/data, evaluation
scope, budget per run and over the approved period, expiry, owner, and disable/
reconciliation procedure. Manual-run approval is not recurring-run authority.
Follow [the paid smoke operator guide](PAID_SMOKE_TESTING.md); do not enable its
workflow or change environment permissions automatically.

## Secrets and runtime state

Never print or commit populated `.env` files, keys, tokens, endpoint/resource identifiers, or runtime manifests. Use supported credential providers; do not add silent authentication fallbacks. Logs must redact secrets. Keep runtime identifiers in the selected demo's ignored `outputs/` or the interactive run location documented by its storage guide.

## Monitor, recover, evaluate, and clean up

Record job/session/checkpoint IDs locally and monitor terminal state with the documented helpers. A timeout is not proof that the remote operation failed or was cancelled. Inspect existing state before retrying an upload or submission; use documented reuse/resume contracts and do not create duplicates blindly.

Save durable checkpoints and driver state before interruption. Distinguish training-state checkpoints from sampler checkpoints and deployed endpoints. Most managed demos stop at training metrics; only Retail adds comparisons with externally provisioned deployments. Report that boundary, and never turn lower loss or a past benchmark into a claim of held-out task improvement.

At handoff, report operations performed, artifacts and remaining resources, evaluation limitations, and ongoing cost exposure without exposing secrets. Cancellation, session unload, local-client closure, and deployment deletion are separate actions. Do not clean up shared or unidentified resources.
17 changes: 16 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,21 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [3.0.0] - 2026-10-07

Interactive training and managed fine-tuning refresh ([#42](https://github.com/microsoft-foundry/fine-tuning/pull/42)).

### Added
- Interactive training cookbook, training recipes, SDK guidance, and supporting documentation.
- Refresh shared documentation, tests, and repository guidance for both training tracks.
- Explicit access-approval notices and the preview sign-up link for interactive training.

### Changed
- Reorganize existing fine-tuning examples, datasets, and notebooks into a `managed_fine_tuning` folder

### Removed
- Legacy standalone fine-tuning agent skill and its auto-discovery symlinks; maintained repository guides and recipes now define the supported workflows.

## [2.1.0] - 2026-04-21

### Added
Expand Down Expand Up @@ -50,7 +65,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
### Added
- Enhanced main README with navigation table, quickstart section, and demo links
- Getting Started guide (`GETTING_STARTED.md`) with step-by-step setup instructions
- Schema documentation (`SCHEMA.md`) for all Sample_Datasets (SFT, DPO, RFT)
- Schema documentation (`SCHEMA.md`) for all managed fine-tuning datasets (SFT, DPO, RFT)
- Troubleshooting sections to demos that were missing them
- Missing `.env.template` and `requirements.txt` for RFT_Countdown demo
- Issue templates for bug reports and feature requests
Expand Down
Loading
Loading