Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/dco.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

require:
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
members: false
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
2 changes: 1 addition & 1 deletion docs/tutorials/differential-privacy.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@
" .with_data_source(df) # .with_replace_pii(enable=False) to disable PII replacement\n",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Unescaped double quotes break the notebook's JSON

The Jupyter notebook format stores cell source as a JSON array of strings. Python's dict literal {"enabled": True} contains double-quote characters that must be escaped as \" inside a JSON string. The file currently lacks this escaping, making the entire .ipynb file invalid JSON — Jupyter will refuse to open it.

Confirmed: python3 -c "import json,sys; json.load(sys.stdin)" < docs/tutorials/differential-privacy.ipynb exits with a JSONDecodeError. Use single quotes in the Python example ({'enabled': True}) or re-save the notebook through Jupyter which will escape the inner quotes automatically.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agent (review-pr): Notebook JSON issue is valid

Agree, this still needs a fix; render it in notebook to verify.

" .with_differential_privacy(dp_enabled=True, delta=recommended_delta)\n",
" .with_train(batch_size=16) # Override the default batch size of 1, which is designed for non-DP training\n",
" .with_generate(use_structured_generation=True) # Improves the percentage of valid records when DP is enabled\n",
" .with_generate(structured_generation={"enabled": True}) # Improves the percentage of valid records when DP is enabled\n",
")\n",
"builder.run()\n",
"results = builder.results"
Expand Down
42 changes: 38 additions & 4 deletions docs/user-guide/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,40 @@ synthesizer.run()
See [Using YAML Config Files with the CLI and SDK](running.md#using-yaml-config-files)
for more detail on combining config files with runtime overrides.

### Python Parameter Construction

The Python SDK accepts both fully nested config objects and compatibility
shortcuts for fields on top-level parameter sections:

```python
from nemo_safe_synthesizer.config import SafeSynthesizerParameters

config = SafeSynthesizerParameters.from_params(
num_records=2000, # generation.num_records
dp_enabled=True, # privacy.dp_enabled
structured_generation={"enabled": True}, # generation.structured_generation.enabled
)
```

Flat keyword arguments are matched by field name against the top-level parameter
sections. Use the nested shape for fields inside nested subobjects, especially
when a generic field name could appear in multiple places:

```python
# Preferred: unambiguous nested form.
SafeSynthesizerParameters.from_params(
structured_generation={"enabled": True},
)

# Also valid: fully nested generation section.
SafeSynthesizerParameters.from_params(
generation={"structured_generation": {"enabled": True}},
)

# Avoid: this configures evaluation.enabled, not structured generation.
SafeSynthesizerParameters.from_params(enabled=True)
```

---

## Training
Expand Down Expand Up @@ -158,10 +192,10 @@ for the full API reference.
| `generation.repetition_penalty` | `1.0` | Penalty for repeated tokens; increase slightly if generation produces repetitive output | 1.0--1.15 typical; start at 1.05 if repetition is a problem |
| `generation.patience` | `3` | Consecutive bad batches before stopping | Leave at default |
| `generation.invalid_fraction_threshold` | `0.8` | Invalid record fraction that triggers the patience counter | Leave at default |
| `generation.use_structured_generation` | `false` | Enable structured output to constrain record format (typically at the cost of reducing the quality of generated records and increasing generation time; use when the pipeline struggles to produce valid records) | Leave off unless the pipeline cannot produce valid records |
| `generation.structured_generation_backend` | `"auto"` | vLLM guided-decoding backend | Leave at `"auto"` |
| `generation.structured_generation_schema_method` | `"auto"` | Schema method (`"auto"`, `"structural_tag"`, `"json_schema"`, or `"regex"`) | Leave at `"auto"`; it picks `"structural_tag"` on xgrammar-capable backends and `"regex"` otherwise |
| `generation.structured_generation_use_single_sequence` | `false` | Match exactly one sequence when `max_sequences_per_example` is 1 | Leave at default |
| `generation.structured_generation.enabled` | `false` | Enable structured output to constrain record format (typically at the cost of reducing the quality of generated records and increasing generation time; use when the pipeline struggles to produce valid records) | Leave off unless the pipeline cannot produce valid records |
| `generation.structured_generation.backend` | `"auto"` | vLLM guided-decoding backend | Leave at `"auto"` |
| `generation.structured_generation.schema_method` | `"auto"` | Schema method (`"auto"`, `"structural_tag"`, `"json_schema"`, or `"regex"`) | Leave at `"auto"`; it picks `"structural_tag"` on xgrammar-capable backends and `"regex"` otherwise |
| `generation.structured_generation.use_single_sequence` | `false` | Match exactly one sequence when `max_sequences_per_example` is 1 | Leave at default |
| `generation.enforce_timeseries_fidelity` | `false` | Enforce time series order, intervals, and timestamps | Enable for time series data |
| `generation.attention_backend` | `"auto"` | vLLM attention backend | Leave at `"auto"` |

Expand Down
28 changes: 17 additions & 11 deletions docs/user-guide/running.md
Original file line number Diff line number Diff line change
Expand Up @@ -864,16 +864,16 @@ flowchart TD

### Structured Generation

Set `generation.use_structured_generation` to `true` to constrain the model's
output so every record matches the dataset schema. This reduces the fraction of
invalid records, typically at the cost of reducing the quality of the generated
records. Use it when the pipeline struggles to produce valid records.
Set `generation.structured_generation.enabled` to `true` to constrain the model's
output toward the dataset schema. This usually reduces the fraction of invalid
records, typically at the cost of reducing the quality of the generated records.
Use it when the pipeline struggles to produce valid records.

=== "CLI"

```bash
safe-synthesizer run \
--generation__use_structured_generation true \
--generation__structured_generation__enabled true \
--data-source data.csv
```

Expand All @@ -885,19 +885,24 @@ records. Use it when the pipeline struggles to produce valid records.
synthesizer = (
SafeSynthesizer()
.with_data_source("data.csv")
.with_generate(use_structured_generation=True)
.with_generate(structured_generation={"enabled": True})
)
```

Pass `structured_generation` as a nested dict or
`StructuredGenerationParameters` object. Do not use a bare `enabled=True`
shortcut for structured generation.

=== "Config reference"

```yaml
generation:
use_structured_generation: true
structured_generation_schema_method: "auto"
structured_generation:
enabled: true
schema_method: "auto"
```

- `"auto"`: picks `"structural_tag"` when `structured_generation_backend` is `"auto"` or `"xgrammar"`, otherwise `"regex"`.
- `"auto"`: picks `"structural_tag"` when `structured_generation.backend` is `"auto"` or `"xgrammar"`, otherwise `"regex"`.
- `"structural_tag"`: uses XGrammar Structural Tag to compose schema-constrained JSONL output.
- `"regex"`: constructs a custom regex from the dataset schema. More comprehensive but slower.
- `"json_schema"`: passes a JSON Schema to the backend. Faster, but may miss edge cases.
Expand Down Expand Up @@ -941,8 +946,9 @@ fraction per batch.

!!! tip "Early stopping"
If the pipeline stops early due to patience, try enabling
`use_structured_generation: true` to constrain outputs to the dataset
schema, or lower `temperature` to reduce the chance of malformed records.
`generation.structured_generation.enabled: true` to constrain outputs to
the dataset schema, or lower `temperature` to reduce the chance of malformed
records.

See [Configuration Reference -- Generation](configuration.md#generation) for the full parameter table.

Expand Down
2 changes: 1 addition & 1 deletion docs/user-guide/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ Generation stopped prematurely because the average fraction of invalid records w
```

: Too many invalid records across `generation.patience` consecutive batches.
Consider retraining with more records, adjusting `training.number_of_input_records_to_sample`, or setting `use_structured_generation=True`.
Consider retraining with more records, adjusting `training.number_of_input_records_to_sample`, or setting `generation.structured_generation.enabled=true`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

For context-length errors during data assembly (`"The number of tokens in an
example exceeds the available context length"`), see
Expand Down
3 changes: 2 additions & 1 deletion src/nemo_safe_synthesizer/config/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from .differential_privacy import DifferentialPrivacyHyperparams
from .evaluate import EvaluationParameters
from .external_results import SafeSynthesizerSummary, SafeSynthesizerTiming
from .generate import GenerateParameters
from .generate import GenerateParameters, StructuredGenerationParameters
from .internal_results import SafeSynthesizerResults
from .job import SafeSynthesizerJobConfig
from .parameters import SafeSynthesizerParameters
Expand All @@ -31,6 +31,7 @@
"SafeSynthesizerResults",
"SafeSynthesizerSummary",
"SafeSynthesizerTiming",
"StructuredGenerationParameters",
"TimeSeriesParameters",
"TrainingHyperparams",
]
Loading
Loading