Skip to content

Commit 9c15d36

Browse files
committed
docs: address resumability review feedback
1 parent 5a44147 commit 9c15d36

2 files changed

Lines changed: 4 additions & 2 deletions

File tree

fern/versions/main/pages/api-reference/tasks/audio-task.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ class AudioTask(Task[dict]):
2828

2929
dataset_name: str
3030
data: dict # _AttrDict subclass — supports attribute-style access
31-
# task_id is inherited from Task and is framework-owned.
31+
# task_id is inherited from Task with init=False.
3232
```
3333

3434
## Audio Manifest Format

fern/versions/main/pages/reference/infrastructure/resumable-processing.mdx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@ Keep checkpoint metadata until you have validated the output. After a successful
155155

156156
- `EmptyTask()` is a class instance with root ID `"0"`. It replaces the former shared `EmptyTask` singleton pattern.
157157
- Source outputs use `Task.get_deterministic_id()` when the task type implements it. `FileGroupTask` hashes its sorted paths, which keeps unchanged file groups stable if other groups are added or removed.
158-
- A custom source task's deterministic ID must be stable, unique within the source, and should not contain underscores. Resumability uses the final underscore-separated lineage segment as the source ID.
158+
- A custom source task's deterministic ID must be stable, unique within the source, and must not contain underscores. Resumability uses the final underscore-separated lineage segment as the source ID, so underscores can make distinct sources collide during checkpoint lookup.
159159
- Otherwise, a source output uses its position. Reordering positional sources can make a checkpoint refer to different work.
160160
- One-to-one results inherit their positional parent and append `0`.
161161
- A single input can fan out deterministically; child positions are appended.
@@ -197,7 +197,9 @@ from nemo_curator.tasks import FailedTask
197197

198198
def process(self, task: DocumentBatch) -> DocumentBatch | FailedTask:
199199
try:
200+
# Replace with the external call used by your stage.
200201
return call_external_service(task)
202+
# Replace TemporaryError with the recoverable exception raised by your service.
201203
except TemporaryError:
202204
return FailedTask()
203205
```

0 commit comments

Comments
 (0)