Operations

Twenty operations: sixteen routed to AuK, four performed locally in numpy.

This page is generated from src/openspeech/ops/catalog.py by docs/generate_operations.py, so it cannot drift from what the code accepts. Regenerate it after changing the catalogue.

Machine-readable versions of the same table:

openspeech ops --json          # CLI
curl localhost:8000/v1/ops     # service

Reading the columns

Duration policy - how the target length follows from the source. AuK needs an explicit length for anything that is not "same as the source", and gets visibly worse when the hint is wrong.
Chunking - whether the operation survives being split across a long recording. safe chunks independently; anchored needs the chunk holding a specific phrase, so it refuses rather than guessing; global depends on the whole recording and warns; none generates from scratch.
Source budget - longest source clip that leaves room for this operation's output inside AuK's ~30 s source-plus-target window. See long-form.

Generation

Produce speech. instruct_tts needs no source audio.

`instruct_tts`

Synthesise speech from a written voice description, with no reference audio.


Runs on	AuK model
Needs source audio	no
Duration policy	`text_estimated`
Chunking	`none`
Upstream reference	AuK docs/COOKBOOK.md 1.2

Parameters

Name	Type	Required	Values	Description
`text`	string	yes	-	The words to speak.
`style_desc`	string	yes	-	Voice and delivery description.

Instruction sent to the model

en: Generate speech based on the following description: "a warm announcer". The content to speak is: "Welcome to the show".
zh: 请基于下面的描述: "a warm announcer",生成语音内容"Welcome to the show".

`zero_shot_tts`

Speak new text in the voice of a reference recording.


Runs on	AuK model
Needs source audio	yes
Duration policy	`text_estimated`
Chunking	`none`
Upstream reference	AuK docs/COOKBOOK.md 1.1

Parameters

Name	Type	Required	Values	Description
`text`	string	yes	-	The words to speak.

Instruction sent to the model

en: Say the following with the same voice: "Welcome to the show"
zh: Say the following with the same voice: "Welcome to the show"

Restoration

Repair what the recording did to the voice.

`enhance_speech`

Remove noise and/or room reverberation, keeping every speaker and the length.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`safe`
Source budget	13.95 s
Upstream reference	AuK docs/COOKBOOK.md 5.1

Parameters

Name	Type	Required	Values	Description
`cleanup_mode`	string	no	`denoise`, `dereverb`, `denoise_dereverb` (default `denoise_dereverb`)	Which degradation to remove.

Instruction sent to the model

en: Preserve all speakers, remove noise and reverberation, and output clean speech of the same length.
zh: 请保留所有说话人，去除噪声和混响，输出等长的纯净语音。

`improve_quality`

Repair a specific degradation - telephone band, muffling, clipping, dropouts.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`safe`
Source budget	13.95 s
Upstream reference	AuK docs/COOKBOOK.md 5.1

Parameters

Name	Type	Required	Values	Description
`effect`	string	no	`telephone effect`, `muffling`, `clipping`, `dropouts` (default `muffling`)	The degradation to repair.

Instruction sent to the model

en: Repair the telephone effect and restore natural, clear speech.
zh: 请修复这段音频的电话感，恢复自然清晰的人声。

Separation

Decide whose voice survives.

`extract_vocals`

Pull the voice out of a mix, either singing only or every human voice.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`safe`
Source budget	13.95 s
Subtypes	`singing_only`, `all_human_voices` (default)
Upstream reference	AuK docs/COOKBOOK.md 5.3

Instruction sent to the model

en: Keep all human voices, including speech and singing, and remove everything else.
zh: 请保留所有人声，包括说话和歌唱，其余声音都去掉。

`separate_speech`

Keep one speaker and drop the others, by talking order or by what they said.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`global`
Source budget	13.95 s
Subtypes	`by_order` (default), `by_content`
Upstream reference	AuK docs/COOKBOOK.md 5.2, 5.4

Parameters

Name	Type	Required	Values	Description
`n`	int	for `by_order`	-default `1`	1-based talking order of the speaker to keep.
`text`	string	for `by_content`	-	Something only the target speaker says.

Instruction sent to the model

en: Keep only the second speaker to start talking and remove all other speakers.
zh: 只保留第二个开始说话的人，去掉其余说话人。

Content

Change what is said.

`content_edit`

Change what is said - replace, insert or delete words - keeping the voice.


Runs on	AuK model
Needs source audio	yes
Duration policy	`content_delta`
Chunking	`anchored`
Source budget	12.13 s
Subtypes	`replace` (default), `insert_before`, `insert_after`, `delete`, `delete_before`, `delete_after`
Upstream reference	AuK docs/COOKBOOK.md 2.1

Parameters

Name	Type	Required	Values	Description
`orig`	string	for `replace`	-	Words to replace.
`new`	string	for `replace`	-	Replacement words.
`text`	string	for `insert_before`/`insert_after`/`delete`/`delete_before`/`delete_after`	-	Words to insert or delete.
`anchor`	string	for `insert_before`/`insert_after`/`delete_before`/`delete_after`	-	Existing phrase the edit is positioned against.

Instruction sent to the model

en: Replace 'quarterly' with 'annual'.
zh: 把‘quarterly’改成‘annual’

`nonverbal_edit`

Add or remove breaths, laughs, coughs and other nonverbal sounds.


Runs on	AuK model
Needs source audio	yes
Duration policy	`shrinks`
Chunking	`anchored`
Source budget	13.29 s
Subtypes	`delete` (default), `add_head`, `add_tail`
Upstream reference	AuK docs/COOKBOOK.md 4.4

Parameters

Name	Type	Required	Values	Description
`sound`	string	yes	-	The nonverbal sound, e.g. breaths, laughs, coughs.

Instruction sent to the model

en: Remove all breaths from the audio.
zh: 删除音频中所有的breaths。

`vocal_edit`

Rewrite lyrics in a singing recording, keeping melody and voice.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`anchored`
Source budget	13.95 s
Upstream reference	AuK docs/COOKBOOK.md 2.2

Parameters

Name	Type	Required	Values	Description
`orig`	string	yes	-	Original lyrics.
`new`	string	yes	-	New lyrics.

Instruction sent to the model

en: Change "rear view" to "like you" in the vocal recording.
zh: 把这段歌词中的“rear view”改成“like you”。

Paralinguistic

Change how it is said, keeping the words.

`accent_edit`

Neutralise a regional accent, keeping the speaker's voice and words.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`safe`
Source budget	13.95 s
Upstream reference	AuK docs/COOKBOOK.md 4.3

Instruction sent to the model

en: Remove the regional accent while preserving the speaker's voice and content.
zh: 请去掉这段语音里的方言口音，保持说话人音色一致。

`emotion_edit`

Change the emotional delivery, keeping words and voice.


Runs on	AuK model
Needs source audio	yes
Duration policy	`emotion_scaled`
Chunking	`safe`
Source budget	12.57 s
Upstream reference	AuK docs/COOKBOOK.md 4.1

Parameters

Name	Type	Required	Values	Description
`emotion`	string	yes	`happy`, `angry`, `sad`, `fearful`, `surprised`, `disgusted`, `calm`, `excited`	Target emotion.

Instruction sent to the model

en: Change the emotion to calm.
zh: 将情感转变为平静。

`voice_edit`

Change the timbre to a written description, keeping the words.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`safe`
Source budget	13.95 s
Upstream reference	AuK docs/COOKBOOK.md 4.2

Parameters

Name	Type	Required	Values	Description
`timbre_desc`	string	yes	-	Target timbre description.

Instruction sent to the model

en: Keep the spoken content unchanged and change the timbre to: "a deep, warm voice".
zh: 请将这段音频的音色修改为符合以下描述的声音：“a deep, warm voice”。

`whisper_edit`

Convert between whispered and normally voiced speech.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`safe`
Source budget	13.95 s
Subtypes	`to_whisper`, `to_normal` (default)
Upstream reference	AuK docs/COOKBOOK.md 4.5

Instruction sent to the model

en: Convert this whispered speech into a normal speaking voice while preserving the speaker and content.
zh: 把这段耳语转换成正常说话的声音。

Acoustic

Mechanical adjustments to pitch, rate and level.

`pitch_edit`

Shift pitch by whole semitones.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`safe`
Source budget	13.95 s
Subtypes	`increase` (default), `decrease`
Upstream reference	AuK src/auk/infer/pe.config.yaml pitch_edit

Parameters

Name	Type	Required	Values	Description
`semitones`	int	yes	`1`, `2`, `3`	Semitones to shift, 1-3.

Instruction sent to the model

en: Raise the pitch by 2 semitones.
zh: 将音调升高2个半音。

`speed_edit`

Change speaking rate without changing pitch.


Runs on	AuK model
Needs source audio	yes
Duration policy	`speed_scaled`
Chunking	`safe`
Source budget	16.74 s
Upstream reference	AuK src/auk/infer/pe.config.yaml speed_edit

Parameters

Name	Type	Required	Values	Description
`speed_multiplier`	float	yes	`0.5`, `0.75`, `1.25`, `1.5`, `2.0`	Rate multiplier.

Instruction sent to the model

en: Adjust the speech speed to 1.5x.
zh: 将语速调整为1.5倍。

`volume_edit`

Change level by whole decibels.


Runs on	AuK model
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`safe`
Source budget	13.95 s
Subtypes	`increase` (default), `decrease`
Upstream reference	AuK src/auk/infer/pe.config.yaml volume_edit

Parameters

Name	Type	Required	Values	Description
`gain_db`	int	yes	`5`, `10`, `15`	Decibels of change, 5/10/15.

Instruction sent to the model

en: Increase the volume by 10 dB.
zh: 将音量升高10分贝。

Mastering

Local DSP. Exact, instant, no GPU time, no context window.

`fade`

Apply fades at the start and end.


Runs on	local DSP
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`global`

Parameters

Name	Type	Required	Values	Description
`fade_in`	float	no	-default `0.0`	Fade-in seconds.
`fade_out`	float	no	-default `0.0`	Fade-out seconds.

`loudness_normalize`

Normalise to a LUFS target with a true-peak ceiling (EBU R128).


Runs on	local DSP
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`global`
Upstream reference	ITU-R BS.1770-4 / EBU R128

Parameters

Name	Type	Required	Values	Description
`target_lufs`	float	no	-default `-16.0`	Integrated loudness target.
`true_peak_db`	float	no	-default `-1.0`	True-peak ceiling in dBTP.

`resample`

Convert to a different sample rate.


Runs on	local DSP
Needs source audio	yes
Duration policy	`equal_length`
Chunking	`global`

Parameters

Name	Type	Required	Values	Description
`sample_rate`	int	yes	-	Target sample rate in Hz.

`trim_silence`

Trim leading and trailing silence.


Runs on	local DSP
Needs source audio	yes
Duration policy	`shrinks`
Chunking	`global`

Parameters

Name	Type	Required	Values	Description
`threshold_db`	float	no	-default `-45.0`	Silence threshold in dBFS.
`pad_seconds`	float	no	-default `0.08`	Padding kept around speech.

Navigation Menu

Repository navigation

operations.md

operations.md

Operations

Reading the columns

Generation

`instruct_tts`

`zero_shot_tts`

Restoration

`enhance_speech`

`improve_quality`

Separation

`extract_vocals`

`separate_speech`

Content

`content_edit`

`nonverbal_edit`

`vocal_edit`

Paralinguistic

`accent_edit`

`emotion_edit`

`voice_edit`

`whisper_edit`

Acoustic

`pitch_edit`

`speed_edit`

`volume_edit`

Mastering

`fade`

`loudness_normalize`

`resample`

`trim_silence`

FilesExpand file tree

operations.md

Latest commit

History

operations.md

File metadata and controls

Operations

Reading the columns

Generation

instruct_tts

zero_shot_tts

Restoration

enhance_speech

improve_quality

Separation

extract_vocals

separate_speech

Content

content_edit

nonverbal_edit

vocal_edit

Paralinguistic

accent_edit

emotion_edit

voice_edit

whisper_edit

Acoustic

pitch_edit

speed_edit

volume_edit

Mastering

fade

loudness_normalize

resample

trim_silence

`instruct_tts`

`zero_shot_tts`

`enhance_speech`

`improve_quality`

`extract_vocals`

`separate_speech`

`content_edit`

`nonverbal_edit`

`vocal_edit`

`accent_edit`

`emotion_edit`

`voice_edit`

`whisper_edit`

`pitch_edit`

`speed_edit`

`volume_edit`

`fade`

`loudness_normalize`

`resample`

`trim_silence`