Skip to content

Commit 3265e65

Browse files
fix(desktop): detect Telegram calls and gate meeting treatment at finalization (#11832)
## What and why - Add Telegram Desktop's shipping bundle ID (`com.tdesktop.Telegram`, normalized to lowercase) and legacy `ru.keepcoder.telegram` ID to native call detection. The proactive chat-app policy consumes the same Telegram ID set so the catalogs cannot drift independently. - Keep `conversation_role` as OPEN-time session provenance. Introduce a separate backend-owned `meeting_treatment_eligible` finalization result, persist it with the durable finalization job, and expose it through both the finalization-status and synchronous from-segments responses. - Gate the meeting Chat arrival and desktop completion notification on that finalization result. Short or mostly silent calls still finalize as ordinary conversations without rewriting their role or provenance. - Count actual speech as the union of non-empty transcript intervals, preventing simultaneous microphone and system-audio transcript twins from roughly doubling the speech measure. Meeting treatment requires at least five minutes of wall-clock duration and at least 60 seconds of deduplicated transcribed speech. Five minutes filters quick calls that should remain ordinary conversations. The smaller speech floor still admits listening-heavy meetings and natural pauses while rejecting long hold/silence captures with only a brief accidental transcription. Interval union makes the floor independent of whether one or both desktop audio streams transcribed the same remote speech. ## Line-count exception Line-Count-Exception: backend/routers/developer.py | 2111 -> 2115 | propagates the new meeting_treatment_eligible field through the existing conversation response model and its two projection sites; splitting this router is unrelated to this fix and would make the change unreviewable. ## Failure class Failure-Class: none No registered failure class matches this macOS native-process catalog omission or the separate finalization-time treatment boundary. `FC-meeting-trigger-title-identity-drift` is a Windows browser-title/event identity contract. The production `isNativeCallApp` regression and shared Telegram ID set guard the catalog boundary; backend behavioral tests guard duration, deduplicated speech, finalization persistence, Chat arrival, and response projection. ## Verification - `PYTHON=/Users/dazheng/workspace/omi/upstream-keep-clean/backend/.venv/bin/python bash test-preflight.sh` (from `backend/`) — 17 passed, 9 optional warnings, 0 failed. - `PYTHON=/Users/dazheng/workspace/omi/upstream-keep-clean/backend/.venv/bin/python BACKEND_PYTEST_WORKERS=6 BACKEND_UNIT_TEST_FILE_LIST=../.backend-meeting-test-files.txt bash test.sh` (from `backend/`; the temporary selector listed the six changed/new backend test files and was then deleted) — 195 passed, including the four dedicated meeting-treatment policy tests. - `PYTHON=/Users/dazheng/workspace/omi/upstream-keep-clean/backend/.venv/bin/python BACKEND_PYTEST_WORKERS=10 bash test.sh` (from `backend/`) — all 856 backend unit-test files were selected and the runner progressed through nearly the entire suite, but one remaining file produced no output for several minutes and the file-isolated runner has no per-file timeout. The run was interrupted and did not produce a terminal pass/fail summary; full-suite status is therefore unverified. - `./scripts/run-swift-ci.sh --test` (from `desktop/macos/`) — could not start because the pinned `/Applications/Xcode_16.4.app` toolchain is unavailable in this environment. - `CLANG_MODULE_CACHE_PATH=/Volumes/scratch/tmp/meeting-detect-swift-cache/clang SWIFTPM_MODULECACHE_OVERRIDE=/Volumes/scratch/tmp/meeting-detect-swift-cache/clang xcrun swift test --jobs 1 -Xswiftc -j1 --disable-index-store --disable-sandbox --skip-update --disable-build-manifest-caching --manifest-cache none --cache-path /Volumes/scratch/tmp/meeting-detect-swift-cache/swiftpm-cache --config-path /Volumes/scratch/tmp/meeting-detect-swift-cache/swiftpm-config --security-path /Volumes/scratch/tmp/meeting-detect-swift-cache/swiftpm-security --scratch-path /private/tmp/meeting-swift-build --package-path desktop/macos/Desktop --filter 'ConferencingAppsTests|TranscriptionFinalizationStateMachineTests|ProactiveAssistantOrchestrationPolicyTests|APIClientConversationCountTests'` — SwiftPM reached the debug build and then failed with opaque `error: fatalError`; tests did not execute. - `CLANG_MODULE_CACHE_PATH=/Volumes/scratch/tmp/meeting-detect-swift-cache/harness-clang SWIFTPM_MODULECACHE_OVERRIDE=/Volumes/scratch/tmp/meeting-detect-swift-cache/harness-clang xcrun swiftc -module-cache-path /Volumes/scratch/tmp/meeting-detect-swift-cache/harness-clang desktop/macos/Desktop/Sources/ConferencingApps.swift .meeting-detect-catalog-harness.swift -o /private/tmp/meeting-detect-catalog-harness && /private/tmp/meeting-detect-catalog-harness` — directly compiled the production classifier; both Telegram IDs classified as native call apps and Omi did not. Printed `Telegram catalog classification passed`; the temporary source was deleted. - `python3 scripts/check_desktop_test_quality.py` (from `desktop/macos/`) — passed with no new source-reading tests, trapping dictionary initializers, or wall-clock waits. - `scripts/swift-format-wrapper.sh lint <nine changed Swift files>` (from `desktop/macos/`) — passed. - `make preflight` — all 11 manifest checks passed. Because the changes were still uncommitted, the preflight runner reported `files=0`; rerun after the commit so diff-scoped selection is meaningful. - `git diff --check` — passed. ## Verification limits - A real Telegram call and named desktop bundle were not exercised because the pinned desktop toolchain is unavailable and SwiftPM cannot complete whole-module emission with the available toolchain. - The full backend suite did not reach a terminal summary; the changed-area component run is green, but the remaining backend files are not claimed as verified. ## Follow-ups - Add Tier-2 detection for unknown CoreAudio processes only after a telemetry-only phase builds an observed allowlist from real process topologies. `isRunningOutput` means an active output stream, not audible speech; games, DAWs, and media apps can satisfy it, while split-process Electron/browser calls can fail a same-process input+output rule. - Design idle-based conversation rotation through the existing serialized meeting-boundary path. It must never become a second rotation authority: a failed meeting rotation can call `stopTranscription()`, so an independent idle race could stop always-on capture. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/BasedHardware/omi/pull/11832?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
2 parents 0d23273 + e5be633 commit 3265e65

31 files changed

Lines changed: 480 additions & 78 deletions

backend/database/conversation_finalization_jobs.py

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -731,6 +731,7 @@ def _mark_finalization_fanout_completed_txn(
731731
dispatch_generation: int,
732732
lease_epoch: int,
733733
now: datetime,
734+
meeting_treatment_eligible: bool,
734735
) -> bool:
735736
snapshot = job_ref.get(transaction=transaction)
736737
if not getattr(snapshot, 'exists', False):
@@ -745,6 +746,7 @@ def _mark_finalization_fanout_completed_txn(
745746
{
746747
'fanout_status': 'completed',
747748
'fanout_completed_at': now,
749+
'meeting_treatment_eligible': meeting_treatment_eligible,
748750
'updated_at': now,
749751
},
750752
)
@@ -756,12 +758,20 @@ def mark_finalization_fanout_completed(
756758
dispatch_generation: int,
757759
lease_epoch: int,
758760
*,
761+
meeting_treatment_eligible: bool,
759762
firestore_client: Any = None,
760763
) -> bool:
761764
client = _client(firestore_client)
762765
transaction = client.transaction()
763766
transactional = firestore.transactional(_mark_finalization_fanout_completed_txn)
764-
return transactional(transaction, _job_ref(client, job_id), dispatch_generation, lease_epoch, _now())
767+
return transactional(
768+
transaction,
769+
_job_ref(client, job_id),
770+
dispatch_generation,
771+
lease_epoch,
772+
_now(),
773+
meeting_treatment_eligible,
774+
)
765775

766776

767777
def _mark_finalization_retryable_txn(

backend/models/conversation.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -348,6 +348,7 @@ class ConversationFinalizationStatusResponse(BaseModel):
348348
retryable: bool
349349
attempt_count: int
350350
task_retry_count: int
351+
meeting_treatment_eligible: bool = False
351352

352353

353354
# MIGRATE: For backward compatibility with the old memories routes and app

backend/routers/developer.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@
5656
from utils.conversations.process_conversation import process_conversation
5757
from utils.conversations import lifecycle as lifecycle_service
5858
from utils.conversations.location import resolve_geolocation
59+
from utils.conversations.meeting_treatment import is_meeting_treatment_eligible
5960
from utils.executors import postprocess_executor
6061
from utils.request_validation import HistoryDays
6162
from utils.llm.memories import identify_category_for_memory
@@ -1084,6 +1085,7 @@ class ConversationResponse(BaseModel):
10841085
id: str
10851086
status: str
10861087
discarded: bool
1088+
meeting_treatment_eligible: bool = False
10871089

10881090

10891091
class UpdateConversationRequest(BaseModel):
@@ -1474,6 +1476,7 @@ def _conversation_response_from_data(conversation: dict) -> ConversationResponse
14741476
id=conversation['id'],
14751477
status=status,
14761478
discarded=bool(conversation.get('discarded', False)),
1479+
meeting_treatment_eligible=is_meeting_treatment_eligible(conversation),
14771480
)
14781481

14791482

@@ -1670,6 +1673,7 @@ def _create_conversation_from_segments(
16701673
id=conversation.id,
16711674
status=conversation.status.value if conversation.status else 'completed',
16721675
discarded=conversation.discarded,
1676+
meeting_treatment_eligible=is_meeting_treatment_eligible(conversation),
16731677
)
16741678

16751679

backend/tests/unit/test_chat_first_proactive_engine.py

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
"""Failure-isolated, content-free proactive-judgment contracts."""
22

3-
from datetime import datetime, timezone
3+
from datetime import datetime, timedelta, timezone
44
import logging
55
from types import SimpleNamespace
66
from unittest.mock import MagicMock
@@ -15,6 +15,12 @@
1515
QuestionOption,
1616
)
1717
from utils.task_intelligence.chat_first_eligibility import ChatFirstEligibility
18+
from utils.conversations.meeting_treatment import (
19+
MIN_MEETING_DURATION_SECONDS,
20+
MIN_TRANSCRIBED_SPEECH_SECONDS,
21+
deduplicated_transcribed_speech_seconds,
22+
is_meeting_treatment_eligible,
23+
)
1824

1925
NOW = datetime(2026, 7, 15, 12, tzinfo=timezone.utc)
2026
SUBJECT = ChatFirstSubject(kind='goal', id='goal-1')
@@ -223,6 +229,9 @@ def test_desktop_meeting_adapter_uses_stored_role_and_skips_non_meeting_or_rotat
223229
'source': 'desktop',
224230
'status': 'completed',
225231
'discarded': False,
232+
'started_at': NOW,
233+
'finished_at': NOW + timedelta(seconds=MIN_MEETING_DURATION_SECONDS),
234+
'transcript_segments': [{'text': 'A substantive exchange', 'start': 0, 'end': MIN_TRANSCRIBED_SPEECH_SECONDS}],
226235
'structured': {'title': 'Ambient capture'},
227236
'external_data': {'conversation_role': 'ambient'},
228237
}
@@ -254,6 +263,35 @@ def test_desktop_meeting_adapter_uses_stored_role_and_skips_non_meeting_or_rotat
254263
persist.assert_not_called()
255264

256265

266+
def test_meeting_treatment_requires_five_minutes_and_deduplicated_speech():
267+
eligible = {
268+
'source': 'desktop',
269+
'discarded': False,
270+
'started_at': NOW,
271+
'finished_at': NOW + timedelta(seconds=MIN_MEETING_DURATION_SECONDS),
272+
'external_data': {'conversation_role': 'meeting'},
273+
'transcript_segments': [
274+
{'text': 'first exchange', 'start': 0, 'end': 35},
275+
{'text': 'second exchange', 'start': 35, 'end': MIN_TRANSCRIBED_SPEECH_SECONDS},
276+
],
277+
}
278+
assert is_meeting_treatment_eligible(eligible) is True
279+
280+
short_call = {**eligible, 'finished_at': NOW + timedelta(seconds=MIN_MEETING_DURATION_SECONDS - 1)}
281+
assert is_meeting_treatment_eligible(short_call) is False
282+
283+
duplicate_streams = {
284+
**eligible,
285+
'finished_at': NOW + timedelta(minutes=20),
286+
'transcript_segments': [
287+
{'text': 'remote stream from mic', 'start': 0, 'end': 45},
288+
{'text': 'same remote stream from system audio', 'start': 0, 'end': 45},
289+
],
290+
}
291+
assert deduplicated_transcribed_speech_seconds(duplicate_streams['transcript_segments']) == 45
292+
assert is_meeting_treatment_eligible(duplicate_streams) is False
293+
294+
257295
def test_proactive_failure_logs_redact_authenticated_uid(monkeypatch, caplog):
258296
uid = 'sensitive-user-123456'
259297
monkeypatch.setattr(

backend/tests/unit/test_conversation_finalization_jobs.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -493,7 +493,8 @@ def test_finalization_completion_requires_durable_fanout_completion():
493493
ref.data = ref.data | fanout.updates[0][1]
494494

495495
completed_fanout = _Transaction()
496-
assert jobs._mark_finalization_fanout_completed_txn(completed_fanout, ref, 1, 4, now) is True
496+
assert jobs._mark_finalization_fanout_completed_txn(completed_fanout, ref, 1, 4, now, True) is True
497+
assert completed_fanout.updates[0][1]['meeting_treatment_eligible'] is True
497498
ref.data = ref.data | completed_fanout.updates[0][1]
498499

499500
completed = _Transaction()

backend/tests/unit/test_conversation_search_date_validation.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -696,6 +696,7 @@ def test_finalization_status_endpoint_exposes_retryable_durable_state():
696696
'retryable': True,
697697
'attempt_count': 2,
698698
'task_retry_count': 1,
699+
'meeting_treatment_eligible': False,
699700
}
700701
with (
701702
patch.object(conv.conversations_db, 'get_conversation', return_value={'id': 'conv-1'}),

backend/tests/unit/test_developer_from_segments_idempotency.py

Lines changed: 56 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,25 @@ def _request(**overrides):
173173
return developer.CreateConversationFromTranscriptRequest.model_validate(data)
174174

175175

176+
def _eligible_meeting_request(**overrides):
177+
data = {
178+
'transcript_segments': [
179+
{
180+
'text': 'substantive meeting discussion',
181+
'speaker': 'SPEAKER_00',
182+
'is_user': True,
183+
'start': 0.0,
184+
'end': 60.0,
185+
}
186+
],
187+
'started_at': NOW,
188+
'finished_at': NOW + timedelta(minutes=5),
189+
'conversation_role': 'meeting',
190+
}
191+
data.update(overrides)
192+
return _request(**data)
193+
194+
176195
def test_no_client_session_id_preserves_create_conversation_path(monkeypatch):
177196
captured = {}
178197

@@ -200,7 +219,7 @@ def _process(uid, language, conversation):
200219

201220
arrival = MagicMock()
202221
monkeypatch.setattr(proactive_engine, 'persist_capture_arrival_intent', arrival)
203-
response = developer._create_conversation_from_segments('uid1', _request(conversation_role='meeting'))
222+
response = developer._create_conversation_from_segments('uid1', _eligible_meeting_request())
204223

205224
assert response.id == 'random-process-id'
206225
assert isinstance(captured['conversation'], CreateConversation)
@@ -281,10 +300,11 @@ def _process(_uid, _language, conversation):
281300
monkeypatch.setattr(proactive_engine, 'persist_capture_arrival_intent', arrival)
282301

283302
response = developer._create_conversation_from_segments(
284-
'uid1', _request(client_session_id='meeting-session-1', conversation_role='meeting')
303+
'uid1', _eligible_meeting_request(client_session_id='meeting-session-1')
285304
)
286305

287306
assert response.id == expected_id
307+
assert response.meeting_treatment_eligible is True
288308
arrival.assert_called_once_with(
289309
'uid1', conversation_id=expected_id, summary='Design review', is_desktop_meeting=True
290310
)
@@ -342,6 +362,9 @@ def test_completed_desktop_meeting_retry_repairs_missing_arrival(monkeypatch):
342362
'source': 'desktop',
343363
'status': 'completed',
344364
'discarded': False,
365+
'started_at': NOW,
366+
'finished_at': NOW + timedelta(minutes=5),
367+
'transcript_segments': [{'text': 'substantive meeting discussion', 'start': 0.0, 'end': 60.0}],
345368
'structured': {'title': 'Design review'},
346369
'external_data': {'conversation_role': 'meeting'},
347370
}
@@ -353,16 +376,46 @@ def test_completed_desktop_meeting_retry_repairs_missing_arrival(monkeypatch):
353376
monkeypatch.setattr(proactive_engine, 'persist_capture_arrival_intent', arrival)
354377

355378
response = developer._create_conversation_from_segments(
356-
'uid1', _request(client_session_id='meeting-session-1', conversation_role='meeting')
379+
'uid1', _eligible_meeting_request(client_session_id='meeting-session-1')
357380
)
358381

359382
assert response.id == expected_id
383+
assert response.meeting_treatment_eligible is True
360384
process.assert_not_called()
361385
arrival.assert_called_once_with(
362386
'uid1', conversation_id=expected_id, summary='Design review', is_desktop_meeting=True
363387
)
364388

365389

390+
def test_short_desktop_meeting_stays_ordinary_conversation(monkeypatch):
391+
monkeypatch.setattr(conversations_db, 'get_conversation', MagicMock())
392+
monkeypatch.setattr(developer.lifecycle_service, 'create_processing_conversation', MagicMock())
393+
394+
def _process(_uid, _language, conversation):
395+
return Conversation(
396+
id='short-meeting',
397+
created_at=NOW,
398+
started_at=conversation.started_at,
399+
finished_at=conversation.finished_at,
400+
source=conversation.source,
401+
language=conversation.language,
402+
structured={'title': 'Short call'},
403+
transcript_segments=conversation.transcript_segments,
404+
external_data=conversation.external_data,
405+
status=ConversationStatus.completed,
406+
)
407+
408+
monkeypatch.setattr(developer, 'process_conversation', _process)
409+
arrival = MagicMock()
410+
monkeypatch.setattr(proactive_engine, 'persist_capture_arrival_intent', arrival)
411+
412+
response = developer._create_conversation_from_segments('uid1', _request(conversation_role='meeting'))
413+
414+
assert response.status == 'completed'
415+
assert response.meeting_treatment_eligible is False
416+
arrival.assert_not_called()
417+
418+
366419
def test_completed_ambient_retry_cannot_reclassify_conversation_as_meeting(monkeypatch):
367420
expected_id = developer._from_segments_conversation_id('uid1', 'ambient-session-1')
368421
monkeypatch.setattr(

backend/tests/unit/test_listen_finalization_cloud_tasks.py

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44

55
import asyncio
66
import json
7+
from datetime import datetime, timedelta, timezone
78
from pathlib import Path
89
import runpy
910
from types import SimpleNamespace
@@ -315,17 +316,20 @@ def test_finalization_status_exposes_retry_and_terminal_state(monkeypatch):
315316
'retryable': True,
316317
'attempt_count': 2,
317318
'task_retry_count': 0,
319+
'meeting_treatment_eligible': False,
318320
}
319321

320322
job['status'] = 'dead_letter'
321323
job['task_retry_count'] = 3
324+
job['meeting_treatment_eligible'] = True
322325
assert lifecycle_service.get_finalization_status('uid-1', 'conversation-1') == {
323326
'job_id': 'job-1',
324327
'status': 'dead_letter',
325328
'terminal': True,
326329
'retryable': False,
327330
'attempt_count': 2,
328331
'task_retry_count': 3,
332+
'meeting_treatment_eligible': True,
329333
}
330334

331335

@@ -1054,6 +1058,9 @@ async def inline_run_blocking(_executor, func, *args, **kwargs):
10541058
source=SimpleNamespace(value=source),
10551059
external_data=external_data,
10561060
discarded=discarded,
1061+
started_at=datetime(2026, 8, 18, 12, tzinfo=timezone.utc),
1062+
finished_at=datetime(2026, 8, 18, 12, tzinfo=timezone.utc) + timedelta(minutes=10),
1063+
transcript_segments=[SimpleNamespace(text='substantive exchange', start=0, end=60)],
10571064
structured=SimpleNamespace(title='Captured title', overview='Captured overview'),
10581065
)
10591066
integrations = AsyncMock(return_value=[])
@@ -1097,7 +1104,12 @@ async def inline_run_blocking(_executor, func, *args, **kwargs):
10971104
else:
10981105
extracted.assert_called_once_with('uid-1', conversation)
10991106
assert disposition == ConversationFinalizationDisposition.completed
1100-
completed.assert_called_once_with('job-1', 2, 3)
1107+
completed.assert_called_once_with(
1108+
'job-1',
1109+
2,
1110+
3,
1111+
meeting_treatment_eligible=(source == 'desktop' and expected_intent_kwargs is not None),
1112+
)
11011113
if expected_intent_kwargs is None:
11021114
capture_arrival.assert_not_called()
11031115
else:
@@ -1440,7 +1452,7 @@ def contract_faithful_process(_uid, _lang, conv, **kwargs):
14401452
assert disposition == ConversationFinalizationDisposition.completed
14411453
derived_runner.assert_called_once()
14421454
integrations.assert_awaited_once()
1443-
complete.assert_called_once_with('job-1', 2, 3)
1455+
complete.assert_called_once_with('job-1', 2, 3, meeting_treatment_eligible=False)
14441456

14451457

14461458
@pytest.mark.anyio
@@ -1520,7 +1532,7 @@ def contract_faithful_process(_uid, _lang, conv, **kwargs):
15201532
)
15211533

15221534
assert disposition == ConversationFinalizationDisposition.completed
1523-
complete.assert_called_once_with('job-1', 2, 3)
1535+
complete.assert_called_once_with('job-1', 2, 3, meeting_treatment_eligible=False)
15241536
safe_target.assert_called_once_with('https://app.test/hook?uid=uid-1')
15251537
webhook_client.post.assert_awaited_once_with(
15261538
pinned_url,

0 commit comments

Comments
 (0)