Skip to content

Full-rate single-ID capture: hardware acceptance filter (web-stream captures are RX-queue decimated) #104

@hypery11

Description

@hypery11

Problem

Community web-stream captures come out downsampled to ~2 Hz per CAN ID. Measured on @DmitroPanteliuk's pin 9/10 captures in #95: 0x129 steering logs at ~10 Hz when the real bus runs it near 100 Hz, and the whole capture lands at ~226 fps when Vehicle CAN carries 2000+ fps. 0x229 lands on a near-perfectly-uniform 0.53s interval, which is a sampling artifact, not natural bus timing.

This blocks the reverse-engineering work that needs every frame:

  • Rolling counter + CRC alias to noise. Consecutive bus counters are missing between samples, so a CRC brute-force can never lock on. This is the real reason 0x229 (SCCM_rightStalk, the native AP-engage stalk) won't crack, and it applies to 0x485 and any counter+CRC id.
  • A stalk/button press shorter than ~0.5s can fall entirely between two samples and never get recorded.

Root cause

It is not the HTTP stream layer. http_can_stream.cpp is full-rate: 3072-frame ring, 160 frames flushed per update, every matching frame recorded.

It is the TWAI RX path. can_driver.cpp:

g.rx_queue_len = 10;                                       // only 10 frames
twai_filter_config_t f = TWAI_FILTER_CONFIG_ACCEPT_ALL();  // accept everything

A 10-frame RX queue with accept-all, on a ~2500 fps bus, overflows between loop iterations (the loop also does WiFi, display, status). rx_missed_count climbs and the controller drops most frames before software ever sees them. The frames that survive are effectively a low-rate sample.

Fix

  1. Hardware acceptance filter for single-ID capture. When the stream is opened with exactly one ?ids= value, reconfigure the TWAI acceptance filter to match only that id. Then the RX queue only ever holds that id (~2-10 fps), never overflows, and every frame is delivered at true full rate. Restore accept-all when the stream ends. Safe to reconfigure mid-run because the stream only runs in Listen-Only mode.
  2. Bump rx_queue_len (10 -> 64) for the general/multi-id case. Pure win, just more RAM.

Scope and limits

  • TWAI first: covers m5stack-atom, esp32-lilygo, waveshare-s3-can, and the can0 side of lilygo-t2can, and it is the reporter's rig (ESP32 + TWAI rx 4 / tx 5).
  • MCP2515 boards (esp32-mcp2515, ttgo-tdisplay, t2can can1) have only 2 RX buffers and benefit even more, but the autowp setFilter path needs its own work plus on-car validation. Follow-up.
  • During a single-ID capture session, the passive readouts (BMS, OTA detect, HW auto-detect) pause for that session by design, since the controller is only accepting the one id. Restored when the stream closes.

Validation

No host harness exists for the driver, so this needs an on-car test. @DmitroPanteliuk runs 2026.14.6 Intel HW3 with an ESP32/TWAI rig and surfaced the issue, so that is the ideal validation path before release.

Capture recipe once shipped

http://<board-ip>:82/stream?ids=229

Idle 3s, one pull-down (TACC), idle 3s, double pull-down (AP), idle 3s. With every 0x229 frame preserved the counter increments cleanly (crack the CRC8 off the idle stretch) and the stalk position field shows its discrete values.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions