Merge pull request #396 from Integration-Automation/dev

Release: round-13 batch 4 (3 features, v178-v180)

Author: JE-Chen

README/WHATS_NEW_zh-CN.md

@@ -1,5 +1,23 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-24) — 匹配前安定门 + 命中稳定性
+
+避免在动画进行中匹配,并确认命中跨帧维持稳定。完整参考:[`docs/source/Zh/doc/new_features/v180_features_doc.rst`](../docs/source/Zh/doc/new_features/v180_features_doc.rst)。
+
+- **`region_stability` / `match_persistence`**(`AC_region_stability`、`AC_match_persistence`):`smart_waits.wait_until_screen_stable` 以布尔门控实时循环——无法对可注入帧序列评分稳定度,也无法检查某*命中*是否维持。`region_stability` 以相邻帧 SSIM 评分(`{stable, mean_ssim, min_ssim}`);`match_persistence` 确认 template 在*每一*帧都找到且中心于 `agree_px` 内一致(`{persisted, n_hits, jitter}`)。重用 `ssim` + `visual_match` + `grounding_consensus`;帧可注入;不导入 `PySide6`。
+
+## 本次更新 (2026-06-24) — 色彩感知模板匹配(HSV)
+
+区分形状相同的红色与绿色状态点。完整参考:[`docs/source/Zh/doc/new_features/v179_features_doc.rst`](../docs/source/Zh/doc/new_features/v179_features_doc.rst)。
+
+- **`match_color` / `match_color_all`**(`AC_match_color`、`AC_match_color_all`):`visual_match` 每个匹配器都先转灰阶,故形状相同的红 vs 绿无法区分;`color_region` 找已知颜色的 blob 却无法对多色字形做模板匹配。本功能在 HSV 色相/饱和度上以色彩*距离*度量(`TM_SQDIFF_NORMED`——相关会把绝对色相正规化掉,使红→绿边与黑→蓝边同分)。重用 `color_region` 的 RGB 加载器 + `visual_match` 的 resize/NMS/`Match`。`channels` 默认 `("h","s")`(平坦饱和度目标用 `("h",)`);纯色 blob 请用 `find_color_region`。不导入 `PySide6`。
+
+## 本次更新 (2026-06-24) — 多模板共识匹配
+
+把同一目标的多个参考裁切投票成单一可信位置。完整参考:[`docs/source/Zh/doc/new_features/v178_features_doc.rst`](../docs/source/Zh/doc/new_features/v178_features_doc.rst)。
+
+- **`match_ensemble` / `vote_centers`**(`AC_match_ensemble`、`AC_vote_centers`):一个按钮以多种状态呈现(默认/悬停/按下)但是单一逻辑目标;`ab_locator` 只选一个策略、`match_template(scales=...)` 只扫一个模板——两者都不融合多参考。本功能匹配每个参考,聚类命中中心,只有在 ≥ `min_votes` 个于 `agree_px` 内一致时才接受,返回 `{point, votes, n_candidates, spread}`——减少换肤/动画 UI 的误判。重用 `visual_match.match_template` + `grounding_consensus`;`vote_centers` 为纯投票核心。不导入 `PySide6`。
+
 ## 本次更新 (2026-06-24) — 逐步评审特征 + 规则式步骤评分
 
 把为代理步骤评分所需的证据打包,并内建规则式评分器。完整参考:[`docs/source/Zh/doc/new_features/v177_features_doc.rst`](../docs/source/Zh/doc/new_features/v177_features_doc.rst)。

README/WHATS_NEW_zh-TW.md

@@ -1,5 +1,23 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-24) — 比對前安定閘 + 命中穩定性
+
+避免在動畫進行中比對,並確認命中跨幀維持穩定。完整參考:[`docs/source/Zh/doc/new_features/v180_features_doc.rst`](../docs/source/Zh/doc/new_features/v180_features_doc.rst)。
+
+- **`region_stability` / `match_persistence`**(`AC_region_stability`、`AC_match_persistence`):`smart_waits.wait_until_screen_stable` 以布林閘控即時迴圈——無法對可注入幀序列評分穩定度,也無法檢查某*命中*是否維持。`region_stability` 以相鄰幀 SSIM 評分(`{stable, mean_ssim, min_ssim}`);`match_persistence` 確認 template 在*每一*幀都找到且中心於 `agree_px` 內一致(`{persisted, n_hits, jitter}`)。重用 `ssim` + `visual_match` + `grounding_consensus`;幀可注入;不匯入 `PySide6`。
+
+## 本次更新 (2026-06-24) — 色彩感知樣板比對(HSV)
+
+區分形狀相同的紅色與綠色狀態點。完整參考:[`docs/source/Zh/doc/new_features/v179_features_doc.rst`](../docs/source/Zh/doc/new_features/v179_features_doc.rst)。
+
+- **`match_color` / `match_color_all`**(`AC_match_color`、`AC_match_color_all`):`visual_match` 每個比對器都先轉灰階,故形狀相同的紅 vs 綠無法區分;`color_region` 找已知顏色的 blob 卻無法對多色字形做樣板比對。本功能在 HSV 色相/飽和度上以色彩*距離*度量(`TM_SQDIFF_NORMED`——相關會把絕對色相正規化掉,使紅→綠邊與黑→藍邊同分)。重用 `color_region` 的 RGB 載入器 + `visual_match` 的 resize/NMS/`Match`。`channels` 預設 `("h","s")`(平坦飽和度目標用 `("h",)`);純色 blob 請用 `find_color_region`。不匯入 `PySide6`。
+
+## 本次更新 (2026-06-24) — 多模板共識比對
+
+把同一目標的多個參考裁切投票成單一可信位置。完整參考:[`docs/source/Zh/doc/new_features/v178_features_doc.rst`](../docs/source/Zh/doc/new_features/v178_features_doc.rst)。
+
+- **`match_ensemble` / `vote_centers`**(`AC_match_ensemble`、`AC_vote_centers`):一個按鈕以多種狀態呈現(預設/懸停/按下)但是單一邏輯目標;`ab_locator` 只選一個策略、`match_template(scales=...)` 只掃一個模板——兩者都不融合多參考。本功能比對每個參考,聚類命中中心,只有在 ≥ `min_votes` 個於 `agree_px` 內一致時才接受,回傳 `{point, votes, n_candidates, spread}`——減少換膚/動畫 UI 的誤判。重用 `visual_match.match_template` + `grounding_consensus`;`vote_centers` 為純投票核心。不匯入 `PySide6`。
+
 ## 本次更新 (2026-06-24) — 逐步評審特徵 + 規則式步驟評分
 
 把為代理步驟評分所需的證據打包,並內建規則式評分器。完整參考:[`docs/source/Zh/doc/new_features/v177_features_doc.rst`](../docs/source/Zh/doc/new_features/v177_features_doc.rst)。

WHATS_NEW.md

@@ -1,5 +1,23 @@
 # What's New — AutoControl
 
+## What's new (2026-06-24) — Pre-Match Settle Gating + Match Persistence
+
+Avoid matching mid-animation, and confirm a hit holds steady across frames. Full reference: [`docs/source/Eng/doc/new_features/v180_features_doc.rst`](docs/source/Eng/doc/new_features/v180_features_doc.rst).
+
+- **`region_stability` / `match_persistence`** (`AC_region_stability`, `AC_match_persistence`): `smart_waits.wait_until_screen_stable` gates a live loop with a boolean — it can't score stability on an injectable frame sequence or check whether a *match* held steady. `region_stability` scores consecutive-frame SSIM (`{stable, mean_ssim, min_ssim}`); `match_persistence` confirms a template is found in *every* frame with the centres agreeing within `agree_px` (`{persisted, n_hits, jitter}`). Reuses `ssim` + `visual_match` + `grounding_consensus`; injectable frames; no `PySide6`.
+
+## What's new (2026-06-24) — Colour-Aware Template Matching (HSV)
+
+Tell a red status dot from a green one of identical shape. Full reference: [`docs/source/Eng/doc/new_features/v179_features_doc.rst`](docs/source/Eng/doc/new_features/v179_features_doc.rst).
+
+- **`match_color` / `match_color_all`** (`AC_match_color`, `AC_match_color_all`): every `visual_match` matcher grayscales first, so red vs green of identical shape is indistinguishable; `color_region` finds known-colour blobs but can't template-match a multi-colour glyph. This matches on HSV hue/saturation with a colour-*distance* metric (`TM_SQDIFF_NORMED` — correlation would normalise away the absolute hue, scoring a red→green edge same as black→blue). Reuses `color_region`'s RGB loaders + `visual_match`'s resize/NMS/`Match`. `channels` default `("h","s")` (use `("h",)` for flat-saturation targets); for solid blobs use `find_color_region`. No `PySide6`.
+
+## What's new (2026-06-24) — Multi-Template Consensus Matching
+
+Vote several reference crops of one target into a single trustworthy location. Full reference: [`docs/source/Eng/doc/new_features/v178_features_doc.rst`](docs/source/Eng/doc/new_features/v178_features_doc.rst).
+
+- **`match_ensemble` / `vote_centers`** (`AC_match_ensemble`, `AC_vote_centers`): a button renders in several states (default/hover/pressed) but is one logical target; `ab_locator` picks one strategy and `match_template(scales=...)` sweeps one template — neither fuses multiple references. This matches each reference, clusters the hit centres, and accepts a target only when ≥ `min_votes` agree within `agree_px`, returning `{point, votes, n_candidates, spread}` — cutting false positives on themed/animated UI. Reuses `visual_match.match_template` + `grounding_consensus`; `vote_centers` is the pure voting core. No `PySide6`.
+
 ## What's new (2026-06-24) — Per-Step Critic Features + Rule-Based Step Scorer
 
 Bundle the evidence to score an agent step, with a built-in rule-based scorer. Full reference: [`docs/source/Eng/doc/new_features/v177_features_doc.rst`](docs/source/Eng/doc/new_features/v177_features_doc.rst).

docs/source/Eng/doc/new_features/v178_features_doc.rst

@@ -0,0 +1,43 @@
+Multi-Template Consensus Matching
+=================================
+
+A button often renders in several states — default / hover / pressed / disabled — yet it is a
+single logical target. ``ab_locator`` A/B-tests *which single strategy wins* and
+``match_template(scales=...)`` sweeps *one* template across scales — neither fuses *multiple
+reference crops* into one vote. ``match_ensemble`` matches each reference, clusters the hit
+centres and accepts a target only when at least ``min_votes`` references agree within
+``agree_px`` — sharply cutting false positives on themed / animated UI.
+
+The voting core (``vote_centers``) is pure-stdlib and reuses ``grounding_consensus`` for the
+clustering, so it is unit-testable with no image; only ``match_ensemble`` itself calls
+``visual_match.match_template`` (testable on a synthetic injected ``haystack``). Imports no
+``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import match_ensemble, vote_centers
+
+    # three reference crops of the same button (default / hover / pressed)
+    result = match_ensemble(["btn_default.png", "btn_hover.png", "btn_pressed.png"],
+                            min_score=0.8, agree_px=10, min_votes=2)
+    if result:
+        click(*result["point"])     # {point, votes, n_candidates, spread}
+
+    # or vote hit centres you already have
+    vote_centers([[100, 100], [102, 98], [400, 400]], agree_px=10, min_votes=2)
+
+``match_ensemble`` returns ``{point, votes, n_candidates, spread}`` for the consensus location,
+or ``None`` if fewer than ``min_votes`` references agree. ``vote_centers`` is the pure voting
+step over candidate centres you supply.
+
+Executor commands
+-----------------
+
+``AC_match_ensemble`` (``templates`` / ``min_score`` / ``agree_px`` / ``min_votes`` /
+``region`` → ``{found, result}``) and ``AC_vote_centers`` (``centers`` / ``agree_px`` /
+``min_votes`` → ``{found, result}``). They are exposed as the MCP tools ``ac_match_ensemble`` /
+``ac_vote_centers`` (read-only) and as the Script Builder commands **Match Ensemble (vote
+references)** / **Vote Centers (consensus)** under **Image**.

docs/source/Eng/doc/new_features/v179_features_doc.rst

@@ -0,0 +1,47 @@
+Colour-Aware Template Matching (HSV)
+====================================
+
+Every matcher in ``visual_match`` converts to grayscale first, so a red versus green status
+indicator of identical shape is *indistinguishable* to ``match_template`` — the discriminating
+signal is thrown away. ``color_region`` finds blobs of a *known* colour but cannot
+template-match a multi-colour glyph by appearance. ``color_match`` matches on the HSV
+hue / saturation channels using a colour-*distance* metric (``TM_SQDIFF_NORMED``, not a
+correlation — correlation normalises away the absolute hue, so a red→green edge and a
+black→blue edge would score the same), locating colour-discriminated targets that grayscale
+matching collapses.
+
+It reuses ``color_region``'s RGB loaders and ``visual_match``'s resize / NMS / ``Match``. The
+``haystack`` is injectable; the search is unit-testable on synthetic arrays. Imports no
+``PySide6``.
+
+Note: like any window metric, a *flat* single-colour patch has no per-channel variance — for
+solid colour blobs use ``find_color_region``; ``color_match`` is for targets with colour
+*structure*.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import match_color, match_color_all
+
+    # locate a red status chip, not the green one of the same shape
+    hit = match_color("status_red.png", channels=("h", "s"), min_score=0.7)
+    if hit:
+        click(*hit.center)
+
+    for m in match_color_all("tag_green.png", channels=("h",)):
+        print(m.center, m.score)
+
+``match_color`` returns the best ``Match`` over the chosen ``channels`` (default ``("h", "s")``;
+use ``("h",)`` for flat-saturation targets), or ``None``. ``match_color_all`` returns every
+match at or above ``min_score`` with overlaps removed by NMS.
+
+Executor commands
+-----------------
+
+``AC_match_color`` (``template`` / ``channels`` / ``min_score`` / ``scales`` / ``region`` →
+``{found, match}``) and ``AC_match_color_all`` (adds ``max_results`` / ``nms_iou`` →
+``{count, matches}``). They are exposed as the MCP tools ``ac_match_color`` /
+``ac_match_color_all`` (read-only) and as the Script Builder commands **Match Template
+(colour/HSV)** / **Match Template All (colour/HSV)** under **Image**.

docs/source/Eng/doc/new_features/v180_features_doc.rst

@@ -0,0 +1,43 @@
+Pre-Match Settle Gating + Match Persistence
+===========================================
+
+Matching mid-animation is a top flakiness source: the spinner is still spinning, a transition
+is half-done, and the matcher fires on a transient. ``smart_waits.wait_until_screen_stable``
+gates a *live polling loop* and returns a boolean — it cannot score stability on an *injectable*
+sequence of frames, nor tell you whether a *match* held steady across them. ``match_stability``
+adds both, on injected frames so it is unit-testable: ``region_stability`` scores how settled a
+frame sequence is (consecutive-frame SSIM), and ``match_persistence`` checks that a template's
+hit holds at the same place across the frames (low jitter), not just in one lucky frame.
+
+It reuses ``ssim.ssim_compare``, ``visual_match.match_template`` and
+``grounding_consensus.consensus_point`` — no new matching code. The frames are injectable
+(ndarray / path / PIL). Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import region_stability, match_persistence
+
+    frames = [grab(), grab(), grab()]   # a few snapshots of the same region
+    if region_stability(frames, settle_threshold=0.99)["stable"]:
+        do_the_match()
+
+    result = match_persistence("button.png", frames, min_score=0.8, agree_px=8)
+    if result["persisted"]:
+        print("steady hit, jitter", result["jitter"])
+
+``region_stability`` returns ``{stable, mean_ssim, min_ssim}`` — ``stable`` when the lowest
+consecutive-frame SSIM clears ``settle_threshold``. ``match_persistence`` returns
+``{persisted, n_hits, jitter}`` — ``persisted`` when the template is found in *every* frame and
+the hit centres agree within ``agree_px`` (``jitter`` is their spread; 0 = rock steady).
+
+Executor commands
+-----------------
+
+``AC_region_stability`` (``frames`` / ``settle_threshold`` → ``{stable, mean_ssim, min_ssim}``)
+and ``AC_match_persistence`` (``template`` / ``frames`` / ``min_score`` / ``agree_px`` →
+``{persisted, n_hits, jitter}``). They are exposed as the MCP tools ``ac_region_stability`` /
+``ac_match_persistence`` (read-only) and as the Script Builder commands **Region Stability
+(frames)** / **Match Persistence (frames)** under **Image**.

docs/source/Eng/eng_index.rst

@@ -200,6 +200,9 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v175_features_doc
    doc/new_features/v176_features_doc
    doc/new_features/v177_features_doc
+   doc/new_features/v178_features_doc
+   doc/new_features/v179_features_doc
+   doc/new_features/v180_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

docs/source/Zh/doc/new_features/v178_features_doc.rst

@@ -0,0 +1,39 @@
+多模板共識比對
+==============
+
+一個按鈕常以多種狀態呈現——預設 / 懸停 / 按下 / 停用——但它是單一邏輯目標。``ab_locator``
+A/B 測試*哪個單一策略勝出*,``match_template(scales=...)`` 跨縮放掃描*一個*模板——兩者都不把
+*多個參考裁切*融合成一次投票。``match_ensemble`` 比對每個參考,聚類命中中心,只有在至少
+``min_votes`` 個參考於 ``agree_px`` 內一致時才接受目標——大幅減少換膚 / 動畫 UI 上的誤判。
+
+投票核心(``vote_centers``)為純標準函式庫,並重用 ``grounding_consensus`` 做聚類,因此可在
+無影像下單元測試;只有 ``match_ensemble`` 本身呼叫 ``visual_match.match_template``(可於注入的
+合成 ``haystack`` 上測試)。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import match_ensemble, vote_centers
+
+    # 同一按鈕的三個參考裁切(預設 / 懸停 / 按下)
+    result = match_ensemble(["btn_default.png", "btn_hover.png", "btn_pressed.png"],
+                            min_score=0.8, agree_px=10, min_votes=2)
+    if result:
+        click(*result["point"])     # {point, votes, n_candidates, spread}
+
+    # 或對你已有的命中中心投票
+    vote_centers([[100, 100], [102, 98], [400, 400]], agree_px=10, min_votes=2)
+
+``match_ensemble`` 回傳共識位置的 ``{point, votes, n_candidates, spread}``,或在少於
+``min_votes`` 個參考一致時回傳 ``None``。``vote_centers`` 是對你提供的候選中心的純投票步驟。
+
+執行器指令
+----------
+
+``AC_match_ensemble``(``templates`` / ``min_score`` / ``agree_px`` / ``min_votes`` /
+``region`` → ``{found, result}``)與 ``AC_vote_centers``(``centers`` / ``agree_px`` /
+``min_votes`` → ``{found, result}``)。兩者以 MCP 工具 ``ac_match_ensemble`` /
+``ac_vote_centers``(唯讀)及 Script Builder 指令 **Match Ensemble (vote references)** /
+**Vote Centers (consensus)**(位於 **Image** 分類下)形式提供。

docs/source/Zh/doc/new_features/v179_features_doc.rst

@@ -0,0 +1,42 @@
+色彩感知樣板比對(HSV)
+========================
+
+``visual_match`` 的每個比對器都先轉灰階,因此形狀相同的紅色與綠色狀態指示燈對
+``match_template`` 而言*無法區分*——具辨別力的訊號被丟棄了。``color_region`` 找*已知*顏色的
+blob,卻無法以外觀對多色字形做樣板比對。``color_match`` 在 HSV 色相 / 飽和度通道上以色彩
+*距離*度量(``TM_SQDIFF_NORMED``,而非相關——相關會把絕對色相正規化掉,使紅→綠邊與黑→藍邊
+得分相同)進行比對,定位灰階比對會塌掉的色彩辨識目標。
+
+本功能重用 ``color_region`` 的 RGB 載入器與 ``visual_match`` 的 resize / NMS / ``Match``。
+``haystack`` 可注入;搜尋可在合成陣列上單元測試。不匯入 ``PySide6``。
+
+註:如同任何視窗度量,*平坦*的單色 patch 在各通道無變異——純色 blob 請用
+``find_color_region``;``color_match`` 適用於有色彩*結構*的目標。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import match_color, match_color_all
+
+    # 定位紅色狀態 chip,而非同形狀的綠色
+    hit = match_color("status_red.png", channels=("h", "s"), min_score=0.7)
+    if hit:
+        click(*hit.center)
+
+    for m in match_color_all("tag_green.png", channels=("h",)):
+        print(m.center, m.score)
+
+``match_color`` 在選定 ``channels``(預設 ``("h", "s")``;平坦飽和度目標用 ``("h",)``)中回傳
+最佳 ``Match``,或 ``None``。``match_color_all`` 回傳所有達到 ``min_score`` 的匹配,重疊以 NMS
+移除。
+
+執行器指令
+----------
+
+``AC_match_color``(``template`` / ``channels`` / ``min_score`` / ``scales`` / ``region`` →
+``{found, match}``)與 ``AC_match_color_all``(另加 ``max_results`` / ``nms_iou`` →
+``{count, matches}``)。兩者以 MCP 工具 ``ac_match_color`` / ``ac_match_color_all``(唯讀)及
+Script Builder 指令 **Match Template (colour/HSV)** / **Match Template All (colour/HSV)**
+(位於 **Image** 分類下)形式提供。