Skip to content

Normalise selectors in plan synthesiser and verifier #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lydakis wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Normalise selectors in plan synthesiser and verifier #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion agents/hybrid.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,16 @@ def _handle_plan_sketch(self, plan_text: Optional[str]) -> None:
if "(" in token and token.endswith(")"):
prefix, arg_str = token.split("(", 1)
verb_name = prefix.strip()
args = [arg_str[:-1]] if arg_str.endswith(")") else [arg_str]
if arg_str.endswith(")"):
arg_str = arg_str[:-1]
raw_args = [part for part in arg_str.split(",") if part.strip()]

def _clean_argument(value: str) -> str:
cleaned = value.strip().rstrip(")").strip()
cleaned = cleaned.strip("\"'")
return cleaned

args = [_clean_argument(part) for part in raw_args]
Comment on lines 388 to +400
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Argument parsing ignores quoted commas and corrupts tokens

When _handle_plan_sketch splits arg_str by "," it does not consider quoting, so a token such as Type(#field, "hello, world") produces args = ["#field", "hello", "world"] instead of keeping the comma inside the second argument. Any plan where argument values legitimately contain commas or closing parentheses will now be broken and the original string cannot be reconstructed. Using a parsing approach that honours quoted strings or balanced parentheses would avoid fragmenting arguments.

Useful? React with 👍 / 👎.

try:
verb = DSLVerb(verb_name)
except ValueError:
Expand Down
9 changes: 8 additions & 1 deletion flappy/synth.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,13 +64,20 @@ def _extract_selectors(self, context: Optional[dict]) -> Sequence[str]:

def _plan_compatible(self, node: DSLNode, selectors: Sequence[str]) -> bool:
if node.verb in {DSLVerb.CLICK, DSLVerb.TYPE} and node.args:
selector = node.args[0]
selector = self._normalise_selector(node.args[0])
if selector and selectors and selector not in selectors:
return False
for child in node.children:
if not self._plan_compatible(child, selectors):
return False
return True

@staticmethod
def _normalise_selector(raw_selector: str) -> str:
token = raw_selector.split(",", 1)[0].strip()
token = token.rstrip(")").strip()
token = token.strip("\"'")
Comment on lines +75 to +79
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Truncating selectors at first comma breaks valid matches

The normaliser now calls raw_selector.split(",", 1) and discards everything after the first comma. Legitimate CSS selectors often contain commas inside attribute values ([aria-label="Hello, world"]) or intentionally select multiple elements (#foo, #bar). After this change _plan_compatible will normalise such selectors to only the prefix, so even when the context contains the full selector the synthesiser (and the identical logic in PlanVerifier._normalise_selector) will reject a compatible plan as missing its selector. The normalisation should respect quoted segments or only strip comma-separated arguments rather than blindly truncating.

Useful? React with 👍 / 👎.

return token


__all__ = ["Sketch", "CandidatePlan", "PlanSynthesiser"]
9 changes: 8 additions & 1 deletion flappy/verify.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,19 @@ def _extract_selectors(self, context: Optional[dict]) -> Sequence[str]:
def _find_missing_selectors(self, node: DSLNode, selectors: Sequence[str]) -> set[str]:
missing: set[str] = set()
if node.verb in {DSLVerb.CLICK, DSLVerb.TYPE} and node.args:
selector = node.args[0]
selector = self._normalise_selector(node.args[0])
if selector and selectors and selector not in selectors:
missing.add(selector)
for child in node.children:
missing.update(self._find_missing_selectors(child, selectors))
return missing

@staticmethod
def _normalise_selector(raw_selector: str) -> str:
token = raw_selector.split(",", 1)[0].strip()
token = token.rstrip(")").strip()
token = token.strip("\"'")
return token


__all__ = ["PlanVerifier", "VerificationResult", "Predicate"]
13 changes: 13 additions & 0 deletions tests/test_flappy_stubs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,19 @@ def test_plan_synthesiser_identity():
assert proposals[0].root.to_dict() == leaf.to_dict()


def test_selector_normalisation_for_comma_arguments():
plan = dsl.DSLNode(verb=dsl.DSLVerb.TYPE, args=["#field, hello"])
sketch = synth.Sketch(root=plan, holes=[])
context = {"selectors": ["#field"]}

proposals = list(synth.PlanSynthesiser().enumerate(sketch, context=context))
assert proposals, "Synthesiser should return the plan when selector matches"

verifier = verify.PlanVerifier()
result = verifier.verify(plan, trace=[], context=context)
assert result.ok, "Verifier should accept plans with normalised selectors"


def test_verifier_stub():
verifier = verify.PlanVerifier()
result = verifier.verify(
Expand Down