Skip to content

A triage SLA, a roadmap of what is accepted, and a daily measurement (DX-11, #1785) - #1930

Merged
Kalyanpulikanti-01 merged 8 commits into
samyama-ai:devfrom
Raghuvarma905:dx-11/1785-triage-sla-roadmap
Oct 22, 2026
Merged

Kalyanpulikanti-01 merged 8 commits into
samyama-ai:devfrom
Raghuvarma905:dx-11/1785-triage-sla-roadmap

Conversation

@Raghuvarma905

Copy link
Copy Markdown
Collaborator

Part of #1785 (DX-11). It publishes the triage policy and a roadmap, and measures the policy every weekday. It does not claim the requirement is met. See "Not in this PR" below.

What DX-11 asks for, and what was there

Public issue triage SLA: first response ≤2 business days; a public roadmap of what is accepted. Policy published + met.

Before this PR:

  • No triage policy. SECURITY.md's 3 and 7 business days apply to vulnerability reports only, as the issue says.
  • No roadmap at all. The issue says ROADMAP.md "reads as a development history". It has since been deleted (Remove stale top-level files #1835, 2026-10-07) for exactly that reason. So DX-11 was really at 0 of 2, not 1 of 2.
  • No measurement. Nobody knew the response times. They are bad. Only two issues have ever come from outside the team, and both waited far beyond 2 business days:
Issue Opened First response Business days
#275 2026-07-02 a comment, 2026-09-30 64.1
#542 2026-08-17 closed, 2026-09-30, no comment 32.6

SUPPORT.md lists both. The policy applies from the day it reaches main. It is not backdated.

What this adds

SUPPORT.md: the policy. GitHub links it from the new-issue page. It covers:

  • where to take a bug, a question or a vulnerability;
  • the promise: a first response within 2 business days;
  • what counts as a response: a maintainer's comment, or a maintainer closing the issue. A label or an assignee does not count, because GitHub doesn't notify the reporter about either;
  • business days: Mon–Fri, Asia/Kolkata, with no holiday exclusions, which is stricter than it has to be;
  • what it doesn't cover: pull requests, the chat, and security reports.

ROADMAP.md: what is accepted, as live links rather than a copied list. Accepted work is an open issue with a horizon label: H1, due 2026-12-31. The page links to the open and completed H1 issues, explains how a request gets accepted or declined, and says plainly that no list of non-goals is published yet. It does not repeat the problem #1835 fixed, because it holds no list that can go stale.

scripts/triage_sla.py and .github/workflows/triage-sla.yml: the measurement.

  • It runs every weekday at 09:00 IST, and on manual dispatch.
  • It reports every outside issue's first-response time in the job summary.
  • Exit 1 means an issue opened since SUPPORT.md reached the default branch missed the SLA.
  • Exit 2 means it could not measure. That is deliberately separate, so a GitHub API outage never reads as a missed SLA.
  • It isn't a PR check, because whether an issue got a reply doesn't depend on a PR's code. Its rules are tested on every PR (below).

Who counts as a maintainer is decided from each person's repository role (triage or higher), via collaborators/{user}/permission. It is not decided from author_association. This org has no public members, and a workflow's GITHUB_TOKEN can't see private membership. So in CI, every maintainer would show as CONTRIBUTOR, and all 785 team issues would look like outside reports. If the role lookup finds no maintainer among the issue authors, the script exits 2 instead of reporting.

scripts/test_triage_sla.py runs in the test job next to the release-gate test. It is offline and checks 12 rules: weekends, a Saturday issue starting its clock on Monday, IST rather than UTC, exactly 2.0 days counting as met while one minute more is missed, a label not being a response, the author's own close not counting, a bot with write access not counting, and more.

Also: a row in CONTRIBUTING's "What CI runs" table, a ROADMAP pointer in its "Where to Start", two rows in scripts/README.md, and one line in README's Contributing section.

Verified locally

Not verified, and why

  • The role lookup has not run under a workflow's GITHUB_TOKEN. I could only run it with a user token, and I haven't confirmed which token permissions the endpoint needs. If it fails in Actions, the first run exits 2 with that message; it won't report wrong numbers.
  • The scheduled run starts only once this reaches main. Scheduled workflows run from the default branch, and this PR targets dev.

Not in this PR

  • "Met." That needs time: a run of outside issues answered within 2 days. It can't be done in a PR.
  • A published list of non-goals. The product spec has one, but it is private and includes business reasoning. Publishing it is a maintainer's decision, so ROADMAP.md says no such list exists yet.
  • The scorecard row. It flips once CH-GOVERNANCE in samyama-graph-competitor-benchmarks reads this. That is a different repository.

…(DX-11, samyama-ai#1785)

SUPPORT.md promises a first response within 2 business days (Mon-Fri,
Asia/Kolkata) to every issue opened from outside the maintainer team. A
response is a maintainer's comment or close; a label or assignee is not,
because the reporter is not notified of either. It lists the record before
the policy honestly: samyama-ai#275 waited 64.1 business days, samyama-ai#542 32.6.

ROADMAP.md links the live H1 issue lists instead of copying them, so it
cannot become the development history samyama-ai#1835 removed. It says plainly that
no list of non-goals is published yet.

scripts/triage_sla.py measures the promise; triage-sla.yml runs it on
weekdays. Maintainers are identified by repository role, not
author_association: the org has no public members, so under GITHUB_TOKEN
every maintainer would read as CONTRIBUTOR. Exit 1 is a missed SLA since
SUPPORT.md reached the default branch; exit 2 is "could not measure".

scripts/test_triage_sla.py pins the rules offline and runs in the test job.

Part of samyama-ai#1785

@Kalyanpulikanti-01 Kalyanpulikanti-01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local PR agent — AI review of exact head 3e259e2.

Verdict: changes_requested.

Reviewed all nine changed files. The policy activation timestamp can precede its actual publication, causing historical issues to fail the new SLA.

  • P2 scripts/triage_sla.py:212 — The original commit's committer date is not the date SUPPORT.md reached the default branch. When this PR lands on dev and later reaches main through a merge that preserves commits, this timestamp remains the earlier development date. Issues opened between that date and publication will then count against a promise that was not yet effective, contradicting SUPPORT.md and potentially failing the first scheduled run. Record the actual activation time explicitly or determine when the file became reachable on the default branch, and add a test covering delayed publication through a preserved-history merge.

The queue is held here for a maintainer; no automatic code edits or merge.

This is an AI review comment, not a human approval or a CI result.

@Kalyanpulikanti-01

Copy link
Copy Markdown
Contributor

Fresh CI on head 3e259e29b5916a9301fe26b341095ee77b5fb1fe has an additional supply-chain blocker, separate from the SLA-date review finding.

Job: https://github.com/samyama-ai/samyama-graph/actions/runs/39544089385/job/119038242432

  • rustls-webpki 0.103.15: RUSTSEC-2026-0382; patched in 0.103.16.
  • webpki-root-certs 1.0.9: RUSTSEC-2026-0383; patched in 1.0.10.

An isolated, unpublished lockfile patch containing only those two version/checksum updates passes cargo metadata --locked and cargo deny check locally; TLS integration tests are in progress. No advisory ignores or protection changes are proposed.

PR #1929 is excluded at the user's request and remains open. This PR is held for the reported review and audit blockers under the requested hold-and-report policy.

@Kalyanpulikanti-01 Kalyanpulikanti-01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local PR agent — AI review of exact head 8b9be52.

Verdict: changes_requested.

Reviewed all nine changed files. The measurement can backdate the policy and count unexplained closures as successful responses.

  • P2 scripts/triage_sla.py:212 — The oldest SUPPORT.md commit's committer timestamp is not the time the policy reached the default branch. When this change travels through dev and is merged into main, that timestamp can precede publication by days or weeks. Issues opened during that interval will incorrectly count against the new promise and can immediately fail the workflow. Record the actual publication time explicitly or derive when the policy entered the default branch, and test a delayed merge.

  • P2 scripts/triage_sla.py:189 — Every maintainer closure becomes a qualifying response, including a closure without any explanation. SUPPORT.md explicitly requires closing 'with a reason' and says the first response explains what happens next. An unanswered issue silently closed within two days therefore reports 'met' despite violating the published policy. Require evidence of the promised explanation before counting a closure, and add a test for an unexplained maintainer close.

The queue is held here for a maintainer; no automatic code edits or merge.

This is an AI review comment, not a human approval or a CI result.

@Kalyanpulikanti-01 Kalyanpulikanti-01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all nine changed files. No concrete defect established, but the central repository-role lookup has not been validated with the workflow's GITHUB_TOKEN. Confirm that the configured workflow correctly distinguishes a known maintainer from an outside reporter; the offline tests do not exercise this integration.

@Kalyanpulikanti-01 Kalyanpulikanti-01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all 10 changed files. The role classifier can misclassify maintainers with custom repository roles, producing incorrect SLA results.

  • P2 scripts/triage_sla.py:224 — A custom repository role returns its custom name in role_name alongside its underlying access level in permission. Choosing the nonempty custom name discards that access level, so a custom role based on write access fails the membership check. Its holder's issues are then classified as external and their responses do not count; the no-maintainer guard will not catch this when another author has a standard maintainer role. Check the underlying permission independently of the role name, and add an offline fixture covering a custom role with write access.

@Kalyanpulikanti-01 Kalyanpulikanti-01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all changed files. No concrete correctness, security, regression, or CI-integrity defects found. Offline tests cover the core SLA rules, and the new CI contract checks role classification using the workflow token. Execution results were not independently verified.

@Kalyanpulikanti-01
Kalyanpulikanti-01 merged commit aa93e8e into samyama-ai:dev Oct 22, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants