Home

Preliminary work, followed by proposed next steps.

Preliminary work

• Sean Davis wrote an Agentic Coding Intro to help beginners get started
• Currently, this repo contains some code-centric skills converted from the BioC How-tos by Sean Davis.
• In parallel, Levi Waldron and Marcel Ramos have created some more generic AI Agent Skills, including skills to create a skill and validate a skill according to standards laid out in that repository.
• Vince Carey proposed an R-package + pkgdown approach to hosting skills (skills copied over from this site)
• Levi Waldron and Marcel Ramos created the BiocReviews GitHub workflow for AI-augmented package review
• Levi Waldron produced a security audit of TAB and core-maintained Bioconductor packages (private repo, ask for access)

Next steps

Developing our approach to Agent Skills

• Either:
  1. Redo one of these HOWTOs in narrative format with a minimum of code snippets, and test how each version behaves differently, or
  2. Create one of each in an area of expertise of an AI working group member.
• Then, decide on whether we will have more narrative or more code-centric Skills
• Merge https://github.com/waldronlab/ai-agent-skills into this site and rename waldronlab -> bioconductor (may be useful to keep them separate while we explore different approaches to Skills)

Testing Agent Skills

Skill validation

• Incorporate the waldronlab validate-skill in a GitHub Action that to validate any edits to skills in the repo
• Ensure that all skills in the waldronlab repo pass validate-skill, before any migration

Sample unit test

Create a unit test for a Skill intended to generate code for a quantitative task. There should be a synthetic or real data set, and one or more right answers that can be tested. This could be adapted from existing package unit tests or vignettes. The unit test would include a) some written instructions (ideally in multiple versions, from vague to detailed), and b) a testable answer that generated code should produce.

CI/CI for Skills

1. Create a GitHub Action that loads the Skill, performs the unit test, and returns a pass/fail
2. Support testing the skill using a matrix of LLM models

Security Audit

Continue with existing core set of packages, and:

• Try using another agent to verify/exploit vulnerabilities
• Try using another agent to create patches

Package review

• Look for true positive / true negative packages, e.g.:

  • “rotate” in EBImage (for EBImage objects) and in SpatialData (for SpatialData objects) conflict. ggtree defines rotate that conflicts with BiocGenerics. Also the ecolitk package. These conflicts are hard to find for reviewers and would be good to automatically detect system-wide.

• Get more people to try BiocReviews on different packages and solicit feedback.

Process

• Give interested collaborators write access to this repo but protect main branch
• Use this repo's issues and potentially a project board.
• Create some "good first issues" and track tests and results etc occurring at different sites, like BiocReviews