-
Notifications
You must be signed in to change notification settings - Fork 860
github: add workflows for gemini #6557
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Add gemini support for review and triage of issues and PRs on github Signed-off-by: flouthoc <[email protected]>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: flouthoc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/hold wait for some configs which needs to be done manually |
|
@nalind @TomSweeneyRedHat @mheon PTAL, code is automatically added from https://github.com/google-github-actions/run-gemini-cli/tree/v0.1.15?tab=readme-ov-file#quick-start |
Luap99
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am completely unable to review this, it is just way to much and I have no idea what this is supposed to do.
I have no idea why it needs all these are all github action workflows compared to the "Github App"s we have been using for reviews that don't clutter a ton of workflows.
How much github runners and runner time will it consume, we are on the github free plan for the whole org if we consume all resources this will be very bad for other projects here.
How are these workflows maintained? I see tons of variable and secretes mention in the file but not defined in the repo. Then there is a pull_request trigger which AFAIK doesn't get access to our secretes anyway (enforced by github) as this is insecure in such context.
Anyway that is not a review, because well I can't review this so maybe I Am missing stuff here completely.
| github.event_name == 'pull_request' && | ||
| github.event.pull_request.head.repo.fork == false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That seems rather useless, all our PRs come form forks normally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code is directly generated/copied from sources as suggested here https://github.com/google-github-actions/run-gemini-cli/blob/v0.1.15/README.md#4-choose-a-workflow I think manually editing them can create maintenance overhead.
This is only for a PoC, we want to see if gemini review can better call out code maintainers, main feature which we want to look into is issue triage. If we don't find PR review appealing we can remove it .
Code here is directly generated from gemini CLI, I don't think we need to maintain it manually. Necessary secret needed are added to repo other secrets are not needed for functionality as written in gemini actions doc https://github.com/google-github-actions/run-gemini-cli/blob/v0.1.15/README.md#repository-variables @Luap99 I hope this answers your concerns ? |
Well even if this is generated it means gemmini updates may require updates to the workflows, etc... How is this going to happen. What if there is a security issue in this? How can we track that? I mean like I said I am unqualified to review this. If others think buildah is the right repo to make such experiments sure go ahead and see what it produces. |
|
@TomSweeneyRedHat @nalind PTAL |
|
just dropping this here https://www.aikido.dev/blog/promptpwnd-github-actions-ai-agents I am not claiming I understand enough about this workflows but these kind of attacks is what I would be worried about. |
@Luap99 In following gemini template a section is included setting guidelines for agents and all prompts. See https://github.com/containers/buildah/pull/6557/files#diff-8994fa45fc4a53feeee42299e66f82c4e7cde1941ba41326c9ba6d742cb9288dR16 but still I am not very certain let me explore it a bit more. |
|
Well to be honest such prompts make me even more nervous, LLM are not know for their correct following of instructions. If that is the security boundary then I am totally unconvinced that this is sane. |
|
I think https://github.com/google-github-actions/run-gemini-cli fixed the recent vulnerability but lets wait for other maintainers to chime in. |
|
Is there a reason we're doing this instead of consuming pre-made actions? Securing AI in the cloud is sufficiently hard that I really do not want it to be my problem. |
|
I know there is already a pre-made Gemini review tool, for example. It's probably better than ours - lot more prompt engineering went into it. What are we gaining by maintaining this ourselves, aside from risk? |
|
@mheon What do we mean by pre-made actions? Github action in this PR is automatically generated by gemini CLI itself and most of the workflows are coming from here https://github.com/google-github-actions/run-gemini-cli , If you referring to market place this is the same action as listed here https://github.com/marketplace/actions/run-gemini-cli |
|
I was really thinking there was an official Gemini code review Github action, but there doesn't appear to be, just a bunch of unofficial ones probably build using the marketplace "Run Gemini CLI" action. I'm still quite concerned about taking sole responsibility for the prompts ourselves. Prompt injection is a real problem and I don't want to have to regularly edit these to keep up with the latest wisdom on preventing it? |
|
There is https://github.com/apps/gemini-code-assist which is used in other containers repos, but AFAIK that only does code reviews on PRs. Not issues or anything else. |
I think we majorly need this for issues and reviewing commit logs, for code review we already have copilot. |
|
Do we actually have Copilot deployed anywhere? We had code review on NV/AV but last I checked it was turned off as it was generally unhelpful in the Rust codebase. |
|
Also, what kind of issue triage are we talking about? I'm worried about having AI do first-touch on issues will discourage folks from actually reporting bugs... |
I am majorly thinking about assigning appropriate labels and tagging relevent maintainers on the issue. |
We can use gemini for code-reviews too if it appears to be helpful. It is included in this PR. |
Add gemini support for review and triage of issues and PRs on github.
Patch in following PR is generated via steps shown here https://github.com/google-github-actions/run-gemini-cli/blob/v0.1.15/README.md#4-choose-a-workflow
What type of PR is this?
What this PR does / why we need it:
How to verify it
Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?