feat: Add GitHub integration with agent_prompts and github_components

[julian-risch]

Related Issues

• Related to Add GitHub integration with agent_prompts and github_components #1650

• Related to Move Agent from experimental to main haystack-experimental#250
  @sjrl brought up that one way to keep example files from the experimental Agent around is an integration chore: Remove Agent after Haystack 2.12 release haystack-experimental#263 (comment)

Proposed Changes:

• Move github_components from experimental to a new integration
• Move agent_prompts from experimental to a new integration

The idea is to enable users to run the example notebook (or a version with updated imports) after having installed this new integration

How did you test it?

New unit tests and I ran all usage examples successfully with a test repo.

I haven't tested it with the notebook yet, which we would need to update first. (tracked by deepset-ai/haystack-cookbook#183 )

Notes for the reviewer

• I suggest we rename github_token parameter to api_key for consistency with many other integrations.
• Some components have github_token: Optional[Secret] = None, because they can work without any token while others use Secret.from_env_var("GITHUB_TOKEN"). I suggest we use Secret.from_env_var("GITHUB_TOKEN", strict=False) where we currently have None as the default.
• The internal implementation of the components differs in how they use _get_headers or _get_request_headers or define headers inline. We could refactor that.
• While we could find a way to set up integration tests, I would rather leave them out of this PR.
• GithubRepositoryViewer has a branch parameter in the run method, which could also be named ref to make more clear it can also be a tag or commit hash
• Similar to the empty lines in prompts and linebreaks: Do we need to add whitespaces in the beginning of new lines. Trailing whitespaces are removed by the linter.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[sjrl]

I think it would be good for us to add examples here in the Readme on how to use or to link to the tutorial/google colab for how to use.

Also another relevant detail I think is that these prompts were optimized using Anthropic models. Could be a useful thing for users to know.

[julian-risch]

google colab in the cookbook and some more examples in an integration page is what I imagine. The README's we currently don't fill out, for example see: https://pypi.org/project/opensearch-haystack/
Might be a good idea to change that and use a copy of the integrations page. I don't see a good reason to keep it empty but I would prefer a consistent solution. I'll talk to Bilge.

[sjrl]

@julian-risch maybe a general comment on the structure here. I see that the prompts aren't being used within the library and I understand they will be used in a future tutorial/colab.

I wonder then if it would be helpful to instead pre-assemble the tools within the repo so users could easily import the tools and immediately pass them to an Agent. What do you think?

[sjrl]

I like this pattern for an incidental reason. By not putting the resolve of the Secret in the init method we allow users to not need to provide the secret until run time which is nice especially when running pipeline validation like in the deepset platform where the secrets are not available to the pipeline in the pipeline builder.

Could be nice to do this in the other components, basically to move the resolve_value() outside of the init method.

[sjrl]

@julian-risch overall this looks really good! I mostly have minor comments and only one larger conceptual one about maybe providing users Tools directly instead of needing to compose them, themselves.

I didn't comb through every line since there is a lot, but it's well tested so it's good to go from my perspective! We can always make quick updates to this if things arise and depending on usage.