Integrate browser-use as a tool

[rjambrecic]

Why are these changes needed?

from autogen.tools.experimental.browser_use import BrowserUseResult, BrowserUseTool

llm_config = ...
user_proxy = UserProxyAgent(name="user_proxy", human_input_mode="NEVER")
assistant = AssistantAgent(name="assistant", llm_config=llm_config)

browser_use_tool = BrowserUseTool(llm_config=llm_config)
browser_use_tool.register_for_execution(user_proxy)
browser_use_tool.register_for_llm(assistant)

result = user_proxy.initiate_chat(
    recipient=assistant,
    message="Go to Reddit, search for 'ag2' in the search bar, click on the first post and return the first comment.",
    max_turns=2,
)

Result:

user_proxy (to assistant):

Go to Reddit, search for 'ag2' in the search bar, click on the first post and return the first comment.

--------------------------------------------------------------------------------
assistant (to user_proxy):

***** Suggested tool call (call_vqAAzVZk86MsBopDupXUUKSX): browser_use *****
Arguments: 
{"task":"Go to Reddit, search for 'ag2' in the search bar, click on the first post and return the first comment."}
****************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION browser_use...
Call ID: call_vqAAzVZk86MsBopDupXUUKSX
Input arguments: {'task': "Go to Reddit, search for 'ag2' in the search bar, click on the first post and return the first comment."}
user_proxy (to assistant):

***** Response from calling tool (call_vqAAzVZk86MsBopDupXUUKSX) *****
{"extracted_content":["🔗  Navigated to https://www.reddit.com","⌨️  Input \"ag2\" into index 4","⌨️  Sent keys: Enter","🖱️  Clicked button with index 14: AG2 vs Autogen, which one to use?","📄  Extracted page as text\n: Go to AI_Agentsr/AI_Agentsr/AI_AgentsA place for discussion around the use of AI Agents and related tools. AI Agents are LLMs that have the ability to \"use tools\" or \"execute functions\" in an autonomous or semi-autonomous (also known as human-in-the-loop) fashion.\n\nFollow our event calendar: https://lu.ma/oss4ai\n\nJoin us on Discord! https://discord.gg/6tGkQcFjBY43KMembers111Online•15 days agoXodnilADMINMODAG2 vs Autogen, which one to use?DiscussionI’m trying to decide betweenAG2andAutoGenfor building a multi-agent system. Both seem powerful, but I’m not sure which one fits my needs better. It's so confusing really.From what I’ve seen:AG2: Focuses on stability and backward compatibility, with features like StateFlow and Reasoner agents. But how does it handle structured outputs and multi-agent workflows?AutoGen: Known for advanced multi-agent collaboration and human-in-the-loop functionality. It integrates well with LLMs, but is it beginner-friendly?Which one would you recommend and why?ThanksBreeze_pm•PromotedTired of clunky project management tools? Breeze is the easy way to stay organized and on track.Learn Morebreeze.pmAdd a commentSort by:BestOpen comment sort optionsBestTopNewControversialOldQ&Amacronancer•15d agoMy impression was that AG2 was a successor architecture for Autogen.I could be wrong, and I have not had the time to read into these very deeply.ReplyreplyCtiPath•14d agoYou’re correct. In November, the developers of AutoGen branched off to form AG2. Much of AG2 is still AutoGen under the hood. But they are making some improvements like the new Captain Agent.ReplyreplyMore repliesLong_Complex_4395•15d agoI am not really conversant with frameworks for AI agents, so I may not understand their capabilities. The first rule of thumb for using any framework is what do you want to achieve?So, what are you building?Do you have a workflow for what you intend to build?What are your expectations?These questions would guide you to frame your question better for people to actually help you.Replyreplyfreudweeks•14d agoAG2 seems much more active. I think autogen is now in a support mode, while MSFT's next version, autogen 0.4 is what they're actively creating features for but is unstable. Since the founders of autogen went to ag2, I think it's a safe enough bet to go to ag2. You can always pick up autogen 0.4 when it comes out later.Replyreply\n","The first comment is: 'My impression was that AG2 was a successor architecture for Autogen. I could be wrong, and I have not had the time to read into these very deeply.'"],"final_result":"The first comment is: 'My impression was that AG2 was a successor architecture for Autogen. I could be wrong, and I have not had the time to read into these very deeply.'"}
**********************************************************************

--------------------------------------------------------------------------------
assistant (to user_proxy):

The first comment on the Reddit post is: "My impression was that AG2 was a successor architecture for Autogen. I could be wrong, and I have not had the time to read into these very deeply." TERMINATE

--------------------------------------------------------------------------------

Related issue number

Closes #640

Checks

• I've included any doc changes needed for https://docs.ag2.ai/. See https://docs.ag2.ai/docs/contributor-guide/documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 0% with 65 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ogen/tools/experimental/browser_use/browser_use.py	0.00%	61 Missing ⚠️
autogen/tools/experimental/__init__.py	0.00%	2 Missing ⚠️
autogen/tools/experimental/browser_use/__init__.py	0.00%	2 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Flag	Coverage Δ
unittests	17.71% <0.00%> (-25.76%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
autogen/tools/tool.py	46.00% <ø> (-42.00%)	⬇️
setup_ag2.py	0.00% <ø> (ø)
setup_autogen.py	0.00% <ø> (ø)
autogen/tools/experimental/__init__.py	0.00% <0.00%> (ø)
autogen/tools/experimental/browser_use/__init__.py	0.00% <0.00%> (ø)
...ogen/tools/experimental/browser_use/browser_use.py	0.00% <0.00%> (ø)

... and 133 files with indirect coverage changes