Workaround for Using Extensions Like MetaMask | Long Action Delays

[FloopyG]

I am new to web scraping and SeleniumBase, so sorry if this question seems pretty stupid.

While trying to use MetaMask with SeleniumBase, I discovered that it is possible to interact with it just as I did with Playwright—by directly accessing the web page whenever I expect a pop-up (e.g., opening chrome-extension://{EXTENSION_ID}/home.html). However, when attempting to do so, I encountered the following exception:

Traceback (most recent call last):
File "D:\Projects\sbWeb3Base\main.py", line 7, in
sb.open(f"chrome-extension://{EXTENSION_ID}/home.html")
File "C:\Users.......\AppData\Local\Programs\Python\Python39\lib\site-packages\seleniumbase\fixtures\base_case.py", line 244, in open
raise Exception('Invalid URL: "%s"!' % url)
Exception: Invalid URL: "chrome-extension://nnmflneemandambaednjdgnbjkjocbcg/home.html"!

I edited page_utils.py at lines 169 and 114 (apparently, these functions verify the given URL) by adding "chrome-extension:" to each. This allowed the extension's page to open successfully. However, any action I try to perform on it takes a long time, whereas actions on other pages within the same session work fine.

I am not sure what causes these long delays when interacting with extension pages, and I haven't been able to figure it out through debugging.

I can provide the source code if needed and would really appreciate any help.

[FloopyG]

After some testing, I figured out that:

• After the first installation (or in our case, the first session load), MetaMask automatically opens a new page, which can interfere with scripts. To handle this, I used user_data_dir to save a persistent browser context (which I need to use anyway). I initially load all of my sessions and then perform the necessary actions.

• The slow action time on extension pages resolves itself if CDP mode is used. I am not sure why, but it helps. This doesn’t fully answer the underlying questions, but for me, these workarounds are good enough. Hopefully, they will help someone else as well.

Also, I just wanted to say thanks for creating SeleniumBase! It has proven to be very useful, easy to use, and will simplify a lot of things for me.

[Arenatdk]

Tell us how you manage Metamask. I was unable to get the DOM in metamask using a regular webdriver.

I like to use CDP mode, but I can't switch tabs and enter Metamask password.

Please show a code example of how you used Seleniumbase and Metamask.

Here's an example of code that didn't work

with SB(uc=True, test=True, user_data_dir="C:\ChromeUser\p2", locale_code="en") as sb:
sb.activate_cdp_mode("https://docs.metamask.io/developer-tools/faucet/")
sb.sleep(2)
sb.cdp.click("[data-testid='hero-cta-connect-metamask']")
sb.sleep(2)
sb.switch_to_window(0) # Don't work in CDP mode. I haven't found a way to switch CDP to another tab
sb.cdp.press_keys("#password", "yourpass")

I will be very grateful if you show code

[FloopyG]

Hi! Switching between tabs in CDP mode is an issue with no good workarounds—at least, I haven’t found any. A clumsy approach I discovered is constantly opening the MetaMask address and then re-opening the required web page. For example, if we want to log into PancakeSwap, we click Connect Wallet, then navigate to the MetaMask page in the same tab, then go back to PancakeSwap and continue with what we need to do. However, whenever we need to sign another message or transaction, we have to repeat this ritual again =D. Obviously, this approach is either too slow or simply won't work for most use cases. Also, even if there were a good workaround for this issue, it would likely result in your browser being detected by anti-bot systems due to the use of multiple pages—at least according to some discussions here. That obviously defeats the purpose of the tool.

Even though SeleniumBase is an excellent tool for web scraping, it’s not ideal for Web3 automation since that involves extensive interaction with browser extensions. Right now, I’m exploring other libraries and tools for my project and would be glad if you share any.