Skip to content

fix: playwright_stealth import and stealth application #1482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

fix: playwright_stealth import and stealth application #1482

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions crawl4ai/browser_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -649,16 +649,7 @@ async def start(self):
else:
from playwright.async_api import async_playwright

# Initialize playwright with or without stealth
if self.config.enable_stealth and not self.use_undetected:
# Import stealth only when needed
from playwright_stealth import Stealth
# Use the recommended stealth wrapper approach
self._stealth_instance = Stealth()
self._stealth_cm = self._stealth_instance.use_async(async_playwright())
self.playwright = await self._stealth_cm.__aenter__()
else:
self.playwright = await async_playwright().start()
self.playwright = await async_playwright().start()

if self.config.cdp_url or self.config.use_managed_browser:
self.config.use_managed_browser = True
Expand Down Expand Up @@ -783,6 +774,7 @@ async def setup_context(
14. Set default timeouts for navigation and download if enabled.
15. Set user agent if provided.
16. Set browser hints if provided.
17. Apply stealth if enabled.

Args:
context (BrowserContext): The browser context to set up
Expand Down Expand Up @@ -838,7 +830,15 @@ async def setup_context(
or crawlerRunConfig.simulate_user
or crawlerRunConfig.magic
):
await context.add_init_script(load_js_script("navigator_overrider"))
await context.add_init_script(load_js_script("navigator_overrider"))

# Apply stealth headers and script to the context
if self.config.enable_stealth and not self.use_undetected and self.config.browser_type == "chromium":
from playwright_stealth import stealth
prop = stealth.Properties(browser_type=stealth.BrowserType.CHROME)
stealth_script = stealth.combine_scripts(prop, None)
await stealth.generate_stealth_headers_async(prop, context)
context.add_init_script(stealth_script)

Comment on lines +835 to 842
Copy link
Contributor

@coderabbitai coderabbitai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

❓ Verification inconclusive

Two fixes for stealth application: await and coverage of existing pages.

  • Missing await on add_init_script (async API); without it, stealth might not register before page creation.
  • add_init_script affects new documents only. If the connected persistent context already has open pages, consider reloading them or documenting that callers should open new pages.
         if self.config.enable_stealth and not self.use_undetected and self.config.browser_type == "chromium":
             from playwright_stealth import stealth
             prop = stealth.Properties(browser_type=stealth.BrowserType.CHROME)       
             stealth_script = stealth.combine_scripts(prop, None)
             await stealth.generate_stealth_headers_async(prop, context)
-            context.add_init_script(stealth_script)    
+            await context.add_init_script(stealth_script)

Optional: reload existing pages after applying stealth to persistent contexts:

# after add_init_script
for p in context.pages:
    try:
        await p.reload(wait_until="domcontentloaded")
    except Exception:
        pass

Please confirm whether stealth must apply to already-open pages in managed/persistent sessions or if “new pages only” is acceptable for your use case.

Fix: await context.add_init_script and reload open pages in persistent contexts

  • Await add_init_script in the async flow — without await the init script may not be registered before pages are created.
  • add_init_script only runs for new/navigated documents; reload existing pages in persistent/connected contexts if you need stealth applied immediately.

File: crawl4ai/browser_manager.py (around lines 835–842)

         if self.config.enable_stealth and not self.use_undetected and self.config.browser_type == "chromium":
             from playwright_stealth import stealth
             prop = stealth.Properties(browser_type=stealth.BrowserType.CHROME)       
             stealth_script = stealth.combine_scripts(prop, None)
             await stealth.generate_stealth_headers_async(prop, context)
-            context.add_init_script(stealth_script)    
+            await context.add_init_script(stealth_script)

Optional: reload existing pages after add_init_script when using persistent/connected contexts:

# after add_init_script
for p in context.pages:
    try:
        await p.reload(wait_until="domcontentloaded")
    except Exception:
        pass

Confirm whether stealth must apply to already-open pages in persistent sessions or if "new pages only" is acceptable.

🤖 Prompt for AI Agents 
In crawl4ai/browser_manager.py around lines 835-842, the init script is added
without awaiting context.add_init_script and existing pages in
persistent/connected contexts are not reloaded, so stealth may not be applied to
already-open pages; change to await context.add_init_script(...) to ensure the
script is registered before pages are created, and if using persistent/connected
contexts iterate context.pages and reload each page (catch and ignore
exceptions) so the init script runs on existing documents — optionally gate the
reloads behind a check that the context is persistent/connected if stealth on
already-open pages is not required.

async def create_browser_context(self, crawlerRunConfig: CrawlerRunConfig = None):
"""
Expand Down