URL blocking / ad blocking

[alexanderadam]

The webdriver standard represents a part of the web testing activity. And developers want to test things that are common things that happen in the real life.

One of the most common things in the web is, that people control is the display of ads (or other elements / scripts such as webminers) with the help of adblockers like uBlock Origin and similar ones. Browsers like Firefox, Google Chrome and Chromium, Brave, Safari or Opera even have functionality like this integrated natively (the same goes for many testing solutions, i.e. when using Chrome 84 or newer or puppeteer).

Since this is so common, it should obviously be tested properly. Developers want to know how an application or website reacts when people are visiting it with adblock functionality enabled.
That is why I believe it should be integrated in the webdriver standard as well.

[andreastt]

I’m not entirely certain what this issue is regarding?

[alexanderadam]

I guess defining a standardized way of blocking elements in user agents.

I'm not sure how this would look like as well but @barancev suggested opening an issue for webdriver regarding content blocking and I thought you folks might rather know what to do. 😉

[andreastt]

We can certainly consider it, but I’m not sure there’s a good way of defining what it means to block a URL in web platform terms. It might be possible for resources loaded in HTML, but not generally for any UA resource.

[shs96c]

I'm not sure how meaningful this is. Most of the drivers out there allow you to install extensions (such as adblockers), so there's no need to bake it into this specification. There are some browsers implementing some form of cookie control or adblocking, and the developers of those browsers can use this spec "as is".

[hugobast]

Going to add to this conversation with the why? (for me). I think this blog post from ThoughtBot does a really good job of explaining why we might want this feature.

[JG127]

My use case is about blocking a ridiculously huge background image. The test suite has to download it over and over again.

[alexanderadam]

Most of the drivers out there allow you to install extensions (such as adblockers), so there's no need to bake it into this specification.

1. Well, Selenium is following the the W3C webdriver and doesn't integrate it, because it's not specified.
2. I would also be open for a specification that describes the integration of extensions. But I have the feeling that this might be much more complex.

[JG127]

What I could imagine is that the driver would map urls to different resources. This way the test could use a test version of JS, download lighter static content from a local source, similar to mocking by simulation of an external source...

[StephanBijzitter]

This would also make testing of offline functionality in PWAs much more easy. In my specific case, I've wrapped all my fetch calls into a helper function that looks a lot like: return window.__offline ? new Response(...) : fetch(...), but that doesn't stop external scripts, images or the service worker from using the network, so it's a bit... meh.

[Dr10s]

Just use --host-resolver-rules for chrome.

[alexanderadam]

Just use --host-resolver-rules for chrome.

This repository is not about certain ways to achieve this with a particular driver.
This repository is about defining WebDriver standard and it describes a platform- and language-neutral wire protocol to remotely instruct the behaviour of web browsers.

[alexanderadam]

It's 2024 now and I stopped using Selenium in all of my projects and switched to other drivers.
However, I think that this issue is still relevant — maybe now more than ever — especially since blocking ads is nowadays a widely used feature in pretty much every user agent.

Or how it's put in this article:

According to a survey of 2,000 Americans conducted by research firm Censuswide, on behalf of Ghostery, a maker of software to block ads and online tracking, 52 percent of Americans now use an ad blocker, up from 34 percent according to 2022 Statista data.

And while blocking ads was mainly just supported via extensions when I created this issue in 2018, it is nowadays even integrated natively in certain user agents.

[whimboo]

@alexanderadam In WebDriver BiDi we allow to add network interceptions which can be used to block certain network requests and as such for blocking elements to be loaded at all. Does that satisfy your needs?

Please note that such a feature cannot be done for the WebDriver classic protocol due to limitations of the protocol.

[whimboo]

I'm going to close this issue in favor of the mentioned network interception feature of WebDriver BiDi. If that doesn't fit your needs please explain what's missing. Thanks.

[alexanderadam]

It might be relevant for others that this might also overlap with webdriver bidi issue 403, which is about extensions and this is the usual way of having adblocking functionality in users browser.

It's still a pity that it took so long although browser extension were introduced with Firefox in 2004, there was no real webdriver specification for most of its existence until this PR in 2023.

But I guess better late than never. 😉

[whimboo]

Please note that there is also a way for WebDriver classic to install addons. It's just not part of the specification and each vendor handles it via extension commands. So the benefit with BiDi is that the command is now officially spec'ed.