We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 9af558c commit 0d9521eCopy full SHA for 0d9521e
README.md
@@ -47,7 +47,6 @@ Install [Python 3](https://www.python.org/downloads/) and the requirements.
47
48
1. Use Selenium headless (maybe using PhantomJS).
49
2. Add support for cookies and basic auth.
50
-3. Add support for custom user agents.
51
-4. Add support for SEO url injection.
52
-5. Create the crawler (including support for POST requests).
53
-6. Document all code.
+3. Add support for SEO url injection.
+4. Implement [Not Your Average Web Crawler](https://github.com/tijme/not-your-average-web-crawler).
+5. Document all code.
0 commit comments