Skip to content

Commit 9ef8bd9

Browse files
committed
updated scraping options
1 parent 8643231 commit 9ef8bd9

File tree

2 files changed

+7
-2
lines changed

2 files changed

+7
-2
lines changed

docs/docs/gpt-researcher/gptr/scraping.md

+6-1
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ You can choose your preferred scraping method by setting the `SCRAPER` environme
1717
```
1818

1919
3. For **production** use cases, you can set the Scraper to `tavily_extract`. [Tavily](https://tavily.com) allows you to scrape sites at scale without the hassle of setting up proxies, managing cookies, or dealing with CAPTCHAs. Please note that you need to have a Tavily account and [API key](https://app.tavily.com) to use this option. To learn more about Tavily Extract [see here](https://docs.tavily.com/docs/python-sdk/tavily-extract/getting-started).
20+
Make sure to first install the pip package `tavily-python`. Then:
2021
```
2122
export SCRAPER="tavily_extract"
2223
```
@@ -80,7 +81,11 @@ Benefits:
8081
Setup:
8182
1. Create a Tavily account at [app.tavily.com](https://app.tavily.com)
8283
2. Get your API key from the dashboard
83-
3. Set your Tavily API key:
84+
3. Install the Tavily Python SDK:
85+
```bash
86+
pip install tavily-python
87+
```
88+
4. Set your Tavily API key:
8489
```bash
8590
export TAVILY_API_KEY="your-api-key"
8691
```

setup.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
from setuptools import find_packages, setup
22

3-
LATEST_VERSION = "0.10.7"
3+
LATEST_VERSION = "0.10.10"
44

55
exclude_packages = [
66
"selenium",

0 commit comments

Comments
 (0)