-
Notifications
You must be signed in to change notification settings - Fork 47
refactor: reorganize documentation structure #850
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
df1c6dc
refactor: reorganize documentation structure
patrikbraborec eec8e45
fix: apply Prettier formatting to documentation files
patrikbraborec a17e3b2
Apply suggestion from @TC-MO
patrikbraborec 4ed3b92
fix: update placeholder token to MY-APIFY-TOKEN in introduction docs
patrikbraborec File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| --- | ||
| id: quick-start | ||
| title: Quick start | ||
| sidebar_label: Quick start | ||
| description: 'Get started with the Apify API client for JavaScript by running an Actor and retrieving results from its dataset.' | ||
| --- | ||
|
|
||
| Learn how to authenticate, run Actors, and retrieve results using the Apify API client for JavaScript. | ||
|
|
||
| --- | ||
|
|
||
| ## Step 1: Authenticate the client | ||
|
|
||
| To use the client, you need an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token under [Integrations](https://console.apify.com/account/integrations) tab in Apify Console. Copy the token and initialize the client by providing the token (`MY-APIFY-TOKEN`) as a parameter to the `ApifyClient` constructor. | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| // Client initialization with the API token | ||
| const client = new ApifyClient({ | ||
| token: 'MY-APIFY-TOKEN', | ||
| }); | ||
| ``` | ||
|
|
||
| :::warning Secure access | ||
|
|
||
| The API token is used to authorize your requests to the Apify API. You can be charged for the usage of the underlying services, so do not share your API token with untrusted parties or expose it on the client side of your applications. | ||
|
|
||
| ::: | ||
|
|
||
| ## Step 2: Run an Actor | ||
|
|
||
| To start an Actor, call the [`client.actor()`](/reference/class/ActorClient) method with the Actor's ID (e.g. `john-doe/my-cool-actor`). The Actor's ID is a combination of the Actor owner's username and the Actor name. You can run both your own Actors and Actors from Apify Store. | ||
|
|
||
| To define the Actor's input, pass a JSON object to the [`call()`](/reference/class/ActorClient#call) method that matches the Actor's [input schema](https://docs.apify.com/platform/actors/development/actor-definition/input-schema). The input can include URLs to scrape, search terms, or other configuration data. | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' }); | ||
|
|
||
| // Runs an Actor with an input and waits for it to finish. | ||
| const { defaultDatasetId } = await client.actor('username/actor-name').call({ | ||
| some: 'input', | ||
| }); | ||
| ``` | ||
|
|
||
| ## Step 3: Get results from the dataset | ||
|
|
||
| To get the results from the dataset, call the [`client.dataset()`](/reference/class/DatasetClient) method with the dataset ID, then call [`listItems()`](/reference/class/DatasetClient#listItems) to retrieve the data. You can get the dataset ID from the Actor's run object (represented by `defaultDatasetId`). | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' }); | ||
|
|
||
| // Runs an Actor and waits for it to finish | ||
| const { defaultDatasetId } = await client.actor('username/actor-name').call(); | ||
|
|
||
| // Lists items from the Actor's dataset | ||
| const { items } = await client.dataset(defaultDatasetId).listItems(); | ||
| console.log(items); | ||
| ``` | ||
|
|
||
| :::note Dataset access | ||
|
|
||
| Running an Actor might take time, depending on the Actor's complexity and the amount of data it processes. If you want only to get data and have an immediate response you should access the existing dataset of the finished [Actor run](https://docs.apify.com/platform/actors/running/runs-and-builds#runs). | ||
|
|
||
| ::: | ||
|
|
||
| ## Next steps | ||
|
|
||
| ### Concepts | ||
|
|
||
| To learn more about how the client works, check out the Concepts section in the sidebar: | ||
|
|
||
| - [Usage patterns](../02_concepts/01_usage-patterns.md) - resource clients, collection clients, and nested clients | ||
| - [Error handling and retries](../02_concepts/02_error-handling.md) - automatic retries with exponential backoff | ||
| - [Convenience functions](../02_concepts/03_convenience-functions.md) - `call()`, `waitForFinish()`, and more | ||
| - [Pagination](../02_concepts/04_pagination.md) - iterating through large result sets | ||
|
|
||
| ### Guides | ||
|
|
||
| For practical examples of common tasks, see [Code examples](../03_guides/01_examples.md). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| --- | ||
| id: usage-patterns | ||
| title: Usage patterns | ||
| sidebar_label: Usage patterns | ||
| description: 'Learn the resource client and collection client patterns used by the Apify API client for JavaScript.' | ||
| --- | ||
|
|
||
| The `ApifyClient` interface follows a generic pattern that applies to all of its components. By calling individual methods of `ApifyClient`, specific clients that target individual API resources are created. There are two types of those clients: | ||
|
|
||
| - [`ActorClient`](/reference/class/ActorClient): a client for the management of a single resource | ||
| - [`ActorCollectionClient`](/reference/class/ActorCollectionClient): a client for the collection of resources | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' }); | ||
|
|
||
| // Collection clients do not require a parameter. | ||
| const actorCollectionClient = client.actors(); | ||
| // Creates an actor with the name: my-actor. | ||
| const myActor = await actorCollectionClient.create({ name: 'my-actor-name' }); | ||
| // List all your used Actors (both own and from Apify Store) | ||
| const { items } = await actorCollectionClient.list(); | ||
| ``` | ||
|
|
||
| :::note Resource identification | ||
|
|
||
| The resource ID can be either the `id` of the said resource, or a combination of your `username/resource-name`. | ||
|
|
||
| ::: | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' }); | ||
|
|
||
| // Resource clients accept an ID of the resource. | ||
| const actorClient = client.actor('username/actor-name'); | ||
| // Fetches the john-doe/my-actor object from the API. | ||
| const myActor = await actorClient.get(); | ||
| // Starts the run of john-doe/my-actor and returns the Run object. | ||
| const myActorRun = await actorClient.start(); | ||
| ``` | ||
|
|
||
| ## Nested clients | ||
|
|
||
| Sometimes clients return other clients. That's to simplify working with nested collections, such as runs of a given Actor. | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' }); | ||
|
|
||
| const actorClient = client.actor('username/actor-name'); | ||
| const runsClient = actorClient.runs(); | ||
| // Lists the last 10 runs of your Actor. | ||
| const { items } = await runsClient.list({ | ||
| limit: 10, | ||
| desc: true, | ||
| }); | ||
|
|
||
| // Select the last run of your Actor that finished | ||
| // with a SUCCEEDED status. | ||
| const lastSucceededRunClient = actorClient.lastRun({ status: 'SUCCEEDED' }); | ||
| // Fetches items from the run's dataset. | ||
| const { items } = await lastSucceededRunClient.dataset().listItems(); | ||
| ``` | ||
|
|
||
| The quick access to `dataset` and other storage directly from the run client can be used with the [`lastRun()`](/reference/class/ActorClient#lastRun) method. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
| id: error-handling | ||
| title: Error handling and retries | ||
| sidebar_label: Error handling and retries | ||
| description: 'Handle API errors and configure automatic retries with exponential backoff in the Apify API client for JavaScript.' | ||
| --- | ||
|
|
||
| Based on the endpoint, the client automatically extracts the relevant data and returns it in the expected format. Date strings are automatically converted to `Date` objects. For exceptions, the client throws an [`ApifyApiError`](/reference/class/ApifyApiError), which wraps the plain JSON errors returned by API and enriches them with other contexts for easier debugging. | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' }); | ||
|
|
||
| try { | ||
| const { items } = await client.dataset('non-existing-dataset-id').listItems(); | ||
| } catch (error) { | ||
| // The error is an instance of ApifyApiError | ||
| const { message, type, statusCode, clientMethod, path } = error; | ||
| // Log error for easier debugging | ||
| console.log({ message, statusCode, clientMethod, type }); | ||
| } | ||
| ``` | ||
|
|
||
| ## Retries with exponential backoff | ||
|
|
||
| The client automatically retries requests that fail due to network errors, Apify API internal errors (HTTP 500+), or rate limit errors (HTTP 429). By default, the client retries up to 8 times with exponential backoff starting at 500ms. | ||
|
|
||
| To configure retry behavior, set the `maxRetries` and `minDelayBetweenRetriesMillis` options in the `ApifyClient` constructor: | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ | ||
| token: 'MY-APIFY-TOKEN', | ||
| maxRetries: 8, | ||
| minDelayBetweenRetriesMillis: 500, // 0.5s | ||
| timeoutSecs: 360, // 6 mins | ||
| }); | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| --- | ||
| id: convenience-functions | ||
| title: Convenience functions | ||
| sidebar_label: Convenience functions | ||
| description: 'Use call(), waitForFinish(), and other convenience functions in the Apify API client for JavaScript.' | ||
| --- | ||
|
|
||
| Some actions can't be performed by the API itself, such as indefinite waiting for an Actor run to finish (because of network timeouts). The client provides convenient `call()` and `waitForFinish()` functions that do that. If the limit is reached, the returned promise is resolved to a run object that will have status `READY` or `RUNNING` and it will not contain the Actor run output. | ||
|
|
||
| [Key-value store](https://docs.apify.com/platform/storage/key-value-store) records can be retrieved as objects, buffers, or streams via the respective options, dataset items can be fetched as individual objects or serialized data. | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' }); | ||
|
|
||
| // Starts an Actor and waits for it to finish. | ||
| const finishedActorRun = await client.actor('username/actor-name').call(); | ||
|
|
||
| // Starts an Actor and waits maximum 60s for the finish | ||
| const { status } = await client.actor('username/actor-name').start({ | ||
| waitForFinish: 60, // 1 minute | ||
| }); | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| --- | ||
| id: pagination | ||
| title: Pagination | ||
| sidebar_label: Pagination | ||
| description: 'Paginate through large result sets and use async iteration with the Apify API client for JavaScript.' | ||
| --- | ||
|
|
||
| Methods that return lists (such as `list()` or `listSomething()`) return a [`PaginatedList`](/reference/interface/PaginatedList) object. Exceptions include `listKeys()` and `listHead()`, which use different pagination mechanisms. | ||
|
|
||
| The list results are stored in the `items` property. Use the `limit` parameter to retrieve a specific number of results. Additional properties vary by method—see the method reference for details. | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client'; | ||
|
|
||
| const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' }); | ||
|
|
||
| // Resource clients accept an ID of the resource. | ||
| const datasetClient = client.dataset('dataset-id'); | ||
|
|
||
| // Maximum amount of items to fetch in total | ||
| const limit = 1000; | ||
| // Maximum amount of items to fetch in one API call | ||
| const chunkSize = 100; | ||
| // Initial offset | ||
| const offset = 0; | ||
|
|
||
| for await (const item of datasetClient.listItems({ limit, offset, chunkSize })) { | ||
| // Processs individual item | ||
| console.log(item); | ||
| } | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| --- | ||
| id: bundled-environments | ||
| title: Use in bundled environments | ||
| sidebar_label: Bundled environments | ||
| description: 'Use the Apify API client for JavaScript in browsers, Cloudflare Workers, and other edge runtimes.' | ||
| --- | ||
|
|
||
| :::warning Advanced | ||
|
|
||
| This applies only to non-Node.js environments (browsers, Cloudflare Workers, edge runtimes). If you're running on Node.js, you can skip this section. | ||
|
|
||
| ::: | ||
|
|
||
| The package ships a pre-built browser bundle that is automatically resolved when your bundler targets a browser environment. If it isn't picked up automatically, you can import it directly: | ||
|
|
||
| ```js | ||
| import { ApifyClient } from 'apify-client/browser'; | ||
| ``` | ||
|
|
||
| For Cloudflare Workers or other edge runtimes that don't provide Node built-ins, you may need to enable Node compatibility in your runtime config (e.g. `node_compat = true`. | ||
|
|
||
| Note that some Node-specific features (streaming APIs, proxy) are not available in the browser bundle. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.