Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ apify_client = ApifyClient('MY-APIFY-TOKEN')
actor_call = apify_client.actor('john-doe/my-cool-actor').call()

# Fetch results from the Actor's default dataset
dataset_items = apify_client.dataset(actor_call['defaultDatasetId']).list_items().items
if actor_call is not None:
dataset_items = apify_client.dataset(actor_call.default_dataset_id).list_items().items
```

## Features
Expand All @@ -46,7 +47,7 @@ Based on the endpoint, the client automatically extracts the relevant data and r

### Retries with exponential backoff

Network communication sometimes fails. The client will automatically retry requests that failed due to a network error, an internal error of the Apify API (HTTP 500+) or rate limit error (HTTP 429). By default, it will retry up to 8 times. First retry will be attempted after ~500ms, second after ~1000ms and so on. You can configure those parameters using the `max_retries` and `min_delay_between_retries_millis` options of the `ApifyClient` constructor.
Network communication sometimes fails. The client will automatically retry requests that failed due to a network error, an internal error of the Apify API (HTTP 500+) or rate limit error (HTTP 429). By default, it will retry up to 4 times. First retry will be attempted after ~500ms, second after ~1000ms and so on. You can configure those parameters using the `max_retries` and `min_delay_between_retries` options of the `ApifyClient` constructor.

### Support for asynchronous usage

Expand Down
2 changes: 1 addition & 1 deletion docs/01_introduction/code/03_input_async.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ async def main() -> None:
'some': 'input',
}

# Start an Actor and waits for it to finish.
# Start an Actor and wait for it to finish.
call_result = await actor_client.call(run_input=run_input)
2 changes: 1 addition & 1 deletion docs/01_introduction/code/03_input_sync.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ def main() -> None:
'some': 'input',
}

# Start an Actor and waits for it to finish.
# Start an Actor and wait for it to finish.
call_result = actor_client.call(run_input=run_input)
3 changes: 1 addition & 2 deletions docs/01_introduction/index.mdx
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
---
id: introduction
title: Overview
sidebar_label: Overview
slug: /
description: "The official Python library to access the Apify API, with automatic retries, async support, and comprehensive API coverage."
description: The official Python library to access the Apify API, with automatic retries, async support, and comprehensive API coverage.
---

import Tabs from '@theme/Tabs';
Expand Down
3 changes: 1 addition & 2 deletions docs/01_introduction/quick-start.mdx
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
---
id: quick-start
title: Quick start
sidebar_label: Quick start
description: "Get started with the Apify API client for Python by running an Actor and retrieving results from its dataset."
description: Get started with the Apify API client for Python by running an Actor and retrieving results from its dataset.
---

import Tabs from '@theme/Tabs';
Expand Down
1 change: 1 addition & 0 deletions docs/02_concepts/01_async_support.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: asyncio-support
title: Asyncio support
description: Use the async client for non-blocking API calls with Python asyncio.
---

import Tabs from '@theme/Tabs';
Expand Down
1 change: 1 addition & 0 deletions docs/02_concepts/02_single_collection_clients.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: single-and-collection-clients
title: Single and collection clients
description: Understand the two types of resource clients, single-resource and collection clients.
---

import Tabs from '@theme/Tabs';
Expand Down
1 change: 1 addition & 0 deletions docs/02_concepts/03_nested_clients.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: nested-clients
title: Nested clients
description: Access related resources directly through nested client methods.
---

import Tabs from '@theme/Tabs';
Expand Down
1 change: 1 addition & 0 deletions docs/02_concepts/04_error_handling.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: error-handling
title: Error handling
description: Handle API errors with the ApifyApiError exception and automatic data parsing.
---

import Tabs from '@theme/Tabs';
Expand Down
9 changes: 5 additions & 4 deletions docs/02_concepts/05_retries.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: retries
title: Retries
description: Configure automatic retries with exponential backoff for failed requests.
---

import Tabs from '@theme/Tabs';
Expand All @@ -10,23 +11,23 @@ import CodeBlock from '@theme/CodeBlock';
import RetriesAsyncExample from '!!raw-loader!./code/05_retries_async.py';
import RetriesSyncExample from '!!raw-loader!./code/05_retries_sync.py';

When dealing with network communication, failures can occasionally occur. The Apify client automatically retries requests that fail due to:
The Apify client automatically retries requests that fail due to:

- Network errors
- Internal errors in the Apify API (HTTP status codes 500 and above)
- Rate limit errors (HTTP status code 429)

By default, the client will retry a failed request up to 8 times. The retry intervals use an exponential backoff strategy:
By default, the client retries a failed request up to 4 times. The retry intervals use an exponential backoff strategy:

- The first retry occurs after approximately 500 milliseconds.
- The second retry occurs after approximately 1,000 milliseconds, and so on.

You can customize this behavior using the following options in the [`ApifyClient`](/reference/class/ApifyClient) constructor:

- `max_retries`: Defines the maximum number of retry attempts.
- `min_delay_between_retries_millis`: Sets the minimum delay between retries (in milliseconds).
- `min_delay_between_retries`: Sets the minimum delay between retries as a `timedelta`.

Retries with exponential backoff are a common strategy for handling network errors. They help to reduce the load on the server and increase the chances of a successful request.
Retries with exponential backoff help reduce the load on the server and increase the chances of a successful request.

<Tabs>
<TabItem value="AsyncExample" label="Async client" default>
Expand Down
1 change: 1 addition & 0 deletions docs/02_concepts/06_logging.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: logging
title: Logging
description: Configure debug logging to inspect API requests and responses.
---

import Tabs from '@theme/Tabs';
Expand Down
6 changes: 4 additions & 2 deletions docs/02_concepts/07_convenience_methods.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: convenience-methods
title: Convenience methods
description: Use high-level methods for running Actors, waiting for results, and accessing data.
---

import Tabs from '@theme/Tabs';
Expand All @@ -12,8 +13,9 @@ import CallSyncExample from '!!raw-loader!./code/07_call_sync.py';

The Apify client provides several convenience methods to handle actions that the API alone cannot perform efficiently, such as waiting for an Actor run to finish without running into network timeouts. These methods simplify common tasks and enhance the usability of the client.

- [`ActorClient.call`](/reference/class/ActorClient#call) - Starts an Actor and waits for it to finish, handling network timeouts internally.
- [`ActorClient.start`](/reference/class/ActorClient#start) - Explicitly waits for an Actor run to finish with customizable timeouts.
- [`ActorClient.call`](/reference/class/ActorClient#call) - Starts an Actor and waits for it to finish, handling network timeouts internally. Waits indefinitely by default, or up to the specified `wait_duration`.
- [`ActorClient.start`](/reference/class/ActorClient#start) - Starts an Actor and immediately returns the Run object without waiting for it to finish.
- [`RunClient.wait_for_finish`](/reference/class/RunClient#wait_for_finish) - Waits for an already-started run to reach a terminal status.

Additionally, storage-related resources offer flexible options for data retrieval:

Expand Down
1 change: 1 addition & 0 deletions docs/02_concepts/08_pagination.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: pagination
title: Pagination
description: Paginate through large result sets using ListPage or generator-based iteration.
---

import Tabs from '@theme/Tabs';
Expand Down
1 change: 1 addition & 0 deletions docs/02_concepts/09_streaming.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: streaming-resources
title: Streaming resources
description: Stream large datasets, key-value store records, and logs without loading them into memory.
---

import Tabs from '@theme/Tabs';
Expand Down
1 change: 1 addition & 0 deletions docs/02_concepts/10_custom_http_clients.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: custom-http-clients
title: Custom HTTP clients
description: Replace the default HTTP client with a custom implementation.
---

import Tabs from '@theme/Tabs';
Expand Down
7 changes: 4 additions & 3 deletions docs/02_concepts/code/03_nested_async.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from apify_client import ApifyClientAsync
from apify_client._models import ActorJobStatus

TOKEN = 'MY-APIFY-TOKEN'

Expand All @@ -9,11 +10,11 @@ async def main() -> None:
actor_client = apify_client.actor('username/actor-name')
runs_client = actor_client.runs()

# List the last 10 runs of the Actor
# List the last 10 runs of the Actor.
actor_runs = (await runs_client.list(limit=10, desc=True)).items

# Select the last run of the Actor that finished with a SUCCEEDED status
last_succeeded_run_client = actor_client.last_run(status='SUCCEEDED') # ty: ignore[invalid-argument-type]
# Select the last run of the Actor that finished with a SUCCEEDED status.
last_succeeded_run_client = actor_client.last_run(status=ActorJobStatus.SUCCEEDED)

# Get dataset
actor_run_dataset_client = last_succeeded_run_client.dataset()
Expand Down
7 changes: 4 additions & 3 deletions docs/02_concepts/code/03_nested_sync.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from apify_client import ApifyClient
from apify_client._models import ActorJobStatus

TOKEN = 'MY-APIFY-TOKEN'

Expand All @@ -9,11 +10,11 @@ def main() -> None:
actor_client = apify_client.actor('username/actor-name')
runs_client = actor_client.runs()

# List the last 10 runs of the Actor
# List the last 10 runs of the Actor.
actor_runs = runs_client.list(limit=10, desc=True).items

# Select the last run of the Actor that finished with a SUCCEEDED status
last_succeeded_run_client = actor_client.last_run(status='SUCCEEDED') # ty: ignore[invalid-argument-type]
# Select the last run of the Actor that finished with a SUCCEEDED status.
last_succeeded_run_client = actor_client.last_run(status=ActorJobStatus.SUCCEEDED)

# Get dataset
actor_run_dataset_client = last_succeeded_run_client.dataset()
Expand Down
17 changes: 12 additions & 5 deletions docs/02_concepts/code/04_error_async.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
import asyncio

from apify_client import ApifyClientAsync
from apify_client.errors import ApifyApiError

TOKEN = 'MY-APIFY-TOKEN'

Expand All @@ -7,9 +10,13 @@ async def main() -> None:
apify_client = ApifyClientAsync(TOKEN)

try:
# Try to list items from non-existing dataset
dataset_client = apify_client.dataset('not-existing-dataset-id')
# Try to list items from a non-existing dataset.
dataset_client = apify_client.dataset('non-existing-dataset-id')
dataset_items = (await dataset_client.list_items()).items
except Exception as ApifyApiError:
# The exception is an instance of ApifyApiError
print(ApifyApiError)
except ApifyApiError as err:
# The client raises ApifyApiError for API errors.
print(f'API error: {err}')


if __name__ == '__main__':
asyncio.run(main())
15 changes: 10 additions & 5 deletions docs/02_concepts/code/04_error_sync.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from apify_client import ApifyClient
from apify_client.errors import ApifyApiError

TOKEN = 'MY-APIFY-TOKEN'

Expand All @@ -7,9 +8,13 @@ def main() -> None:
apify_client = ApifyClient(TOKEN)

try:
# Try to list items from non-existing dataset
dataset_client = apify_client.dataset('not-existing-dataset-id')
# Try to list items from a non-existing dataset.
dataset_client = apify_client.dataset('non-existing-dataset-id')
dataset_items = dataset_client.list_items().items
except Exception as ApifyApiError:
# The exception is an instance of ApifyApiError
print(ApifyApiError)
except ApifyApiError as err:
# The client raises ApifyApiError for API errors.
print(f'API error: {err}')


if __name__ == '__main__':
main()
4 changes: 2 additions & 2 deletions docs/02_concepts/code/07_call_async.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ async def main() -> None:
apify_client = ApifyClientAsync(TOKEN)
actor_client = apify_client.actor('username/actor-name')

# Start an Actor and waits for it to finish
# Start an Actor and wait for it to finish.
finished_actor_run = await actor_client.call()

# Starts an Actor and waits maximum 60s (1 minute) for the finish
# Start an Actor and wait up to 60 seconds for it to finish.
actor_run = await actor_client.start(wait_for_finish=60)
4 changes: 2 additions & 2 deletions docs/02_concepts/code/07_call_sync.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ def main() -> None:
apify_client = ApifyClient(TOKEN)
actor_client = apify_client.actor('username/actor-name')

# Start an Actor and waits for it to finish
# Start an Actor and wait for it to finish.
finished_actor_run = actor_client.call()

# Starts an Actor and waits maximum 60s (1 minute) for the finish
# Start an Actor and wait up to 60 seconds for it to finish.
actor_run = actor_client.start(wait_for_finish=60)
3 changes: 2 additions & 1 deletion docs/03_guides/01_passing_input_to_actor.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: passing-input-to-actor
title: Passing input to Actor
title: Pass input to an Actor
description: Run an Actor with custom input data and retrieve results.
---

import Tabs from '@theme/Tabs';
Expand Down
1 change: 1 addition & 0 deletions docs/03_guides/02_manage_tasks_for_reusable_input.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: manage-tasks-for-reusable-input
title: Manage tasks for reusable input
description: Create and manage tasks with reusable input configurations.
---

import Tabs from '@theme/Tabs';
Expand Down
1 change: 1 addition & 0 deletions docs/03_guides/03_retrieve_actor_data.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: retrieve-actor-data
title: Retrieve Actor data
description: Fetch, paginate, and merge datasets from Actor runs.
---

import Tabs from '@theme/Tabs';
Expand Down
3 changes: 2 additions & 1 deletion docs/03_guides/04_integration_with_data_libraries.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: integration-with-data-libraries
title: Integration with data libraries
title: Integrate with data libraries
description: Load Apify dataset items into Pandas DataFrames for analysis.
---

import Tabs from '@theme/Tabs';
Expand Down
3 changes: 2 additions & 1 deletion docs/03_guides/05_custom_http_client.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: custom-http-client-httpx
title: Using HTTPX as the HTTP client
title: Use HTTPX as the HTTP client
description: Replace the default Impit HTTP client with one based on HTTPX.
---

import ApiLink from '@site/src/components/ApiLink';
Expand Down
1 change: 1 addition & 0 deletions docs/04_upgrading/upgrading_to_v2.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: upgrading-to-v2
title: Upgrading to v2
description: Breaking changes and migration guide from v1 to v2.
---

This page summarizes the breaking changes between Apify Python API Client v1.x and v2.0.
Expand Down
1 change: 1 addition & 0 deletions docs/04_upgrading/upgrading_to_v3.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
id: upgrading-to-v3
title: Upgrading to v3
description: Breaking changes and migration guide from v2 to v3.
---

This page summarizes the breaking changes between Apify Python API Client v2.x and v3.0.
Expand Down