AsyncSession

Submit payloads and fetch their jobs inside an async with block, on asyncio or trio.

Usage

Source

AsyncSession(
    *,
    username,
    password,
    transport=None,
    retry_limit=600.0,
    pending_limit=600.0,
    progress_interval=10.0
)

The block holds the task group that submits and checks while the caller’s loop body runs. Leaving it stops every unfinished run.

Parameters

username: str

The API user’s username.

password: str

The API user’s password.

transport: httpx2.AsyncBaseTransport | None = None

The transport of every request, such as a FakeOxylabs. None uses the fake of the innermost open with FakeOxylabs() block, or else Oxylabs over HTTP/2.

retry_limit: float = 600.0

The seconds after a request’s first failure until it stops retrying.

pending_limit: float | None = 600.0

The seconds after a job’s acceptance until oxy stops checking it. None waits without limit.

progress_interval: float | None = 10.0

The seconds between progress lines. None turns the lines off.

Raises

ValueError

If username or password is empty.

Methods

Name	Description
__aenter__()	Open the client and the task group that runs the runs.
__aexit__()	Stop every unfinished run, and close the client.
execute()	Run the payloads, and return the run once its last job finishes.
get()	Return a job as it stands, at once.
stream()	Start a run, and return its jobs as they finish.

---

__aenter__()

Open the client and the task group that runs the runs.

Usage

Source

__aenter__()

---

__aexit__()

Stop every unfinished run, and close the client.

Usage

Source

__aexit__(*exc_info)

A second Ctrl+C while the runs stop makes them stop at once, without their run logs.

---

execute()

Run the payloads, and return the run once its last job finishes.

Usage

Source

execute(
    payloads,
    *,
    realtime=False,
    destination=None,
    run_log=None,
    check_storage=True,
    output_types=()
)

It takes the arguments of stream.

---

get()

Return a job as it stands, at once.

Usage

Source

get(job_id, *, output_types=())

A pending job has empty results, and so has a finished job whose results expired.

Parameters

job_id: str

The job’s ID.

output_types: Sequence[_OutputType] = ()

The output types of each result, as in stream.

Raises

OxylabsError

If the API returns an error, or a network failure lasts past the retry limit.

---

stream()

Start a run, and return its jobs as they finish.

Usage

Source

stream(
    payloads,
    *,
    realtime=False,
    destination=None,
    run_log=None,
    check_storage=True,
    output_types=()
)

Parameters

payloads: Payload | Iterable[Payload]

One payload or several, each submitted as one job.

realtime: bool = False

Submit each payload through Realtime, one request per payload, instead of Push-Pull. oxy never falls back from one integration method to the other.

destination: str | os.PathLike[str] | ObjectStore | None = None

The folder that oxy writes each done job’s body to, unchanged, as <job_id>.json, before the run yields the job. oxy passes a string with :// to obstore.store.from_url, which reads credentials from the environment as Polars does. Any other string or path names a local folder, which oxy creates. Pass a store for explicit credentials or settings. A failed write stops submission and every later write.

run_log: str | os.PathLike[str] | ObjectStore | None = None

The folder that oxy writes the run log to, as <run start>.jsonl, with one JSON line per payload in input order. It takes the same types as destination. oxy writes an empty file before the first submission, and replaces it when the run ends or stops. So an empty file means a second Ctrl+C or a killed process, and jobs may have billed without a record.

check_storage: bool = True

Submit the first payload of each storage_url alone, and hold the rest of that storage_url until the first upload succeeds, so a wrong bucket bills one job. If that upload fails, the held payloads stay unsubmitted. A job object without statuses cannot show the upload, so the rest go out once the first job finishes. False submits every payload at once, and oxy still checks each upload.

output_types: Sequence[_OutputType] = ()

The output types of each result, sent as the API’s type parameter. Empty returns the default type of each job.

Raises

ValueError

If realtime or destination is set and a payload sets storage_type.

obstore.exceptions.BaseError

If oxy cannot list destination or run_log, or cannot write the empty run log.