The command line

The oxy command runs jobs from a shell, through the same Session that the library uses.

Install

uv tool install 'oxyscraper[cli]'

This installs two commands, oxy and oxyscraper, which do the same thing. uv add oxyscraper installs the library alone, and its commands then print the install command above.

oxy reads the credentials of a Web Scraper API (Classic) user from OXY_WSA_USERNAME and OXY_WSA_PASSWORD. No option takes the password, so it never lands in your shell history, and oxy loads no .env file.

Run jobs

oxy run SOURCE INPUT... runs one job per input, and prints each done job to stdout as one line of JSON, the body that the API returned. So a redirect writes a file that pl.read_ndjson reads:

oxy run universal https://sandbox.oxylabs.io/products/1 https://sandbox.oxylabs.io/products/2 > results.ndjson

Running 2 payloads with Push-Pull
Finished 2 payloads in 1.00s: 2 done

An input with :// goes in url, and any other input in query, so most sources need no flag. -k names another input key, such as -k product_id.

Each parameter that oxy.Payload types has its own option, which oxy run --help lists with its allowed values. -p KEY=VALUE sets any other parameter as a string, and -p KEY:=JSON sets a JSON value. --dry-run prints each payload and the most results they can bill, and sends nothing:

oxy run amazon_search 'standing desk' 'desk lamp' --domain de --pages 2 -p sort_by=price_low_to_high --dry-run

Would submit 2 jobs, which bill at most 4 results
{"source": "amazon_search", "query": "standing desk", "pages": 2, "domain": "de", "context": [{"key": "sort_by", "value": "price_low_to_high"}]}
{"source": "amazon_search", "query": "desk lamp", "pages": 2, "domain": "de", "context": [{"key": "sort_by", "value": "price_low_to_high"}]}

oxy builds each payload with its source’s model when one exists, so a misspelt key or value exits with code 2 before any request:

oxy run amazon_product B07FZ8S74R -p domian=de

error: Invalid value: input B07FZ8S74R: domian: Extra inputs are not permitted

Usage: oxy run [OPTIONS] [source] [inputs]...

Read stdin

Without inputs, oxy run SOURCE reads one input per line of stdin. Without a source, it reads one payload per line, as JSON in the API’s shape, so a dry run’s output feeds a later run:

oxy run universal --dry-run < urls.txt > payloads.ndjson

Would submit 4 jobs, which bill at most 4 results

oxy run -d results < payloads.ndjson

Running 4 payloads with Push-Pull
Finished 4 payloads in 1.00s: 4 done, 4 written

-d writes each done job to <job_id>.json in a folder or a bucket instead of stdout, as destination does in Destinations and run logs. A URL such as gs://bucket/path names a bucket.

Recover a run

--run-log DIR writes a run log, one line per payload with its state, job ID and payload:

oxy run universal -d results --run-log logs < urls.txt

Running 4 payloads with Push-Pull
warning: Stopped checking job 7500000000000000007, because the API returned 404 Not Found: universal https://sandbox.oxylabs.io/products/1
warning: Job 7500000000000000009 faulted: universal https://sandbox.oxylabs.io/products/3
Finished 4 payloads in 1.00s: 2 done, 1 faulted, 1 unfetched, 2 written
Wrote the run log to logs/20261005T165658.150Z.jsonl

The run exits with code 1, because a job faulted and oxy stopped checking another. jq reads the run log, so two commands recover the run without paying twice. The first resubmits the faulted payloads:

jq -c 'select(.state == "faulted") | .payload' logs/20261005T165658.150Z.jsonl | oxy run -d results

Running 1 payload with Push-Pull
Finished 1 payload in 1.00s: 1 done, 1 written

The second fetches the unfetched jobs, which may have finished since:

jq -r 'select(.state == "unfetched") | .id' logs/20261005T165658.150Z.jsonl | oxy get -d results

oxy get JOB_ID... takes IDs as arguments or one per line of stdin. For a job that is still pending, faulted or whose results expired, it writes a warning and exits with code 1.

The run log redacts the credentials in a storage_url. So a tos or s3_compatible payload needs its secret again before a resubmission.

Output

stdout holds JSON only, and stderr holds every message, in uv’s style. A warning starts with warning:, a hint with hint: and an error with error:. On a terminal, a live line shows the run’s progress, and elsewhere a plain progress line prints every 10 seconds. -v adds a debug line for each change of state and each retry.

The exit code tells the outcome:

Code	Meaning
0	Every job is done.
1	A job faulted, or the run was incomplete.
2	An option, an input or a payload was invalid, or a credential is missing.
130	Ctrl+C stopped the run.
143	SIGTERM stopped the run, which oxy handles as Ctrl+C, so the run log is still written.