Fetching

Analog renders pages in a local headless browser, so JS-built content is captured by default. Beyond that, pages hide content behind interactive controls — filters, load-more buttons, per-item toggles, pagination — and Analog gives you an explicit knob for each, with each default stated below.

Capture options

• reveal_all (default: on) — before capturing, a “show everything” control (an “All” filter, “Clear filters”) is clicked, so pages that default to a subset yield their full content. Opt out with --no-reveal-all to capture the page’s default view as-is.
• --load-all (default: off) — click a “Load more” / “Show more” button repeatedly until everything behind it is loaded. Off by default because large pages can take many presses.
• --expand-all (default: off) — expand per-item disclosure toggles (a card’s “+”, an accordion row) so detail the page folds away per item is captured. Expansion is paced politely and capped.
• --pages N (default: 1) — follow the page’s own pagination and merge the results, up to N pages (max 50). Analog follows the next link each page itself renders — never a constructed URL, and only on the page’s own host — pausing politely between fetches. The sweep is disclosed in the output: pages covered, why it stopped, duplicates dropped. Each page costs one extraction request. Pagination that operates in place — numbered buttons that swap the visible batch without changing the URL, common on hosted job boards — works too: --pages N drives the page’s own controls in the fetching browser and merges the batches by the same rules, and the result says which mechanism ran (page_sweep.mechanism).

Python: the same names as keyword arguments — analog(url, pages=5, expand_all=True).

Disclosures

When a control exists but wasn’t clicked (because its knob is off), it’s reported under browse_actions rather than silently ignored — so you learn the page has a load-more button, and can decide. Likewise pagination: a single-page fetch of a paginated collection says so (“this looks like page 1 … these records cover this page only”) with the --pages hint attached. And when the page states its own total the way pagination widgets do (“1-60 of 2,595”), the result reports it as pagination.stated_total — the page’s claim, shown beside Analog’s count, so a mismatch is information rather than a puzzle.

Analog clicks what the page offers every visitor, follows links the page itself renders, and reports what it chose not to do.