Skip to content

Extract a page

From the shell:

Terminal window
analog get https://quotes.toscrape.com/js/

One call turns the rendered page into structured records — every quote, with its author and tags — and prints a handle (like 20260719-k7m2p9) you can use to re-open, query, and export the result later without re-fetching.

In Python:

from analog import analog
result = analog("https://quotes.toscrape.com/js/")
for section in result.structured_content:
print(section.label, len(section.records))

Every result opens with a token-cheap orientation view — read it before pulling records:

Terminal window
analog open <handle>
pagination: this looks like page 1 of a paginated collection (its links reach page 2) — these records cover this page only. Analog can follow the site's own next links and merge the pages into one result; pass --pages 2 to analog get.
3 sections extracted.
section[0] 10 records · 3 fields
sample: tags=[4 values], text="“The world as we have created it is a process of our thinking. It cannot be chan…(+35 chars)", text_2="by Albert Einstein"
section[1] navigation 2 links (header)
section[2] navigation 2 links (footer)
Shortened values above: full via analog export <handle>.

Every section appears at its place in the page’s reading order — navigation sections included, one compact line each — so the section numbering never has gaps. The sample: line shows one record so you can see the schema carrying real values; multi-entry lists collapse to their count ([4 values]), and the full values stay in the records and exports. Long values are shortened for orientation with a compact …(+N chars) marker — never the page’s own — and one closing line names where the full values live; the stored value remains intact. When a page holds structure that wasn’t extracted, a page outline block reports every such region and what became of it (see Work with results). If the page needed more fetching (pagination, load-more, per-item expansion), see Fetching.

A page often has more than one kind of content — a storefront’s product grid plus a reviews carousel — so results are a list of sections, in page order. Each section carries records (list of dicts), fields (name → type and coverage), and its own markdown.

Analog never silently edits your data. When something is withheld, uncertain, or partial, the result says so:

  • Field names are marked as guesses when they are guesses. A semantically rich page names cleanly; a thin page falls back to positional names like text_2 — meaning the page gave the field no usable label, not that extraction failed. Rename in one batch with analog rename-fields <handle> text_2=title — and the rename is remembered for the URL, so future fetches of the same page arrive with your names already applied (conservatively: when the page’s fields have visibly changed shape, Analog skips the rename rather than guessing, and you rename once more). If the page merely reshuffles its own placeholder names while the column’s values stay put, your name follows the column. --once renames a single saved result without being remembered.
  • Withheld content is counted. Rows or columns that carried nothing readable are withheld with a disclosed count, never silently.
  • Structural odd-ones-out are flagged. A header row zipped in with data rows is pointed out by index, with a ready-made way to drop it. Dropping it changes the view, not the saved source result.
  • Pagination coverage is stated. If the page’s own markup shows it’s page 1 of 5, the result says these records cover this page only.
  • Values are never truncated by Analog. A value ending in is the page’s own visible truncation, preserved exactly.

Sponsored placements are data too: pages ship with ads, so ads come through as records — with the platform’s own label (“Sponsored”) preserved as a field value, so filtering them is a one-line choice:

Terminal window
analog export <handle> -f csv --where badge!=Sponsored

analog get <url> --mode local fetches the page and converts it to markdown entirely on your machine — no extraction, no backend call, no sign-in. Useful for a one-off read, or for judging a page before extracting it.