Skip to content

Extract a page

From the shell:

Terminal window
analog get https://quotes.toscrape.com/js/

One call turns the rendered page into structured records — every quote, with its author and tags — and prints a handle (like 20260719-k7m2p9) you can use to re-open, query, and export the result later without re-fetching.

In Python:

from analog import analog
result = analog("https://quotes.toscrape.com/js/")
for section in result.structured_content:
print(section.label, len(section.records))

Every result opens with a token-cheap orientation view — read it before pulling records:

Terminal window
analog open <handle>
pagination: this looks like page 1 of a paginated collection (its links
reach page 2) — these records cover this page only. Analog can follow
the site's own next links and merge the pages into one result:
pages=N (CLI: --pages N).
1 section extracted.
structured_content[0] 11 records of 11 detected items · 4 fields
note: record 1 of 11 diverges from this section's structure — possibly
page chrome rather than a data row (review with `analog open <handle>
--complete`; drop by index after export)
first: text="Login", text_2="Quotes to Scrape", tags_url=["https://quotes.toscrape.com/"]
last: text="by Steve Martin", text_2="“A day without sunshine is like, you know, night.”", tags=["humor", "obvious", "simile"]
page outline:
structure: footer
navigation:
footer (2 links): GoodReads.com · Zyte

That’s real output. The first: / last: bounds give you a section’s range at a glance (here they expose record 1 as page chrome and the last record as real data), the notes carry Analog’s disclosures, and the pagination line states coverage — with the knob that extends it. The closing blocks are the page’s own map: the outline reports every significant region and what became of it, and navigation lists the page’s nav links — both queryable on the saved result (see Results). If the page needed more fetching (pagination, load-more, per-item expansion), see Fetching.

A page often has more than one kind of content — a storefront’s product grid plus a reviews carousel — so results are a list of sections, in page order. Each section carries records (list of dicts), fields (name → type and coverage), and its own markdown.

Analog never silently edits your data. When something is withheld, uncertain, or partial, the result says so:

  • Field names are marked as guesses when they are guesses. A semantically rich page names cleanly; a thin page falls back to positional names like text_2 — meaning the page gave the field no usable label, not that extraction failed. Rename in one batch with analog rename-fields <handle> text_2=title — and the rename is remembered for the URL, so future fetches of the same page arrive with your names already applied (conservatively: when the page’s fields have visibly changed shape, Analog skips the rename rather than guessing, and you rename once more). If the page merely reshuffles its own placeholder names while the column’s values stay put, your name follows the column. --once renames a single saved result without being remembered.
  • Withheld content is counted. Rows or columns that carried nothing readable are withheld with a disclosed count, never silently.
  • Structural odd-ones-out are flagged. A header row zipped in with data rows is pointed out by index, with a ready-made way to drop it. Dropping it changes the view, not the saved source result.
  • Pagination coverage is stated. If the page’s own markup shows it’s page 1 of 5, the result says these records cover this page only.
  • Values are never truncated by Analog. A value ending in is the page’s own visible truncation, preserved exactly.

Sponsored placements are data too: pages ship with ads, so ads come through as records — with the platform’s own label (“Sponsored”) preserved as a field value, so filtering them is a one-line choice:

Terminal window
analog export <handle> -f csv --where badge!=Sponsored

analog get <url> --mode local fetches the page and converts it to markdown entirely on your machine — no extraction, no backend call, no sign-in. Useful for a one-off read, or for judging a page before extracting it.