Defuddle

Extract the main content and metadata from a web page with the defuddle CLI. Defuddle strips navigation, ads, and boilerplate, returning readable article content rather than a raw HTML dump. Sources can be a URL, a local HTML file, or stdin.

This skill depends on the defuddle CLI (npm package defuddle). When it is not installed, or when the result feeds something durable (archival snapshots, extractions compared over time), run it through npx with a pinned version so results stay reproducible:

npx --yes [email protected] parse <source>

A locally installed defuddle is fine for one-off extractions; record its version when the output is kept.

Choose the output format

Need	Command
Readable Markdown	`defuddle parse <source> --markdown`
Content plus all metadata	`defuddle parse <source> --json`
One metadata field	`defuddle parse <source> --property <name>`
Complete page snapshot	`defuddle parse <source> --frontmatter --markdown --output <file>`

--markdown (-m) prints extracted content as Markdown. Tables become Markdown tables and code fences survive conversion.
--json (-j) is the discovery mode: one JSON object with content and every metadata field. Inspect it before pulling individual properties.
--property (-p) names are the JSON keys: title, description, author, published, site, domain, language, image, favicon, wordCount, content, contentMarkdown. They are case-sensitive camelCase — --property word_count fails with “Property not found”; use wordCount.
--frontmatter (-f) prepends YAML frontmatter (title, site, source, domain, language, word_count) to the content. Combined with --markdown and --output (-o), this produces a complete page snapshot in one step.

Other options: --user-agent <string> (-u) for servers that reject the default agent, and --lang <code> (-l) as a BCP 47 language hint for multilingual sites.

Read from stdin when the page needs to be fetched another way (cookies, custom headers, curl):

curl -sL <url> | defuddle parse - --markdown

Verify the extraction before using it

Treat the output as a claim to check, not a completed fetch.

Fetch failures print Error: ... but exit 0 — the exit code is not a success signal. Check the output itself.
Confirm real content arrived: a non-empty body, a plausible wordCount, and the expected topic. A result that is empty, a single line, or a login/cookie-consent/paywall notice means extraction failed — report that instead of summarizing the stub as the source.
JavaScript-rendered pages (SPAs) often return little or no content. Try once; if the result is clearly incomplete, report the limitation and fall back to another route (the page’s native Markdown or API, a reader-mode proxy, or asking the user) rather than presenting a partial extraction as complete.
On 403/FORBIDDEN, retry once with --user-agent set to a browser string. If it still refuses, report the failure with the URL.

Boundaries

HTML only. For PDFs, JSON APIs, or pages behind authentication, use an appropriate tool instead of forcing defuddle.
When serving as the extraction step for a wiki-ingest URL snapshot, the wiki-ingest workflow stays in charge: provenance, retrieval date, and commit discipline belong to it. Record the defuddle version as the extraction method alongside any omissions.