How do I install rs-trafilatura for Scrapy?

`pip install rs-trafilatura scrapy`. Add `ITEM_PIPELINES = {"rs_trafilatura.scrapy.RsTrafilaturaPipeline": 300}` to settings.py. Yield items with 'body' or 'html'.

What does rs-trafilatura extract from web pages?

Title, author, date, main content (text or Markdown), page type (article/forum/product), quality score, language, site name, description.

Is rs-trafilatura faster than other Scrapy extractors?

Yes—~44ms/page via Rust. Beats Python alternatives, negligible vs. network latency.

🛠️ Developer Tools

Rust Sneaks into Scrapy: rs-trafilatura's Pipeline That Scrapers Actually Need

Scrapy crawlers have limped along with pokey extractors for years. rs-trafilatura drops in Rust horsepower, turning raw HTML into gold without breaking a sweat.

Open Source Beat Apr 03, 2026 3 min read 14 views

Diagram of Scrapy spider pipeline with rs-trafilatura extraction

⚡ Key Takeaways

Zero-config pipeline adds rich extraction to any Scrapy item with HTML. 𝕏
Rust speed (44ms/page) + page types/quality scores for smarter pipelines. 𝕏
Drops junk automatically; exports to JSONL for easy downstream processing. 𝕏

Published by

Open Source Beat

Community-driven. Code-first.

#rs-trafilatura #rust python #scrapy #web scraping

Worth sharing?

Get the best Open Source stories of the week in your inbox — no noise, no spam.

Originally reported by Dev.to

⚡ Key Takeaways

The 60-Second TL;DR

Open Source Beat

Share this article

Worth sharing?

Related Stories

rs-trafilatura Supercharges Crawl4AI: 1.7% F1 Boost on Real-World Benchmarks

rs-trafilatura Unlocks Firecrawl's Hidden Precision

rs-trafilatura Cracks Web Scraping's Non-Article Nightmare

Rust's Dynamic Duo: rs-trafilatura Turbocharges spider-rs Crawls

Stay in the loop