So this is tailored towards kind of a "reader view" for models right? Can it handle images, tables, shadow DOMs too? Like there are 3 use cases I have now - one is a simple text view for models to understand it, one is a "web clip" mode which would ideally preserve images and media, and one is to extract tabular data from web pages. Which ones is this good at?
Images pass through as they are considered main content. Same with tables.
Pulpie will return all main content on a page as HTML/Markdown. I’m not sure I fully understand “which one this is good at?”. perhaps you can try the model on hugging face and let me know if the results look good?
It's good looking, and I liked it. The trial page accessed from the hugging face website is a very inefficient experience when I use Mozilla and the dark theme, FYI.
We see far better performance with models. Heuristics break on richer content like codeblocks, formulae, quotes, etc. In our testing, our model was 25 F1 points better than Trafilatura.
If I had to reckon, it's because the web comes in very many shapes, and outsourcing that work to a generalist LLM/SLM like GPT Nano is expensive, and doing it deterministically will never catch all the edge cases as well as a purpose-built encoder when run at webscale.
Looks like they are including Trafilatura in the comparison tables, which I've used before with pretty decent results, but it still has trouble with some pages. Looks like the pulpie f1 scores are quite a bit better, especially for the hard cases.
Would be curious how it runs on more modest hardware though, I'm using it for a small bookmark archiving tool and being able to run it on my small mini-pc homelab would be nice.
Why does the 'Quality vs Cost of Web Content Extraction' chart not have zero cost at the origin? Up to the right does not have to mean better; we can read.
I did some research on this about 10 years ago. I spent 2 days hand labelling data from scraped news sites. Then built a good old fashioned Random Forest model to classify html nodes based on some feature engineering. turns out the P tag and the number-of-words threshold get you 90% of the way there, on news sites anyway. Great thing about RF models is they tell you which features are the most important. fun little project (apart from the 2 days of data labelling).
ongrats on the release! The architectural insight to switch from a bandwidth-bound decoder (token-by-token generation) to a compute-bound encoder (single forward pass over 8k chunks) is brilliant—the 20x speedup and cost drop from $159k to $7.9k per billion pages is massive for web-scale pipelines.
As someone building AI developer utilities and document tools, I have two quick technical questions:
How well does the <|sep|> block-marker architecture handle heavily obfuscated HTML or adversarial SEO spam where boilerplate is styled to look like semantic body text?
Have you tested running pulpie-orange-small (210M) quantized (e.g., INT8/FP8) on consumer edge GPUs or CPU-only setups for local RAG pipelines?
Amazing work on open-sourcing the teacher and distilled weights on Hugging Face!
We haven't run a targeted eval against SEO spam yet. However, with Pulpie, each block gets labeled by what the text actually says rather than what the tags look like. Wrapping boilerplate in semantic tags fools rule based extractors precisely because they judge structure. Pulpie doesn't. The closest benchmark we have for this is the WebMainBench difficulty split, where pulpie-orange-small holds 0.813 on the hard subset. For comparison, trafilatura scores a 0.526.
For quantization, we haven't benchmarked INT8 or FP8. Everything in the post ran on L4 and A100. That said, I expect it to go well for a few reasons. It's a single forward pass over the page, so the workload is compute bound rather than bandwidth bound, which is why the L4 held up so well against the A100 and why cheaper cards should degrade gracefully. At 210M params the small model is roughly 420MB in FP16 and half that in INT8. So it fits on any consumer GPU with room to spare. Also, one pass classification tends to survive 8 bit quantization better than autoregressive generation since there is no error accumulation across decode steps.
Pulpie will return all main content on a page as HTML/Markdown. I’m not sure I fully understand “which one this is good at?”. perhaps you can try the model on hugging face and let me know if the results look good?
https://huggingface.co/spaces/feyninc/pulpie
Would be curious how it runs on more modest hardware though, I'm using it for a small bookmark archiving tool and being able to run it on my small mini-pc homelab would be nice.
As someone building AI developer utilities and document tools, I have two quick technical questions:
How well does the <|sep|> block-marker architecture handle heavily obfuscated HTML or adversarial SEO spam where boilerplate is styled to look like semantic body text? Have you tested running pulpie-orange-small (210M) quantized (e.g., INT8/FP8) on consumer edge GPUs or CPU-only setups for local RAG pipelines? Amazing work on open-sourcing the teacher and distilled weights on Hugging Face!
We haven't run a targeted eval against SEO spam yet. However, with Pulpie, each block gets labeled by what the text actually says rather than what the tags look like. Wrapping boilerplate in semantic tags fools rule based extractors precisely because they judge structure. Pulpie doesn't. The closest benchmark we have for this is the WebMainBench difficulty split, where pulpie-orange-small holds 0.813 on the hard subset. For comparison, trafilatura scores a 0.526.
For quantization, we haven't benchmarked INT8 or FP8. Everything in the post ran on L4 and A100. That said, I expect it to go well for a few reasons. It's a single forward pass over the page, so the workload is compute bound rather than bandwidth bound, which is why the L4 held up so well against the A100 and why cheaper cards should degrade gracefully. At 210M params the small model is roughly 420MB in FP16 and half that in INT8. So it fits on any consumer GPU with room to spare. Also, one pass classification tends to survive 8 bit quantization better than autoregressive generation since there is no error accumulation across decode steps.