SemanticScuttle - klotz.me

Tags: cloudflare* + crawler*

0 bookmark(s) - Sort by: Date ↓ / Title /

1. Cloudflare’s Markdown for Agents automatically make websites agent-ready

   Cloudflare converts HTML to Markdown on the fly when an AI agent requests it via the `Accept: text/markdown` header.

   2026-02-23 Tags: cloudflare, markdown, llm, html, crawler, scraper, scuttle, summarizer by klotz
2. Perplexity Says Cloudflare Is Blocking Legitimate AI Assistants

   Perplexity defends its AI assistants against Cloudflare’s claims, arguing that they are not web crawlers but user-triggered agents.

   2025-08-05 Tags: perplexity, cloudflare, ll., assistant, crawler, robots.txt, hallux by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0