0 bookmark(s) - Sort by: Date ↓ / Title /
The article discusses the author's experience with Amazon's FriendlyCrawler, which overloaded the author's website by crawling at a very high interval. The author criticizes the crawler's disregard for robots.txt and provides solutions for blocking such traffic using CloudFlare.
First / Previous / Next / Last
/ Page 1 of 0