Why not just rate limit every user to realistic human rates. You just punish any... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		karn97 on April 2, 2025 \| parent \| context \| favorite \| on: Wikipedia is struggling with voracious AI bot craw... Why not just rate limit every user to realistic human rates. You just punish anyone behaving like a bot.

mrweasel on April 2, 2025 | [–]

Because, as pointed out in another post about the same problem: Many of these scrappers make one or two requests from one IP and then move on.

guerrilla on April 2, 2025 | [–]

Sold. Pay by page retrieval rate.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact