Chapter 4 of Bad Computers by Michael Veale and Robert Gorwa

Network: Scraping Sites and Crawling Agents

Modern AI systems, including agentic systems, interact with a wide variety of software tools that allow interfacing with external services and data sources, often via Web scraping or API access. Such tools are also part of misuse pipelines, for example by powering convincing scams or fraud. Powerful intermediaries exist at the network layer between the AI system and these third-party services, most notably among them infrastructural firms such as Cloudflare. In order to respond to the rapid expansion of scraping for AI training and use, these intermediaries have been positioning themselves as new gatekeepers, with bot detection tools such as CAPTCHAs and new cryptographic bot authentication tools. They have been using their market leverage to reshape standards. While these efforts have to date been focused on protecting creative content, news and server load, the same tools may have a powerful role in throttling misuse. At the same time, such infrastructural reshaping comes at a high cost — endangering the openness of the Internet itself.

Download chapter 4 as a PDF (pages 93–115, 452 KB).

How to cite

Michael Veale and Robert Gorwa, Bad Computers: How the Wider AI Stack Can Stop the Worst AI Misuse (MIT Press forthcoming 2027) ch 4.

Citation data for reference managers: BibTeX or RIS. This is a preprint of a book forthcoming from MIT Press in 2027. Page numbers follow this preprint and will differ from the published edition.