Chapter 5 of Bad Computers by Michael Veale and Robert Gorwa

Hosting: Moderating Third-Party Models

There are now many hosting platforms which allow lay or professional users to access, download and sometimes also tweak and manipulate various AI models. Since generative AI systems easily produce both illicit and legitimate material, and because hosts generally do not have visibility into the content generated by those systems (which can be downloaded and run locally or deployed in a cloud environment), it is difficult for hosts to effectively deal with the plethora of ways in which they are implicated in AI-powered intimate image abuse and other thriving misuse ecosystems. We begin by tracing the history of AI hosting platforms from early software development repositories to today’s big players like Hugging Face, examining the legal and technical challenges facing hosting platforms seeking to do moderation in good faith. We then offer case studies demonstrating the distinct challenges facing three different types of hosting platform: general-purpose AI hosts like Hugging Face, specialized image generation platforms like Civitai and app stores increasingly needing to deal with AI apps or apps with AI integrations. While this layer of the stack is characterized by some wicked challenges (we focus in particular on the dual-use nature of AI models, the imperfect information that platforms need to deal with when making judgments about specific models in a sea of content and the ability of motivated actors to obfuscate misuse capabilities), we conclude by outlining the ample low-hanging fruit that could be picked if host actors brought on the requisite capacity and expertise to proactively deal with the most obvious forms of abuse.

Download chapter 5 as a PDF (pages 116–146, 536 KB).

How to cite

Michael Veale and Robert Gorwa, Bad Computers: How the Wider AI Stack Can Stop the Worst AI Misuse (MIT Press forthcoming 2027) ch 5.

Citation data for reference managers: BibTeX or RIS. This is a preprint of a book forthcoming from MIT Press in 2027. Page numbers follow this preprint and will differ from the published edition.