Aileen Nielsen, Too Accurate A.I., 2024 Mich. St. L. Rev. 425 (2025).
Abstract: Legal scholarship and regulatory proposals for artificial intelligence (AI) have primarily focused on detecting or preventing AI misbehavior or mistakes. The proliferation of high-performance generative AI, such as ChatGPT and DALL-E, has quite saliently demonstrated that legal problems and policy challenges can also emerge when AI applications perform their assigned tasks too well rather than too poorly, as when humans improperly or unethically rely on AI to undertake tasks they are expected to do themselves or when ready access to high performance AI undermines demand for human services and thereby causes economic disruption. Beyond potential legal or policy challenges, this Article makes a stronger claim, arguing that the increasingly common phenomenon of too accurate AI could imminently create substantial particularized harms to individuals as well as widely dispersed costs to society. Recognizing high AI performance as a potential source of harm is an essential step towards better design and regulation of AI, particularly in horizontal regulatory initiatives such as the EU's AI Act and recent U.S. initiatives and proposals. The Article proceeds in four parts. Part I provides a motivating example of how accuracy is associated with both the problems and also the proposed solutions for a contested but common algorithmic practice: content recommendation. Part II shows how AI accuracy can undercut widely shared normative values, relating this observation to findings from digital ethics, economics, and computer science. Part III examines two recent federal legislative proposals and a recent executive action aimed at taming perceived threats from AI to demonstrate a persistent conceptual lacuna in proposed AI regulation: ignored accuracy harms. Part IV proposes a taxonomy of the mechanisms that bring about accuracy harms, empowering scholars and policymakers to systematically recognize and address accuracy harms from diverse sources. AI regulation will achieve better (and better-defined) outcomes when lawmakers recognize that high accuracy is one of many AI attributes that can shape society for better or for worse.