Kris Snibbe/Harvard file photo
I. Glenn Cohen explains why we should scrutinize what is and then ponder what should be
This is part of a series called Focal Point, in which we ask a range of Harvard faculty members to answer the same question.
Focal Point
I. Glenn Cohen
QUESTION: What is one thing wrong with the world that you would change, and why?
Afamous phrase in philosophy, associated with David Hume among others, says that one cannot go from “is” to “ought.” The (contestable) claim is that moral judgments cannot be derived solely from factual premises. But a related maxim seems even more important today — “is” is not “ought.”
When I speak to the media, students, policy-makers, or even friends and acquaintances, there is a pernicious tendency towards, to use (or slightly misuse) a phrase from behavioral economics: “status quo bias.” It means the tendency to treat the status quo as morally or legally justified, rather than subjecting it to the same scrutiny as the arguments made against those who would change the way things are.
Let me give a tangible example. One topic I work on is the use of machine learning and other artificial intelligence in health care. A well-founded critique of some of this work and similar work in policing and other areas is that because of the data sets used as training data, the results of AI models produce bias; more specifically, they produce outcomes for disfavored groups (such as African Americans) that are inaccurate in a way they are not for favored groups (such as white Americans).
“[T]he mere fact that you started with something … does not mean you were entitled to it.”
This is a bad thing. But to determine its implication for policy there is another necessary step: comparing the pattern of results against the status quo in order to make an all-things-considered judgment about the desirability of using such models. In other words, even if these models produce some bias along these lines, so does the status quo! For example, the physicians not using AI currently practice medicine may have all sorts of built-in, implicit assumptions about African American patients that lead to different diagnoses, treatment plans, etc. So one relevant question — and to be clear, I am not saying “the only right question” — that often goes unasked is: Would relying on this AI model produce less bias than the status quo? This is not to say that there aren’t lots of other reasons against pursuing an AI model, such as concerns about health data privacy, transparency, liability, but instead to insist we make an apples-to-apples comparison. When you consider a concern about a new proposal, you should apply the same concern to the status quo.
This doesn’t mean that we shouldn’t try to produce AI that is even less biased than the one we started with. But at the same time, to use another philosophical chestnut, we should “not let the Perfect be the enemy of the Good”; sometimes an imperfect improvement may be desirable because it improves on the status quo. I say “sometimes” because the ethical world is complex. Some might distinguish taking action that leaves bias in place from inaction that leaves bias in place.
This articulation of the problem has been pretty straightforward. But this problem can be subtle and make itself felt in what law professors (and others) like to call “baseline problems.” The problem is in demanding a reason to justify shifting an entitlement without asking for the same justification as to how the entitlement originally became the status quo.
Consider this example loosely inspired by a real case: You own a hotel with a pool deck with great sun. I own the hotel next door and I want to add 10 floors, which will block your sun. You sue me, claiming that I should be prohibited from building, or, if I do, that I must compensate you for the loss of value to your hotel, because my addition will reduce the popularity of your hotel and thus your revenue.
Who do you think should prevail?
Notice that the words “block your sun” makes the very error I warned of, treating the status quo allocation of entitlements as settled. Was the sun shining on your pool deck something that you were entitled to? Why? We could instead say that by prohibiting me from building you are depriving me of my right to improve my hotel and add revenue to my business.
My point is not about who is right. The critique is instead that the mere fact that you started with something (sun on your pool deck) does not mean you were entitled to it. We need an argument as to why the entitlement belongs to you and not to me. And in many arguments around the world, between neighbors, scholars, or our representatives in Congress, this simple insight is ignored. The “is” in which we happen to find ourselves is unreflectively given a moral status and an argument is needed for shifting things, but no one asks why the baseline allocation is justified.
— I. GLENN COHEN
James A. Attwood and Leslie Williams Professor of Law
Faculty Director, Petrie-Flom Center for Health Law Policy, Biotechnology and Bioethics