The following article, “In Defense of Secrecy,” by HLS Professor Noah Feldman appeared in the February 10, 2009 issue of The New York Times Magazine.

Given the pervasive secrecy of the Bush-Cheney administration, and the sorry consequences of that disposition, President Barack Obama’s early emphasis on openness in government seems almost inevitable. One of the first official communications issued by the new administration, on Jan. 21, ordered government agencies to adopt a presumption in favor of disclosure when responding to Freedom of Information Act (FOIA) requests and called for new FOIA guidelines to replace those promulgated under Bush. A later directive instructed the heads of all government agencies to strive for “transparency and open government.” Ornamenting the first order was a quotation from the great progressive reformer Justice Louis Brandeis: “Sunlight is said to be the best of disinfectants.”

There is something charming about Obama’s reference to his fellow Harvard Law School graduate, right down to the fussy, lawyerly precision of the exact quote, which is often rendered more straightforwardly as “sunlight is the best disinfectant.” And Obama’s instinct for candor and openness is certainly a refreshing change from the shadowy practices of the last administration. At the same time, Brandeis’s metaphor, based on a medical theory now long refuted (alcohol is a much better disinfectant), can be taken too far. The effective operation of even the most democratic government requires secrecy and surprise as well as transparency and predictability.

On the surface, it might seem that more and better information about the government’s decisions (and decision-making processes) is always preferable, especially if the information is provided before events transpire. As good-government advocates are right to remind us, if the public learns of an important decision only after it has been made, advocacy groups and concerned citizens cannot influence it. They can only give their endorsement or disapproval after the fact and hope it has some effect the next time around. When it comes to the proposal and drafting of legislation, we expect public engagement, even as we acknowledge that special-interest groups can use that process to torpedo innovative laws in the way the insurance industry brought down health care reform during President Clinton’s first term.

The value of transparency also applies to the financial markets — which is what Brandeis had in mind when he spoke of “sunlight.” The derivatives that gave rise to the current economic crisis were opaque to almost everyone. Ascertaining the true value of such complex instruments was (and remains) mind-bendingly difficult. Indeed, in the absence of a central clearinghouse or of regulatory disclosure requirements — of the sort now being proposed by Treasury Secretary Timothy Geithner — it was impossible for anyone to know just how many obligations were owed and owned, and by whom.

Yet there are many circumstances in which secrets are critical. Consider the quotidian business of government inspection: it requires the element of surprise, or else regulated industries could game the system by preparing for oversight on specified days and places. A grander example comes from diplomacy. Whether to negotiate with Iran is an important topic of debate, but the actual negotiation, and even the steps leading up to it, cannot successfully be conducted under the glare of public scrutiny. Neither side would take the risks necessary for real engagement as long as its high-risk efforts could be exposed to denunciation and ridicule if it failed.

The financial bailout has its own needs for secrecy. It is essential for Congress to debate what sorts of industries or companies should be saved and how, and it must authorize the money, just as it must pay for a war. But the decisions to bail out AIG and to let Lehman Brothers fail, whatever their merits, were the sorts of immediate, crisis-driven judgment calls that could not have reasonably been subjected to extensive public debate, or that would have been improved by public lobbying and interest-group advocacy. If specific companies could lobby for bailouts, and their competitors could argue against, the bailouts would turn into political football — which is what arguably distorted the effort to bail out the auto companies.

The balance of transparency and secrecy is not always easy to strike, of course. Consider the question of whether to make public the interrogation tactics to be used on suspected terrorists. Public discussion and deliberation is crucial to ensure that interrogation will proceed within the bounds of law and without torture. Yet it does not follow that the specific methods we use should be shared with the public, since doing so might invite training by terrorists in how to evade or withstand them.

Obama presumably knows all this; his pronouncements about transparency so far have been so general as to be largely symbolic. That he issued a statement urging his subordinates to “increase and improve opportunities for public participation in government” without providing any specifics suggests that he wants to change the background tone of government — but also that he recognizes the limits of transparency.

Recognizing these limits may seem a bit cynical, especially after eight years of an administration that favored secrecy excessively and executed its policies poorly. But our constitutional system allows us to set boundaries on openness, provided the public remains the final arbiter of whether it benefits from being kept out of the loop. The test must always be whether the public, after the fact, considers itself better off — and that judgment can be made by only the public itself, as it recently was, by way of elections. If we know more about what our government is doing in the next four years than we did in the last eight, that will be to our advantage. We shouldn’t forget, though, that when it comes to secrecy, transparency, like sunlight, is only a figure of speech — one that disappears the closer you look at it.

Noah Feldman, a contributing writer, teaches law at Harvard University and is a fellow at the Council on Foreign Relations.