When the Government Goes Top Secret, Who Can Write Its History?

THE DECLASSIFICATION ENGINE: What History Reveals About America’s Top Secrets, by Matthew Connelly

It’s no secret that government secrecy has gone haywire. Classified documents have turned up in the homes of President Biden and former Vice President Mike Pence, where they were perhaps inadvertently stashed. That’s a far cry from the case of former President Donald Trump, who, it appears, deliberately concealed top-secret records at his Florida estate, unlawfully keeping them from their rightful owners: the National Archives and, ultimately, the people of the United States.

But the problem is worse than malfeasance at Mar-a-Lago. As Matthew Connelly details in “The Declassification Engine,” the U.S. government itself has long been guilty of the same behavior on a mind-boggling scale, robbing Americans of the right to know what’s been done in their names in the not too distant past.

How have past presidents made alliances with tyrants? How did they decide to assassinate America’s enemies as a matter of foreign policy? Why did the Pentagon lose its 20-year war in Afghanistan? What was behind Trump’s bromance with Vladimir Putin? What did they say to each other behind closed doors? Maybe our grandkids will find out. Maybe not: It’s classified.

America’s secrecy system is a Cold War creation that went out of control long ago. To tackle the issue in his book, Connelly takes on three tasks: describing the system’s dark labyrinth; cataloging the catastrophic effects of robotic classification on our ability to understand the past; and proposing a computer-driven assault on the empire of secrets. He succeeds admirably in two out of three; the search engine he’s devised is off to a slow start.

Connelly, a professor of history at Columbia University, traces the system’s origins to the Manhattan Project. The atomic bomb was born a secret, and Lt. Gen. Leslie Groves Jr., who directed the crash program to build it, helped create the present-day classification system to keep it that way. This didn’t stop spies for the Soviets from stealing crucial details about the bomb before America used it in 1945. It did keep Harry S. Truman from knowing the size of the atomic arsenal until a year or two into his presidency. Then, in 1947, came the creation of the C.I.A. and the modern national security state. The machinery of classification and the stockpile of secrets have grown exponentially ever since. And now military and intelligence officials keep Americans from knowing the scope of their own history.

By presidential order, most national security records of the White House, the State Department and the C.I.A. should be declassified within 10 to 25 years, after page-by-page reviews to shield intelligence sources and methods. These records, when released by the National Archives, the agency tasked with sorting and preserving them, are a foundation of presidential libraries. Yet, Connelly reports, the archives confronts a backlog of some 600 million unexamined presidential records, and it will take 250 years for its staff to process Freedom of Information Act requests for documents at the George W. Bush Presidential Library alone. The tradition of presidential libraries serving as oases for invaluable research is crumbling; the Bush library may be the last to fulfill that purpose.

By law, the State Department must issue a full and accurate account of American foreign policy. These annals, the “Foreign Relations of the United States,” published since 1861, cast light on what American leaders really thought and really did as they projected the nation’s power around the world. But after decades of struggle, the C.I.A. and the Pentagon still maintain an iron hand over the “Foreign Relations” series, whose editors now strain to produce one volume a year. Some records dating to 1956 haven’t seen the light of day. The annals of the Reagan administration have only recently begun to emerge. Today, as Connelly writes, the 1980s and 1990s “are — with few exceptions — an undiscovered country.” And at the rate things are going, the release of the records of American foreign policy in the 20th century won’t be complete until … the 22nd.

Connelly has defined an existential crisis: the suppression of American history. Ten years ago, he sought to address it. He and his colleagues wanted to build the world’s biggest archive of declassified government records. They resolved to create a sophisticated search engine that could use machine-learning algorithms to predict the contents of redacted text and examine documents’ metadata for telling patterns. Their goal was to break the declassification logjam and “reveal what the government did not want us to know, and why they did not want us to know it.”

I’m skeptical. It’s a soaring ambition, and the results published here are a handful of needles in a mountain of hay. I think it will take many years for Connelly’s engine to begin to achieve his aspirations. Still, it’s a start. “We won’t find all the answers, of course,” he concedes, “but we can get measurably closer to the truth.”

This leaves reporters and activists as the only consistent counterforce against secrecy in government. Over the years, The New York Times and The Washington Post have unveiled the illegal surveillance of dissident Americans under Presidents Johnson and Nixon, detailed the Reagan administration’s sales of weapons to Iran, exposed the National Security Agency’s domestic spying, disclosed the C.I.A.’s torture of prisoners in black sites and documented the lies and corruption that permeated the Pentagon’s failures in Afghanistan.

In 1971, The Times obtained the Pentagon Papers, the secret history of the war in Vietnam, from the rebellious military analyst Daniel Ellsberg, whom the Justice Department sought to imprison for espionage. The papers showed plainly that the government had been lying to the public about the war for years. When the Nixon administration sued The Times to stop publication, its Washington editor, Max Frankel, filed an affidavit with the court, which Connelly quotes. Frankel, no radical, wrote that the government routinely abused its authority “by imposing secrecy where none is justified or by retaining it long after the justification has become invalid. … To hide mistakes of judgment, to protect reputations of individuals, to cover up the loss and waste of funds, almost everything in government is kept secret for a time and, in the foreign policy field, classified as ‘secret’ and ‘sensitive’ beyond any rule or law or reason. Every minor official can testify to this fact.”

“The Declassification Engine” makes the case that the culture of secrecy diminishes democracy. And it has now become a culture of destruction as well. As Connelly writes, Trump had a habit of ripping up presidential papers. Now the State Department is contemplating doing much the same, and on a grand scale: unleashing government-engineered algorithms to assign foreign policy documents for automatic deletion. If this happens, Connelly says, “the great majority of important historical records will be destroyed, automatically, and much of what’s left will be dreck. It is, in a very literal sense, the end of history as we know it.”

Tim Weiner, a former national security reporter for The Times, is working on a book about the 21st-century C.I.A.

THE DECLASSIFICATION ENGINE: What History Reveals About America’s Top Secrets | By Matthew Connelly | Illustrated | 540 pp. | Pantheon | $32.50