The Data.gov Archive at the Harvard Law School Library Innovation Lab 

At the Harvard Law School Library, we have 39 early manuscript copies of Magna Carta, and now we also have over 300,000 public datasets published by the United States federal government.

In February, our Library Innovation Lab launched the Data.gov Archive, a 17-terabyte archive of every dataset published on data.gov by the U.S. federal government. The archive allows our research community — meaning anyone in the world! — to access reliable data whether or not it remains available from its original source.

We did this work for the same reason we collected 39 early manuscript copies of the Magna Carta: because it is vital – in planning our laws, our government, and our future – to be able to remember what happened and from where we came.

The Data.gov Archive was created with custom software, developed at the Lab, to download datasets and attach full metadata, cryptographic signatures, and timestamps, which makes the dataset easier and cheaper to host and to share with other archives. This is part of our larger work, such as Century Scale Storage and Perma.cc, in figuring out how to make digital archives as robust and durable as physical ones so they can last for as long as our Magna Carta copies have.

This work also shows the extraordinary value of having software engineers embedded within one of the world’s largest law libraries. Mixing software development into our work lets us respond to emerging issues and opportunities quickly and effectively, grounded in the Library’s centuries-long traditions while looking to the future.

Jack Cushman is Associate Director of Innovation and Director of Library Innovation Lab at Harvard Law School Library