Colouring in the last bits: the end of an era

Exactly ten years ago, on April 26th 2002, one of the Library’s archivists sat at a PC and started our archive catalogue programme, went through various options to set up an import of data, selected the source file and clicked OK…
…and a matter of seconds later, our archive database had received its first load of converted catalogue data and we had set out on a journey from paper to digital, a journey completed a few days ago with the conversion of our last paper archive list.

 It is hard, now, to think back to those early days, so accustomed are Library users and staff to the archive database. The complexities of rendering a hierarchical archive catalogue on the screen – plus, for a long time, a lack of agreement as to what an archive catalogue should actually do [1] – meant that good catalogue software for archives developed more slowly than that for libraries, and it was only in the 1990s that a move to database catalogues gained some momentum. (Happily, this took place at the same time as the explosion in use of the World Wide Web, so archive catalogues when they did arrive were able to jump straightaway to giving readers web access, rather than forcing them to work with pre-web technologies such as Telnet.) Up till that time, the reader wanting to use our archives and manuscripts sources was confronted by six meaty published volumes of manuscript cataloguing, and some two hundred spiral-bound word-processed lists of twentieth-century archive collections.

For the reader wanting a particular organisation or individual’s papers, this worked, and usership was steady. For the reader wanting thematic access across the collection, however, life was much harder. Some topics, of course, came up repeatedly, and for these we compiled sources guides describing material relating to particular themes or areas: many of these, radically recast and updated, are still to be found on our website. For the reader with a new, unusual or quirky topic, however, rather than one of the staples such as shell-shock or abortion, it could be difficult to know where to start. A few years ago, for instance, a user of the Rare Materials Room wanted to know about our sources relating to Marmite: the sort of question that would have been virtually impossible to answer before the database. [2]

The database was to revolutionise all this, and to democratise access to archives and manuscripts profoundly. It enabled readers to ask the questions that they wanted to have answered, not simply the ones we had managed to predict; and by opening up the database to a wider range of enquiries, saw some hitherto quieter areas of the collection, such as the American manuscripts, receiving heavier use than ever before.

First, though, we had to get the data into it. The turn of the Millennium saw a large number of collaborative exercises launched to get archive data onto the web, and it was from one of these that our first converted catalogues came. Access to Archives, or A2A, was a programme dedicated to converting archive catalogues to web-readable form and mounting them on the National Archives website, enabling a web presence for those many repositories which, at that stage, did not have their own site. One strand of the project worked to convert documents relating to London’s contacts with the wider world, through exploration, trade and colonism, and it was material submitted to this which came back to us as a digital file in early 2002 and was loaded on that fateful April 26th. Among the items made available to readers in that first tranche were the journals of naval surgeon Fleetwood Buckle (MSS.1395-1404 and 5656), the papes of Sir James Cantlie, the tropical medicine specialist (MSS.1456-1499, 6931-6941 & 7920-7941), the diaries of the Tsimshian Native American Arthur Wellington Clah (WMS/Amer.140)… we were up and running.

From that point onwards, the process of retroconversion (to use the appropriate jargon) never stopped. We experimented with cutting and pasting word-processed catalogue into the database but the process was so complex (most of the old lists had been laid out on the page with copious paragraph markers and tab-stops, all of which needed to be removed) that it worked out quicker and cheaper to re-key catalogues. A dedicated project officer started work that year, working entire days at marking up catalogues in coloured pens (each colour corresponding to a database field) before posting the results to New Delhi, from which a spreadsheet of rekeyed data would appear with terrifying speed in the inbox. Little by little, several thousand records at a time, the database grew; little by little the profits of a particular stationers’ on Cleveland Street, which boasted the best selection of coloured markers in Fitzrovia, swelled. At the end of this project, well over 75% of our data was in there. In the time since then, members of the archives department have converted the remaining catalogues in spare half-hours here and there (evening and Saturday duties have been particularly fertile times). Now, that era has ended. After a little under ten years, the final catalogue has been converted and loaded: the catalogue of the Chartered Society of Physiotherapy‘s papers, which had drawn the short straw and ended up last of all, was converted and loaded last week. You have only to go to Wellcome Images and search for the phrase SA/CSP to get an idea of the sort of things now described online: we expect the scrapbooks on the “massage scandals” that led to the CSP’s formation to notch up a large number of hits all on their own…

2012, then, sees the end of an era: all of our archive catalogues online at last. It comes, as luck would have it, just as a new set of technical developments needs work: the wholesale digitisation of archive holdings, among other library materials, and the need to make catalogue records and images link up in as smooth and seamless an experience for the reader as we can manage. The archivists, needless to say, are not putting their feet up at this particular moment; it is, however, time for a few minutes of retrospect and quiet satisfaction as a long-running project comes to a close.

[1] For those who are interested, the breakthrough came in 1997 when the International Council on Archives set out the snappily-named ISAD(G) – the International Standard on Archival Description (General).
[2] It turns out that we have quite a few.

Images:
1/ The import screen for our CALM archive catalogue programme (backstage client): further information from the manufacturers here.
2/ Napoleon’s tomb, St. Helena, from MS.1395, one of the papers of Fleetwood Buckle that were among the first items to be loaded to the database.
3/ Some of the actual pens used in retroconversion, kept for posterity.