Lost resources

To lose one database is a misfortune; to lose four feels like a conspiracy. Recently three online databases and one audiovisual digital archive with underlying database, on each of which I worked for several years, have been removed from public access, either through policy or misfortune. Two disappeared from view a couple of years ago, two just a couple of months ago. Years of work by some good people to create unique and useful resources have ended up – for the time being – with nothing.

I am not going into the history of what engineered these disappearances. I will describe each lost resource, because there are researchers out there who may not know that they are no longer available. I will indicate what survives and where such remnants can be found. And I will end with a few words on the importance of preserving dynamic, research-based databases.

News on Screen

News on Screen is a database of British newsreels and cinemagazines. It documented over 185,000 individual stories, from 1910 to 1983, and though not fully comprehensive it was very nearly so. As well as having records of issues and stories (with dates, issue numbers, operator credits etc) it had digitised copies of camera operators’ dope sheet (their record of what they had filmed), shot lists, commentary scripts and other supporting documentation. The individual records also linked out to available copies online for such newsreels as Pathé News and British Movietone News. There were background articles, video showcases and a directory of people who worked for British newsreels and cinemagazines. It was considered a model research resource, unmatched by any other newsreel resource globally. It was widely cited and linked to online.

Designed for use by historians, film students and footage researchers, it had its basis in the Slade Film History Register, a document centre for the study of history through film established in 1969 by Thorold Dickinson, film director and Britain’s first professor of film. In 1975 it was acquired by the British Universities Film & Video Council (BUFVC), which built up the resource and published its newsreel records, first in microfiche and in 2000 as a CD-ROM and online, thanks to a grant from the Higher Education Funding Council for England (HEFCE). Further grants from the Arts and Humanities Research Board (AHRB) added first the digitised documents and then substantial information on cinemagazines; a later grant from the David Lean Foundation added a section on Lean’s work as an editor with Gaumont Sound News. In April 2026 Learning on Screen (a renamed BUFVC) took down the site, without public announcement. The database and its digitised files remain the property of Learning on Screen and may possibly be accessed onsite by request. There are no plans for it to return online.

News on Screen can still be found in a rudimentary, archived form on the Internet Archive – this has some of the background texts, but not the main database nor the staff directory, as the IA is unable to archive databases. The 2000 CD-ROM, entitled British Universities Newsreel Project, only has the original newsreel records and not the digitised documents or the cinemagazines records. Copies are held in a few libraries, including the British Library.

International Database of Shakespeare on Film, Television and Radio

The International Database of Shakespeare on Film, Television and Radio (usually shortened to Shakespeare) was a database of audiovisual Shakespeare production worldwide, from the 1890s to the present day. There were over 9,000 records, with full cast and production credits, synopses, references and so on. As well as film, television and radio productions of Shakespeare it included some non-commercial recordings of live performances, where these could be accessed by researchers. It innovated in particular in its emphasis on radio, a subject seldom touched by Shakespeare studies, and the database’s scope was greater than any other audiovisual Shakespeare resource, online or in print. The project was hosted by the BUFVC with funding over a three-year period (2005-2008) from the Arts and Humanities Research Council (AHRC). In April 2026 it was taken offline by Learning on Screen (the renamed BUFVC), after having not been updated for some time. There are no plans for it to return online. It may possibly be accessed onsite by request.

Although an archived version of the database’s front page can be found on the Internet Archive, the database itself does not work, as the IA is unable to archive databases.

The London Project

The London Project is a database documenting the cinema business in London before the First World War. Between 2000 and 2006 Birkbeck, University of London hosted the AHRB Centre for British Film and Television Studies, which had several university partners. One of the Centre’s research strands was The London Project, whose main outcome was a database which listed all film and ancillary businesses in London 1894-1914 and all cinemas and related film venues for the period 1906-1914. The project also produced a number of essays, a touring exhibition, and the installation of some shop window blue plaques in Cecil Court, one-time home of the British film industry. The London Project was one of a number of international databases documenting cinemas and their audiences, all part of a drive to bring sociological techniques into film studies, and it was regularly used by film historians and local historians.

The London Project was taken offline by Birkbeck in 2024. Hopes have been raised that it may return, but to date it remains unseen. As with the databases described above, although its front page and some background texts can be found on the archived version on the Internet Archive, the database itself does not work, as the IA is unable to archive databases.

Broadcast News

Broadcast News is a digital audio and video archive of British television and radio news, established by the British Library in 2012 as part of its drive towards a multimedia news capacity rather than only collecting newspapers. Using software developed by Cambridge Imaging Systems (now Reuters Imagen), it recorded television and radio programmes off-air, under an exception in the UK Copyright Act, eventually across some thirty channels, including BBC channels, ITV, Channel 4, Al-Jazeera English, CNN, France 24, RT and NHK World (although channels such as Al-Jazeera come from overseas, all had UK offices and were licensed for broadcast in the UK by Ofcom – therefore they were British news). Metadata was captured automatically through the transmission files, and the capture of subtitles meant that many of the TV programmes could be searched for by word, a huge boon for researchers. Programmes were made available in the British Library’s reading rooms within a few hours of broadcast. Owing to copyright restrictions, access to the archive and its underlying database could only be offered on site. The service added some 30 hours of TV and 50 hours of radio per day. By 2023 it had around 200,000 programmes in its archive, mostly television and radio news programmes, but also a number of general interest broadcasts.

On 28 October 2023, as has been well documented, the British Library fell victim to a ransomware cyberattack which targeted its digital infrastructure. It has taken the BL nearly three years to repair the damage done to its systems, and the work is not over yet. One resource that has remained closed is Broadcast News. So far as I know, the database itself was wrecked, but the audio and video files have survived. However, without the database and its dedicated player, the archive cannot be used. There are plans eventually to make the programmes available via the British Library’s own player, but this is a long way off and it will be difficult to have the player match the functionality that Broadcast News originally enjoyed. And there have been no further programmes added by the British Library since 28 October 2023. Whether this will ever happen remains unstated. There is a listing of the television news holdings 2012-2022 (so excluding radio and non-news TV programmes), 114,380 of them to be precise, on the British Library’s Research Repository. Producing that was just about the last thing I did before retiring from the BL in July 2022.

Each of the resources listed here I established or managed at the institution that hosted them; each came to grief after I left. I’m not going to touch on culpability, if any, except for mine in not having put in greater protection measures when I had the opportunity. What I do want to raise is the importance of preserving research databases, because it’s something we’re not doing at all (at least in the UK) – and we could be.

Our world is controlled by databases, which bring order and management to the data that now dictate our lives. But a research database is a smaller thing – specifically, it’s a form of publication. Information is gathered by a research team, often with funding from a government-supported bodies, notably those overseen by UK Research and Innovation (UKRI), such as the above-mentioned Arts and Humanities Research Council (AHRC). These bodies give out research grants (usually to university research teams) which result in outputs, often books but sometimes databases. Books are preserved, thanks to our libraries and, in the UK, the system of Legal Deposit that ensures that what gets published in print gets preserved.

Research databases are not preserved. There is no system in place in the UK (or anywhere else, so far as I know) to ensure that research outputs are officially preserved in perpetuity, as though they were books, when they are in the form of a database. There are systems in place to preserve raw data, such as the UK Data Service, which focusses on economic, population and social research data and supports the UK Data Archive, while a Heritage Science Data Service is in development (there used to be an Arts and Humanities Data Service, but that ceased operating in 2008). But raw data is just that – the data gathered in a consistent form and often held in something like an Excel spreadsheet. A database is something that you can see, query, explore and which has a life. Often that life is because it is being constantly updated (a so-called dynamic database). One of the perils for research databases is that they rapidly lose currency when they cease to be updated. They start to look uncared-for. And once you start neglecting a living thing, you are on the road to killing it off.

Databases are difficult to archive – certainly they can’t just be hoovered up by a web archive, as noted above. You need to be able to capture the constant changes if it is a live, dynamic database. Alternatively you need control of, or be able to duplicate, the underlying management system. You may also want to archive the web front end, with any associated web pages, and the digital files to which the database provides access. There are huge challenges involved.

There is, however, a means for preserving databases in the UK – it just hasn’t been implemented as yet. The Legal Deposit Libraries (Non-Print Works) Regulations of 2013 allow for the collecting of non-print – i.e. electronic – publications, just the same as books. It’s because of these Regulations that we have the incredible UK Web Archive, preserving billions of web objects. Dynamic databases are just as much a non-print publication covered by the Regulations as a website, as the Legal Deposit libraries – among them the British Library – recognise. In their Joint Collecting Framework for Legal Deposit 2023-2030 document it states:

Our tools and infrastructure that enable collecting at large scale are vital in ensuring that our collections are representative, given the scale of publishing in the UK. Although our aim is to be comprehensive, there are some cases where our ability to collect at scale does not meet the volume and complexity of publishing. Some limitations are manifest in long waiting times to process newly received digital content or to create the necessary workflows for deposit by a new publisher. There are other cases where we have not yet developed the capability to collect (eg for some types of social media, or publications that are delivered as dynamic databases).

So that’s it. The intention is there, but the capability is not, as yet. It certainly will be a complex undertaking, and quite probably an expensive one. Just at the moment the British Library is focussed on getting its digital basics back in operation, following the devastating impact of the 2023 cyberattack. But meanwhile publications are being lost. When the legislation is in place and the technical task, though challenging, is not impossible, then that is a huge shame.

My lesson from all this is never again to get involved in research databases, though as I am a retired archivist and curator, that isn’t going to happen anyway. All I can collect now are regrets.

Links:

• The screengrabs for News on Screen, International Database of Shakespeare on Film, Television and Radio, and The London Project all come from the Internet Archive. That for Broadcast News comes from my records
• For a list of what British Library resources are available or remain unavailable post-cyberattack, see bl.libguides.com/currently-available. Broadcast News is not alone
• For further discussion on the importance of preserving research databases, see my 2017 post Cinema Contexts
• On the history of using newsreels for the study of history, including the journey from Slade Film History Register to News on Screen, since my earlier blog post Newsreels and History
• On the rationale behind the The London Project, see Ian Christie’s essay ‘Screening the City‘
• On the definitions and difficulties of database archiving, see Heiko Müller, ‘Database Archiving‘, Digital Curation Centre briefing paper (2009)