Is LTFS the answer for digital AV archives?
I had the opportunity to talk about the Linear Tape File System (LTFS) at the SEAPAVAA conference in Bangkok. The theme of the conference was ‘Redefining The Audiovisual Archives in the Digital Age”. The theme shaped a range of thoughtful and diverse presentations, all of which were well received by the audience. My impression of the conference overall was that the SEAPAVAA community shares a positive and enthusiastic attitude towards archives in a digital environment, as well the opportunities the new paradigm presents.
My presentation, Refreshing the Archive: Experience in Archiving, Restoring and Content Exchange with Linear Tape File System (LTFS), focused on the long-term storage layer of digital archiving.
At DAMsmart, we are digitising AV material for a diverse range of public and private organisations. In most cases, we are delivering a preservation component that is maintained ‘off line’ due to data size and infrequent access requirements. However, when access is required either for verification of the data integrity, or for access to the master format, access should be without the boundaries imposed by proprietary vendors or poor implementations of open formats. This is where I believe LTO and LTFS promotes real opportunities for the digital AV archive owners and custodians.
The LTO format is a long established standardised format (particularly in an IT context!), however the method of writing digital information has either been written in a proprietary format, only available through the vendors software or licensing fees, or through wrappers not originally designed for linear tape. Open formats do exist, however they have proved to be troublesome in the audiovisual ‘Big Data’ scenario. For example, TAR is an open method of writing data to tape, however it is a compression format, rather than a file system. The advantages of writing directly to a file system such as LTFS, which is operating system independent, allows for unmitigated access to the file directory structure on tape. File based access is critical under the following circumstances:
- Database corruption
- Hardware system failure
- Vendor support which is driven by the commercial market
- Flat file based access can still be achieved without the reliance on a specific application
I think Peter Van Gardernan of Artefactual Systems sums it up well:
‘File system technology is long-proven and extremely robust, typically outlasting the lifespan of enterprise information systems. Making the file system the focal point of micro-services operations is noteworthy as a long-term preservation strategy because it provides archivists with the option of direct, unmediated access to archival storage.’
Van Garderen, P. “Archivematica: Using Micro-Services and Open-Source Software to Deliver a Comprehensive Digital Curation Solution” 2010.
Adopting the LTO and LTFS combination for archiving digital AV media can present both economical advantage and data security. And if used in conjunction with a growing suite of management technologies, LTFS can ensure digital archives are highly accessible and safe for the long-term.
Feel free to get in contact with me if you are interested to know more and I would be happy to share my presentation.
