Sustainable Audiovisual Collections Through Collaboration: Proceedings of the 2016 Joint Technical Symposium
235
Sustainable Audiovisual Collections Through Collaboration: Proceedings of the 2016 Joint Technical Symposium
235eBook
Available on Compatible NOOK devices, the free NOOK App and in My Digital Library.
Related collections and offers
Overview
The art and science of audiovisual preservation and access has evolved at breakneck speed in the digital age. The Joint Technical Symposium (JTS) is organized by the Coordinating Council of Audiovisual Archives Associations and brings experts from around the world to learn of technologies and developments in the technical issues affecting the long-term survival and accessibility of audiovisual collections. This collection of essays is derived from presentations made at the 2016 JTS held in Singapore and presents an overview of the latest audiovisual preservation methods and techniques, archival best practices in media storage, as well as analog-to-digital conversion challenges and their solutions.
Product Details
| ISBN-13: | 9780253027139 |
|---|---|
| Publisher: | Indiana University Press |
| Publication date: | 12/22/2021 |
| Sold by: | Barnes & Noble |
| Format: | eBook |
| Pages: | 235 |
| File size: | 36 MB |
| Note: | This product may take a few minutes to download. |
| Age Range: | 18 Years |
About the Author
Dietrich Schüller is director emeritus of the Phonogrammarchiv at the Austrian Academy of Sciences and is a specialist in audiovisual preservation and re-recording.
Lars Gaustad directs the film preservation efforts at the National Library of Norway.
Rachael Stoeltje is the director of the Indiana University (IU) Libraries Moving Image Archive and established the IU film collections into the IU Libraries Moving Image Archive.
An award-winning creative director and designer for main titles in feature films and television, Vicki Shively is an Masters in Library Science candidate in digital libraries at Indiana University with a focus on moving image archives.
George Boston spent his professional career as a sound technician and manager with the BBC, including their music studios in London, Audio Unit in Manchester, and the Open University Production Centre in Milton Keynes.
Read an Excerpt
CHAPTER 1
Let's Emulate the Sound of Colours!
Reto Kromer
Abstract
Let's emulate the sound of colours from the past through the best techniques that are technologically available today. By keeping all the possibilities available for our successors, they will be free to debate, modify and improve on our own work.
Keywords
digital film restoration, audiovisual preservation, analogue, digital, photochemical, colour reproduction, ProRes, TI/A, FFV1, FLAC, Matroska, FFmpeg, QCTools, MediaConch, DCP, OpenDCP, data archiving, conservation, restoration, migration.
Collaboration
Very little of the available technology and tools have been designed and manufactured especially for audiovisual archiving purposes. On the contrary, most of the technology and tools that we are using in our daily archival work have been designed and developed for today's production and post-production needs. Archivists, conservators, and restorers are continuously adapting this technology and tools to fulfill specific preservation and access needs. Therefore, we only rarely have, for example, a carbon-arc projection of a toned and stenciled nitrate print, but usually, we will enjoy a modern digital projection that emulates, as best as possible, that specific historical look through a file. Emulation can, of course, also be seen in a pedagogic sense: a master teaching to his disciples. So this is indeed important!
Analogue and Digital
In the late 1980s and the early 1990s, when I started considering digital methods for both film restoration and audiovisual conservation, I was mainly regarded as a fool — or at best a candid dreamer. Many years after, before the end of 2013, when I closed down our photochemical laboratory, I was often considered a person who missed the boat in the switch to the new digital world because, occasionally, I still preferred to apply the "old-school" analogue, photochemical methods. Ironically, what I was actually doing was capitalizing on the best potential offered by both worlds and mixing them together to achieve the best possible results. This method created the most historically accurate presentation of a masterpiece or a document in the modern screening context. The penultimate job my photochemical lab did was the preservation of amateur films shot on Kodachrome film stock. Generally, for this type of preservation, we would use the Fujifilm daylight 64 ASA camera negative as an intermediate film stock because it provides the best possible quality in analogue colour reproduction. Sadly, by then that film stock was no longer manufactured, and we used the very last reels stored in our refrigerator.
The Joint Technical Symposium's (JTS) audience is a very special one. As the JTS name implies, this is a symposium of technically interested individuals with crossover specializations.
The audio community was the first, and the quickest, to move from analogue to digital, and the relatively small size of their data files aided in their early technological adaption. Also, audio technicians seemed to adapt to the new technology easier than other film disciplines, most likely because the move from analogue mechanical recording, to analogue magnetic recording, to digital magnetic recording, to file-based recording was more natural and less polemic in the sound fields.
The broadcast community shifted to digital just after the audio communities to take advantage of the digital possibilities for production and postproduction. At this early stage, they rarely took into account the new challenges for preservation in this digital medium, as well as not understanding the fundamental differences between restoration and enhancement.
Unfortunately, the film preservation community is still in the middle of paranoiac contradictions. It appears that most of their energy, time and money is used for the continuous and duplicitous restoration of a few dozen films they call "canon." Some film purists think that even digital-born films should be preserved onto film stock. In today's worst/best scenario we can record onto film only 1/64 of the image quality from the digital file. That is less than 2%. This data loss means that more than 98% of the data are tossed into the rubbish bin. This situation, regrettably, exists because while camera sensors are continuously improving, re-recorders onto film are not. The market for this particular type of recording equipment is gone.
If analogue storage is the answer to all audiovisual preservation needs, why do we have to struggle that much with analogue conservation and restoration issues?
Standardisation
The proceeding three examples are relevant file formats to the film preservation field. If one could be established as a standard principle for the moving image community, collaboration would be more effective and efficient.
ProRes is often hated by archivists because it is a proprietary format of Apple. However, this is a de facto standard in postproduction. The Society of Motion Picture & Television Engineers (SMPTE) wishes to standardise this recording format. At this time it is unclear if it is only the Apple ProRes 422 HQ that is meant to be standardised, or the entire ProRes format family, such as the 4444 XQ. The hope is that the community avoids the mistakes they made with the standardisation of the CineForm or VC-5 codec. Essential information is still kept secret by GoPro, and not all metadata described in the standard matches the metadata generated by GoPro's products. An unfortunate situation indeed!
A group of scholars from the University of Basel in Switzerland has the goal to define a TIFF format that contains all the technical metadata that is relevant to preservation. Adobe, having refused the name TIFF/A as analogous to PDF/A, has chosen to name the new file format TI/A (Tagged Image for Archival). ISO (International Organisation for Standardisation) is the body where this new recommendation should be submitted, possibly amended, accepted, and finally published.
The IETF (Internet Engineering Task Force) has established a workgroup called CELLAR (codec encoding for lossless archiving and real-time transmission). Cellar is currently standardising the lossless video codec FFV1, the lossless audio codec FLAC, and the extensible media container Matroska, which is based on EBML (extensible binary meta language), a binary XML format. Once adopted, these standards could also be submitted to ISO and SMPTE for additional validation.
Open-Source Software
Today, there are plenty of open-source software options which do not require you to be an engineer to use them. These digital tools form a complete archival ecosystem. The following software is extremely useful and highly recommended.
FFmpeg is a complete, cross-platform solution to record, convert, and stream audio and video. It would benefit every audiovisual archivist to be literate in FFmpeg.
QCTools offers an extremely wide range of strong quality control tools for video preservation, developed by the Bay Area Video Coalition and Dave Rice.
Most audiovisual archivists are familiar with MediaArea's MediaInfo. MediaArea is currently developing the new tool MediaConch. This tool consists of an implementation checker, policy checker, reporter, and fixer that targets preservation-level audiovisual files; specifically Matroska, LPCM (linear pulse-code modulation) and FFV1, for use in memory institutions.
The Digital Cinema Package (DCP) has replaced prints in commercial film distribution for theatrical projection. DCP is a unique format that was designed by the movie industry, conversely, because of how it is constructed, it can make a film archivists' preservation work difficult.
Despite these issues, the archivist should give consideration to DCP when deciding the best software tools for their collections. It allows greater control over the manner in which the audience sees a historical film. The archivist can encode the right colours, the correct aspect ratio, an adequate projection speed, and the DCP will be screened that way in all cinema theatres worldwide. OpenDCP is one of the tools that makes this possible.
Open-Source Hardware?
In the analogue world, things may appear simple. Figure 1 shows the minimal equipment needed to create the illusion of movement. In the digital world, equipment with that kind of simplicity is currently not available.
Twelve years ago, when we built our first scanner, we only modified one camera part within an optical step printer. The projector part, the lenses, the pre-wet device that we had previously developed, as well as the mechanical movement was retained without modification.
Also, the optical camera, number 128 from Richard Craß in Berlin, was the first one we converted from analogue to digital. Many others made similar conversions. It is possible to build, in your kitchen, a relatively simple machine for cleaning all kinds of magnetic tapes.
For archiving large amounts of audiovisual data, LTO is often chosen. The magnetic tape based solution LTO (linear tape-open) with LTST (linear tape file system) is indeed a good solution for data archiving in the real world.
Conservation
In short, analogue conservation focuses on the chemistry of the base, and the emulsion or the magnetic coating, and concludes that one should store materials in a cool and dry environment.
On the other hand, digital conservation concludes that one should keep every single bit unchanged, and instead focuses on the container, codes and the so-called "raw" data.
Briefly, as for the migration process, from January through June of 2014, we migrated the digital archive of our company from LTO-4 to LTO-6. It was decided to change some file formats, and to convert them en passant into other more robust formats for future accessibility. We started with roughly 1000 tapes and ended up with fewer than 300 tapes. Although we reduced the storage volume in the security cabinet by more than two-thirds, there are now more consistent data. Additionally, we have migrated 5.7 PB of data and only encountered one single error. This was resolved by using the archive's second back-up copy.
This kind of work must be done through an automated process to avoid the "human error" factor. However, archivists do need to have the skills to deal with technical issues. It is crucial to be able to read and understand the technical information provided by the industry, and to choose the solution that best fits their archival needs.
Restoration
Throughout my life, I have spoken many times about the ethics of conservation and restoration, including organizing a full conference in 2010 about this very topic. Through age and experience, I have come to agree thoroughly with the thought: "Restoration consists of replicated errors of the past." We, as a professional community, should be able to do more in our restoration work than just replicate the errors of the past.
The first software for digital image restoration, primarily, selected the region where a problem occurred, called ROI (region of interest), and that area would be defocused a small amount resulting in the problem becoming less visible. To mask the fact that the image is less sharp, the contrast is increased to compensate, and the viewer has the illusion that the problem is resolved.
Modern restoration programs work at the pixel level. The software tries to fix the actual problem that a pixel has, or that a small group of pixels has, rather than try to mask it. Additionally, these newer programs are more complex and need more powerful computer processing for optimal efficiency. This assortment of tools was developed mainly by the game industry for their special effects. If the archivist uses them cum grano salis, then true restoration work can be performed, and not simply camouflage the existence of a problem. Just as importantly, any restorative interventions need to be carefully documented. It can be a difficult but essential task.
Recent developments
Significant advances have been made in the audiovisual and game industry software.
The accessibility of high dynamic range allows the image reproduction to contain a more accurate colour representation (for example of Dufaycolor, Kodachrome, or Technicolor), and can be used to improve audiovisual archiving.
The higher frame rate options allow the archivist to encode precisely the frame rates used during the silent film era. Also, by multiplying frames and adding black frames to emulate the projector's shutter blades, it gives a more accurate, as well as a more pleasant viewing experience. Additionally, theatrical access is improved dramatically.
Furthermore, other technological fields, such as forensic science, has developed new tools, allowing us to read all the information off of a magnetic tape, and to interpret it by using their software. Also, the algorithms developed for DNA sequencing can be used to process the optical soundtrack from an archival scan. This should be done frame by frame and edge to edge, with a little vertical overscan. By utilizing diverse technologies, these programs will open the most doors for more accurate conservation and restoration work.
Summa summarum: Let's emulate the sound of colour through the best techniques that are technologically available today. By keeping all the possibilities available for our successors, they will be free to debate, modify and improve our own work.
Having graduated in mathematics and computer science, Reto Kromer became involved in audiovisual conservation and restoration thirty years ago. He was head of preservation at the Swiss National Film Archive and lecturer at the University of Lausanne. He has been running his own preservation companies and lecturing at the Bern University of Applied Sciences and the Academy of Fine Arts Vienna. His current research includes colour spaces, CLUT, and codec programming and emulation.
Acknowledgments
I entered this field in late October 1986. During these past 30 years, I have had the pleasure to meet many individuals who have been fundamental in developing my career. I would like to thank, warmly, all the persons who have been most crucial for my own professional development: Marguerite Engberg, Alan Masson, John Pytlak, Dominic Case, Paul Collard, Luigi Pintarelli, Paul Read, Kris Kolodziejski, Martin Sawyer, Carole Delessert, Hermann Wetter, Rémy Pithon, László Gloetzer, Charly Huser, Sam Kula, Ray Edmondson, Jim Lindner, Grover Crisp, Michael Friend, Peter Adelstein, Jean-Louis Bigourdan, Charles Poynton, John Graham-Cumming, Nicole Martin, Dave Rice, Misty De Meo, Yvonne Ng, Agathe Jarczyk, and David Pfluger. My work and the work of the teams I have had the privilege to lead has always been collaborative — collaboration on different levels: between institutions, individuals, companies, and institutions, and/or individuals, and/or companies.
I also wish to acknowledge the help provided by Adrian Wood.
CHAPTER 2
BFI Film Forever: Unlocking Film Heritage
Charles Fairall
Abstract
Unlocking Film Heritage (UFH) is one of the three key strategic aims within the BFI's 2012–17 National Lottery-funded Film Forever strategy. With an ambition to digitise, preserve, and make accessible 10,000 film titles, the BFI has consulted with partners across UK collections to harmonise and document technical standards for both essence and metadata and negotiated a common tariff on behalf of the project for commercial digitisation and restoration services. All films digitised through UFH are being presented to the UK public through the web-based BFI Player, with the majority being free to view.
In addition to the curation, digitisation and presentation of 10,000 films, the BFI has built a state of the art data centre, and is creating a new Digital Preservation Infrastructure, designed to operate seamlessly with the existing collections database (CID). It will enable processes for ingest, media asset management, long-term preservation, and access to benefit all BFI digital collections. Through the UFH project, technology transformation is being applied across the BFI conservation centre with the introduction of the latest digital film image/sound scanning and restoration systems. While maintaining all essential analogue expertise, existing conservation staff, through a programme of training and hands-on practice, will develop new skills and processes, enabling full end to end archive workflows to meet the needs and demands for both analogue and digital preservation and access.
(Continues…)
Excerpted from "Sustainable Audiovisual Collections Through Collaboration"
by .
Copyright © 2017 Coordinating Council of Audiovisual Archives Association (CCAAA).
Excerpted by permission of Indiana University Press.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
Table of Contents
1. Introduction, Rachael Stoeltje
2. Let's Emulate the Sound of Colours! Reto Kromer
3. BFI Film Forever: Unlocking Film Heritage, Charles Fairall
4. New and Improved! Experiences from the Introduction of the National Library of Australia's Second Digital Collection Management System, Mark Piva
5. Changing Gears: Fast-Lane Design for Accelerated Innovation in Memory Organisations, Johan Oomen, Maarten Brinkerink, Bourke Huurnink
6. Organized Metadata Management System Using Auto-Generated Post-Production Scrip, Nobu Yamashita
7. Canalizing the Maelstrom of Metadata: Extensions on the Hourglass Model, Brecht Declercq
8. Using Film Annotation Tool as Part of the Restoration Process, Franz Hoeller
9. The Restoration of The Thousand-Stitch Belt (1937), Masaki Daibo, Tomohiro Hasegawa, and Kazuki Miura
10. Migration of Archived Video Footage: Challenges and Solutions for Hardware, Software, and Workflow Issues, Frank Pavuza
11. An Investigation into the Restoration of Wire Recordings in the 1950s, Nie Manying, Wang Xi, Zhang Shuxia, and Yan Jie
12. Findings from the Digitization of 78 RPM Discs, George Blood
13. Playback Methods for Phonogram Images on Paper, Patrick Feaster
14. Present and Future Applications of Optical Soundtrack Scan Technology, Gilles Barberis
15. Practical VisualAudio: A Long Journey to Tangible Results, Stefano S. Cavaglieri
16. Sound Reproduction of the First Speaking Films, Thierry Delannoy
17. Marginal Analysis of Digital Video Archives Restoration Project, Yang Haisheng and Jiang Huijun
18. Digital Video Damage in Archives: Detect, Repair and Prevent—Results from the DAVID Project, Peter Schallauer&Franz Hoeller
19. PREFORMA and the MediaConch Project: Open Source Tools for Open Standards, Erwin Verbruggen, Dave Rice, Bert Lemmens, Jérôme Martinez, Ashley Blewer, Emanuel Lorrain, Antonella Fresa, and Claudio Prandoni
20. From Tape To File: Solutions For A Data-Centric Migration Workflow in The Haus Des Dokumentarfilms for the Magnetic Tape Collection of the Landesfilmsammlung Baden-Württemberg, Michelle Carlos
21. Transfer Quality-Controlled Archive Digitization Approaches for Large Tape-Based Repositorie, Jean-Christophe Kummer
22. Review and Comparison of FFV1 Versus Other Lossless Video Codecs for Long-Term Preservation, Peter Bubestinger-Steindl
23. Hierarchical Storage and Managing Media Assets for the Libraries of the Future, Kia Sang Hock and Adrian Chan
24. The Role of Optical Storage Technologies in Future Digital Archives, Morgan David and Yuji Sekiguchi
25. The (Carbon) Black Ops of Recording Tape Sticky Shed Syndrome Exposed, Charles Richardson and Martin Atias
26. Neutralization of Critical Stages in Nitrate Film Degradation, Alfonso Espinsoa and Cesar de la Rosa Araya
27. Fighting the Decay: Permanent Refreshment of Acetate Media, Nadja Wallaszkovits
28. The Sticking Point: Dealing with Blocked Motion Picture Films, Mick Newham
29. Biocontamination at the French Film Archives: A Study of its Origin and its Remediation, Betrand Lavédrine, Daniel Borenstein, Marie Dubail, Benoit Furdygiel, Chantal Garnier, Martine Gillet, Geneviève Langlois, Thi-Phuong Nguyen, and Malalanirina S. Rakotonirainy