David Zimmerman Shares Best Practices of Data Management, Storage, and Recovery with Fillmmakers
Filmmakers work tirelessly to transform their visions into footage, using their talents to get the most out of every setting and performance. Unfortunately, sometimes filmmakers do not pay enough attention to the technical side of their creations, specifically how to properly manage their camera’s storage capabilities and how to keep their footage safe.
The output and the way new cameras write data is different from manufacturer to manufacturer. This means each type of camera has its own formatting rules in terms of how it “writes” data to the card. The cards are typically CFast, SD, Micro SD, or the Sony proprietary SxS, with storage capacities of usually 256 GB. The cards are higher capacity to account for the large file sizes and are also able to better handle the speed of higher-end cameras which produce stunning imagery and crisp sound.
Older movies were created as a single streaming file, but modern cameras have audio and video as separate files and use XML to sync them together. When these files become corrupted, their inherent complexity means it’s not an easy fix. When files were outputted as AVI or MOV and there was a problem, the truncated file would still play. But now corruption often means the file is damaged to a point that it will not play at all.
Preventing Damaged Storage Cards
Often the cinematographer does not recall exactly what was happening when the card malfunctioned. It can simply be the battery running out during shooting, which interrupts the camera’s ability to properly “write” the data to the card. The files involved with modern high-end cameras are so large that the recording process is faster than the actual file writing process. There’s a delay between the two actions, and if the camera operator ejects the card too quickly then it cannot sync properly. So the cinematographer might see a card is full and then eject it too soon, and then not realize until later that they lost some of their footage. After shooting a scene, wait for the write or activity LED to stop blinking to give the card ample time to finish writing any cached data to the storage media.
There are other reasons cards can become unusable:
- The operator might have accidentally formatted the card in another device, which means it will not work properly with the camera. Always format the card in the camera you are going to use.
- Switching cards between two different camera types or brands. The differences in formatting structures will not be compatible and can cause errors.
- Static electricity can damage the cards, making them unreadable.
- Exposing the cards to moisture, from spilled water or coffee to condensation from changes in humidity.
Create a Data Plan
An independent filmmaker does not need an 80-page corporate data management plan, but they should create a condensed version that adds some structure to data protection. A formal plan instills accountability and forces you to think of your footage as a valuable asset that must be protected. You also want to test and refine the plan periodically, to ensure you and any crew members are always following the best practices for data storage and management.
The plan should provide a road map for all of your current data sources, including files that contain info on shooting locations, story ideas, and other back-end information. And of course you’ll want to account for all of your footage, detailing where it all currently resides and where you will store it in the future. Centralize the data to a few trusted storage options. Perhaps you have some old footage in a Dropbox account that you haven’t used in a year, but want to use for a new project. Log in and move the files to another cloud provider along with your current work. Organizing the data on the front end means you won’t have to waste time scrambling to find content during a shoot and ensures you do not lose some “hidden treasures” from past work.
The Importance of Backups
A fatal error for filmmakers is to use their camera’s SxS or CFast cards as long-term storage solutions. The cards are meant only as a placeholder for your film, because they’re fragile and susceptible to errors and corruption. Once they are full, they should be promptly removed and the content should be duplicated for storage.
Copying the files to a cloud-based storage provider is essential. Moving a movie from a 256 GB card to the cloud can take a day due to the file sizes involved, but it’s an inexpensive way to safeguard your film. You should also keep the original files on the card and then store the card in a fireproof media safe for protection. The cards can be relatively expensive, but you should buy new cards for every project to avoid the formatting issues and to have an instant file backup. Another option is to use a private server to hold the files, which limits the chances of a security breach.
Redundant file backups are essential, so create “backups of your backups” by also moving files to a high-capacity external hard drive or a second cloud provider. It’s a classic risk-reward where you are protecting your precious works from damage while outlaying a minimal expense.
A recovery firm that specializes in film footage can be a director’s best hope for saving their scenes.Advanced recovery firms will receive cards from the customer and are typically working off of limited information. The client often does not know the cause of the card’s corruption (unless it was physically damaged), so the recovery company must perform several functions to try to image the media. They will use both Linux and Windows to try to access the data, and once the imaging process is complete they’ll run proprietary tools to recover it in a usable form.
If you must use a recovery company to save footage, ensure they support high-end cameras that utilize faster cards and shoot at much higher resolution. File formats change frequently, and you want a recovery partner that dynamically adjusts to the latest formats for cameras including the Nikon D500, Sony 7R II, Panasonic DMC GH4, Fuji XT-1, and various others.
Avoid trying to recover the content yourself using a free utility you found on the web, as most of these do not work and they’re often riddled with malware. You also want a recovery firm that is experienced with protecting valuable footage, understands the need for signing NDA’s, and has the physical and network security in place to protect your footage while it’s in their possession.
Understanding the best practices of data management, storage, and recovery should be in the toolkit of every filmmaker who wants to protect their footage for years to come.