Archive for the ‘Craft of the Video Shooter’ Category

With the advances in artificial intelligence, post-production rotoscope tools are now so good that some DPs are asking if we still need to use green screen at all, or at least, in quite the same way.

Suddenly, it seems any background can be replaced in seconds, allowing DPs to shoot the most complex compositing assignments faster and more economically. Today’s AI-powered rotoscoping tools are powerful and robustl as producers can repurpose a treasure trove of existing footage packed away in libraries and potentially reduce the need for new production or costly reshoots.

In 2020, Canon introduced the EOS 1D Mark III camera for pro sports photographers. Taking advantage of artificial intelligence, Canon developed a smart auto-focus system by exposing the camera’s Deep Learning algorithm to tens of thousands of athletes’ images from libraries, agency archives, and pro photographers’ collections. In each instance, when the camera was unable to distinguish the athlete from other objects, the algorithm would be ‘punished’ by removing or adjusting the parameters that lead to the loss of focus.

Ironically, the technology having the greatest impact on DPs today may not be a camera-related at all. When Adobe introduced its Sensei machine-learning algorithm in 2016, the implications for Dos were enormous. While post-production is not normally in most DPs’ job descriptions, the fact is that today’s DPs already exercise post-camera image control, to remove flicker from discontinuous light sources, for example, or to stabilize images.

In 2019, taking advantage of the Sensei algorithm, Adobe introduced the Content Aware Fill feature for After Effects. The feature extended the power of AI for the first time to video applications as editors could now easily remove an unwanted object like a light stand from a shot.

The introduction of Roto Brush 2 further extended the power of machine learning to the laborious, time-consuming task of rotoscoping. While Adobe’s first iteration used edge detection to identify color differences, Roto Brush 2 used Sensei to look for uncommon patterns, sharp versus blurry pixels, and a panoply of three-dimensional depth cues to separate people from objects. 

Roto Brush 2 can still only accomplish about 80% of the rotoscoping task, so the intelligence of a human being is still required to craft and tweak the final matte.

So can artificial intelligence really obviate the need for green screen?  In THE AVIATOR (2004), DP Robert Richardson was said to have not bothered cropping out the side of an aircraft hangar because he knew it could be done more quickly and easily in the Digital Intermediate. Producers, today, using inexpensive tools like Adobe’s Roto Brush 2, have about the same capability to remove and/or rotoscope impractical objects like skyscrapers with ease, convenience, and economy.  

For routine applications, it still makes sense to use green screen, as the process is familiar and straightforward. But the option is there for DPs, as never before, to remove or replace a background element in a landscape or cityscape where green screen isn’t practical or possible.

Can AI-powered rotoscoping tools like Adobe’s Roto Brush really replace green screen? Some DPs think so, especially in complex setups such as many cityscapes.

Roto Brush has learned to recognize the human form, and is thus able to isolate it, frame by frame, from a background. But even with the power of AI, Roto Brush still requires some human input.

[Screenshot from CM de la Vega ‘The Art of Motion Graphics’ https://www.youtube.com/watch?v=uu3_sTom_kQ]
.

The elimination of the mirror box from DSLRs brings significant benefits to shooters, as evidenced by the raft of new full-frame cameras around, like the Panasonic Lumix S1, Sony Alpha 7, Canon EOS R – and the Nikon Z6.

The 24Mpx Z6 features a full-frame ‘Z’ 55mm mount – the widest available of any DSLM [digital single lens mirrorless]. This enables Nikon to more efficiently utilize the full width of the camera sensor. More practically for shooters, the elimination of the mirror box enables lens manufacturers to produce more compact, lighter weight zooms because the lens mount can be placed further back closer to the sensor.

Nikon’s shallow 16mm flange focal distance is the shortest among the major DSLM manufacturers. For shooters, this means that Nikon’s ‘S’ wide-angle zooms can be made with a less bulky front element group – a welcome development for shooters feeling burdened by the front-heavy optics of the past.

The 58mm Nikkor Z f/0.95 Noct lens produces truly amazing full-frame pictures; the lens creates a distinctive flare and coma around point sources that gives shooters a wide range of new options along with an extremely narrow depth of field.

When designing a drive, engineers assume that some energy will always be applied. Leaving any drive on a shelf without power for years, will almost certainly lead eventually to lost data. For those of us with vast archives stored on umpteen drives sitting idle in a closet or warehouse, the failure of a single large drive or RAID array can be devastating.

Mechanical drives are inherently slow, relatively speaking, so moving large files can significantly reduce one’s productivity. Beyond that, the high failure rate of mechanical drives is an ongoing threat. The distance between the platters is measured literally in wavelengths of light, so there isn’t much room for dislocation of the spinning disks due to shock. This peril from inadvertent impact is in addition to the normal wear and tear of mechanical arms moving continuously back and forth inside the drive.

For documentary shooters operating in a rough-and-tumble environment, the move to flash storage is a godsend. SSDs contain no moving parts, so there is little worry from dropping a memory card or SSD while chasing a herd of wildebeests, or operating a camera in a high vibration environment like a racecar and fighter jet.

SSDs also offer 100x the speed of mechanical drives, so it’s not surprising that HDDs these days are rapidly losing relevance. The lower cost per gig still makes mechanical drives a good choice for some reality TV and backup applications, but as camera files grow larger with higher resolution 4K production, the SSD’s greater speed becomes imperative order to maintain a reasonably productive workflow.

Right now the capacity of mechanical drives is reaching the upper limit. Due to heat and physical constraints, there are only so many platters that can be placed one atop of the other inside an HDD. SSD flash memory, on the other hand, may be stacked in dozens of layers; with each new generation of module offering a greater number of bits. Samsung is moving from 256Gb flash chips in 48-layers to 512Gb chips in 64-layers.

So how much should you exercise your HDD and SSD drives? Samsung states its consumer drives can be left unpowered safely for about a year. In contrast, enterprise data drives found in rack servers are designed for heavy use with continuous data loads, and offer only a six-month window of reliability without power. Such guidelines are vital to keep in mind as some shooters may not use a particular drive or memory card for many months or even years, and for them, it is important to power up their SSDs from time to time, to ensure a satisfactory performance and reliability.

One other thing.  SSDs have a limited life expectancy. The silicon material in flash memory only supports so many read-write cycles, and will, over time, eventually lose efficiency As a practical matter, the EOL of solid-state drives should not pose much of a problem however. Depending on the load and level of use, most consumer SSDs writing 10-20GB per day have an estimated EOL of 120 years. Most of us I would think will have replaced our cameras, recording media, and storage drives, long before then.

All drives, including SSDs, require regular exercise. Ordinary consumer drives should be powered up at least once a year to maintain reliable access to stored data. Professional series and enterprise-level drives require powering up twice as often, about every six months, to ensure maximum efficiency and long life.

We all know that shooting in log can improve exposure latitude and dynamic range. In short, log capture allows us to record more professional-looking images, as the brightest highlights in sun-dappled scenes, for example, may often be accommodated without clipping or loss of detail.

Practically, shooting in log may also mean utilizing less fill light, which can be helpful on low-budget run-and-gun style productions. I always try to reduce the amount of gear in general on a set; and shooting in log can help reduce the grip and lighting complement substantially – a good thing in my opinion.

Remember more gear = less work.

Giving you an idea of how I work in log, when shooting with the VariCam LT, for example, I assign Y-Get to user button 1. Placing the EVF cross-hairs over the brightest part of a scene, with the iris adjusted to 69 IRE, the camera recording V-Log will accommodate 3-4 stops of additional latitude before clipping or a noticeable loss of highlight detail. Setting exposure in this way is simple and effective, and eliminates the need for external exposure meters and pricey reference monitors.

While shooting in log makes sense for most shooters, broadcast news and some non-fiction shooters may find the hassle and inconvenience of working with LUTs not be worth it, or even possible, given the tight time constraints and lack of serious post-production in most news and public affairs programming.

This scene captured in log is shown with and without a LUT applied. Despite the advantages of shooting in log, some producers may see working with LUTs as a hassle and an inconvenience.

Since the advent of tapeless workflows it has been necessary to safely and securely offload camera original footage from memory cards and onboard drives.  For many of us this can be a process fraught with trepidation, which is why I and most industry professionals use Imagine Products’ ShotPut Pro to handle the offloading chore.

Of course we need an effective checksum to verify the integrity of the transfer and SPP has done that very well for almost a decade. In that regard ShotPut Pro version 6 retains the key checksum options from fastest to slowest; most data wranglers I know will opt for XXHASH (the fastest option) or MD5.

One valuable new feature in ShotPut Pro’s latest version is the PAUSE & RESUME function since on so many productions these days we are pressed for time and forced to interrupt the transfer of large data files. On many shows there are simply not enough hours in a day to permit the uninterrupted offload of the gargantuan 4K and higher resolution camera files.

Provided that the application is not closed or quit, SPP6 will resume and complete the transfer of large files without reinitiating the entire transfer. SPP6 completes the immediate file in progress so it does not truncate or interrupt a file prior to pausing or interrupting the transfer.

How many times have we been forced to move from hotel room to a moving vehicle and into a new hotel room while attempting to offload large camera drives and media cards? Shotput Pro 6 addresses the needs of frazzled data wranglers in precisely this unenviable position.

ShotPut Pro 6’s new PAUSE & RESUME feature enables users to interrupt a long data-heavy offload for completion later. This ability to transfer large files in intervals is long overdue!

This summer I am again in East Africa leading a camera and visual storytelling workshop at the Zanzibar International Film Festival. The hunger for knowledge in this part of the world never ceases to amaze me as my students demonstrate an eagerness to learn and practice the fundamental lessons of good visual storytelling and effective camera operation.

Zanzibar is located off the coast of East Africa  at 6º south latitude. The light at dusk is positively mesmerizing. Here a traditional dhow passes off the coast of Stone Town 8 July 2016.

Despite the technical challenges DSLR cameras dominate the filmmaking landscape in this part of the world.

The opportunity to integrate a range of local color is a great advantage of shooting in Zanzibar.

Lulu. The star of our scenario. In real life she is a popular fashion model.

The emphasis of local cinema is on actors and performance which is as it should be. Technical issues aside it is after all what audiences really care about.

On many shows we are increasingly shooting RAW and/or in a multitude of compressed formats with different cameras utilizing different flavors of log.  ARRI Alexa, Canon C300, Sony FS7, Panasonic VariCam, GoPro, Blackmagic, DSLRs – just keeping all the color spaces and log profiles straight can be a major challenge.

The Academy Color Encoding System (ACES) has simplified things for folks at the high end of the food chain. But for the rest of us toiling in typical broadcast productions and independent features, the convoluted post-camera wrangling of color and LUTs has become an unwelcome hassle with the wholesale wrangling of .cube, .aml, and .ctl files.

Thankfully we now have Latice, a powerful and versatile LUT management tool that greatly minimizes this ongoing hassle. To be clear it is not intended to compete with or replace grading applications like Davinci Resolve.  Think of it more as a LUT Swiss Army Knife, able to view, transcode, and conform, a wide array of color spaces and profiles.

So if you’re shooting B-roll on a Canon C300 Mark II and we need to conform to the A camera which is a Sony FS7, Lattice can convert the Canon Log2 files to Sony S-Log, which we then import into Resolve for simple and straightforward color grading in a single consistent color space.

The Mac-based app features a very straightforward interface, which offers plenty of hooks for tweaking.  As cameras like the Panasonic VariCam 35 are enabling the creation of 3D LUTs in camera it becomes a simple matter in Lattice to convert Panasonic’s V-Log file to something else, like Sony S-Log or Blackmagic’s Film Emulation (BMD) LUT.

Lattice doesn’t eliminate entirely the complexity and hassle of wrangling LUTs post-camerabut it sure makes the ordeal a whole lot easier.

I use it regularly and recommend it.

Lattice is a very simple LUT management tool. If you shoot with multiple cameras and employ various color spaces Lattice will conform the different LUT flavors to a single format for grading inside Davinci Resolve (or other color correction environment).

With the advent of post-camera filter software like Tiffen’s Dfx4 and powerful color grading tools like Davinci Resolve, it’s easy to see how some shooters, even some relatively accomplished ones,  no longer see the need for an on-camera filter. But this can be a big mistake. Especially given today’s onslaught of low- and mid-level 4K camcorders and DSLRs that produce overly harsh albeit very high resolution images.

Truth is, the impact of some optical filters cannot be effectively recreated in post. The polarizer, for example, is the only filter capable of increasing contrast and resolution;  it is simply not possible to add picture detail post-camera if the detail was not captured in the first place.

Then there is the matter of finishing filters like the Tiffen Satin and Black Satin, and the Schneider Digicon and Black Magic types. These filters have become more or less obligatory in recent years, to mitigate the clinical brash look characteristic of many low-cost 4K cameras. The Blackmagic URSA, AJA Cion, and other relatively economical camcorders and DSLRs, often exhibit very poor shadow integrity along with a harsh roll-off in the highlights, which can only be effectively ameliorated by a proper finishing  filter.

Today’s sharper diffusion filters from Tiffen, Schneider, and others, produce very little scatter and halation, and for all intents and purposes, are invisible to the viewer, but produce a more flattering look with lower noise in the shadows and smoother more pleasing highlights. Most importantly, these filters maintain sharpness in the pupil of the eye, while subtly blending and softening the skin tones and around the eye sockets and face.  This is possible because these filters are designed from the outset with the proper telecentricity to accommodate modern digital sensors with deep bucket photosites.

Older series Black Pro Mist, Soft F/X, Fogs, and Double Fogs, produce too much scatter to be useful with modern cameras fitted with CMOS sensors. Still, having said this, I do find a 1/8 Tiffen Black Pro Mist filter can be useful to match certain type vintage optics, like an old Cooke zoom, to the latest generation Zeiss CP.2.

Economical 4K cameras like the Blackmagic URSA tend to produce excessively sharp, clinical-looking images. A proper finishing filter like the Tiffen Black Satin is imperative to glean the best possible performance from such cameras fitted with high resolution CMOS sensors.

The data and power cables poking in and out of my MacBook Pro are pure crap. Take a look at how I’m managing my AC charger cord.  I’ve added three cable ties over a sleeve of black camera tape to ensure adequate stress relief and thus forestall the eruption of arcs and sparks from the broken connection.

Why can’t cable manufacturers simply provide sufficient stress relief in the first place?

The matter of diseased trouble-prone cables is not a new phenomenon. After 35 years in the business I know from hard-won experience that 90% of failures in the field are due to defective cables and connectors. Thinking about it now my ARRI 16SR in 1976 was a true godsend. The camera utilized onboard batteries that eliminated completely failure-prone cables and plugs. It gave me great peace of mind that this disproportionate cause of failure was gone forever.

Thunderbolt is an impressive technology and we’ve grown to rely on the less than robust cables for our most critical tasks from off-loading original camera footage to preparing our backup volumes. With data rates up to 40Gbps in Thunderbolt3 the flow of data is sensitive to a range of cable snafus, especially at the point where the cable enters the rigid connector where most failures due to fatigue occur.

For manufacturers, the cost of producing cables with proper stress relief might amount to a few pennies per unit, but as shooters and content creators whose business is capturing, transferring, and managing critical data, the extra dollar or two per cable at retail is worth it, if only to avoid the indignity of having to affix a raft of cable ties simply to ensure a solid and reliable connection.

A cable usually fails at the point it enters the solid connector. Affixing a series of plastic cable ties can provide increased stress relief and help forestall an intermittent connection.

One of the truisms of our profession has long been a camera system is only as good as its optics. While computational non-optical lenses in devices like the iPhone have obviated the need in some applications for finely crafted optics, the demand persists for high-performance glass at the high end of our business, especially in light of the latest 4K and higher resolution cameras.

The new Sony PXW-FS7 4K camcorder is well-balanced and robust, with superb ergonomics. Particularly notable, however, is Sony’s own 28-135mm F4 lens that comes with it.  It is not the typical crappy package lens that usually accompanies new mid-range cameras.

Employing precisely sculpted aspheric elements and low-dispersion glass the FS7 4K zoom represents a real breakthrough in economical s35mm lens technology. Remarkably free of chromatic aberrations – the main reason cheap lenses look cheap – the lens compares favorably with much pricier optics; its relatively slow F4 maximum aperture posing less of a challenge these days for shooters employing cameras like the FS7 that shoot virtually noise-free at ISO 2000 and higher.

Sony’s lens uses precise servo control of zoom and focus to enable a constant F-stop throughout the zoom range. Some shooters will object to the limited 5:1 zoom for documentary work but keep in mind the lens is a large format zoom.  If you’re looking for a 23x capability you should consider a 2/3-inch camera like the Varicam HS, which is ideal for travel and sports, and certain wildlife titles, that tend to employ long zoom telephoto lenses.

Sony’s new 28-135mm E-mount zoom offers very high performance in an economical S35mm format lens.

The zoom even at maximum magnification  is remarkably free of chromatic aberrations.