Yes, the aging Harrison Ford in ‘Indiana Jones and the Dial of Fate’ is an impressive technological breakthrough from ILM that far surpasses the much debated work on ‘The Irishman’. No, they didn’t rely solely on artificial intelligence or machine learning. Instead, the de-aging for the 1944-set Nazi combat prologue (with Ford looking 35 years younger) was achieved with every tool in their VFX arsenal – known collectively as the “ILM FaceSwap” – and by more than 100 artists. .
It was necessary? Yes, according to director James Mangold, who told IndieWire’s Anne Thompson: “I thought there would be a way to take the audience through Harrison’s current era by greeting him first with an energetic homage to the previous films. He faces the audience with a young Indy in full bloom and full action, and then you make a sharp turn. He takes it all away and lets the audience address the difference between now and then in a brazen way. This was the cut I was most excited about for the film.
Related stories
‘Joy Ride’ director Adele Lim hits back at critics with her crazed comedy ‘objectifies men, targets whites’
“Sound of Freedom” is cleaning up the box office because you can buy tickets for strangers
The prologue, with the Nazis losing the war and hoarding their loot, also introduces the audience to the villain Voller (Mads Mikkelsen, also aged) and Archimedes’ discovery of Antikythera (the titular quadrant), which continues in 1969 when the search for the MacGuffin resumes. Ford was also rejuvenated for a flashback involving a fight over Antikythera in 1954.
The ambitious project was overseen by ILM’s Andrew Whitehurst, London-based visual effects production supervisor and creative director, and visual effects supervisor Robert Weaver. They benefited from Ford’s on-set performance (he began production at age 78), as well as the availability of hundreds of hours of footage from the first three “Indiana Jones” films. This allowed them to capture the actor in every possible way (especially the lighting conditions).
“FaceSwap is an umbrella term for a variety of different technologies,” Whitehurst told IndieWire. “We knew intuitively when we started this project, but it proved true very quickly, that there wasn’t going to be a technological magic wand that was going to work for every single take.
“So we knew we were going to use a lot of references from the previous films,” Whitehurst continued. “We knew we wanted to shoot with more cameras, we knew we’d have to build a fully realized 3D CG head, use some machine learning, use painting techniques, use a keyframe animation, use lighting effects, everything. And FaceSwap is the technology of taking one face and replacing it with another face, whether it’s someone completely different or someone of a different age.”
The key was to build a suite of tools so artists could select the correct balance of components to achieve the desired result for each shot. “I think that was the big step forward for us conceptually,” added Whitehurst. “And I think we’ve probably surprised ourselves on a few occasions with what worked on which shots and why.”
The burning issue of artificial intelligence (which is a topic of concern in the ongoing SAG-AFTRA and WGA negotiations and is the driving force behind IATSE’s core principles on the matter) has played a useful role via machine learning. The proprietary program compiled and analyzed all existing reference material in 2D, including how the age-specific face should behave. “This repository of the franchise was just a treasure,” Weaver told IndieWire, “from the direct reference aspect to the look and believability of what we needed to achieve, plus the ability to feed that image into a machine learning process .”
But the driving force was Ford’s performance, powered by an improved version of the light-based capture system used in “The Irishman” called FLUX (integrated by Maya for body animation). “He AND the character and he’s in terrific shape,” Weaver added, “and so having that and being able to visually reference the previous films, being able to blend those two together, got us where we needed to be.”
Lighting was essential. As with “The Irishman”, the main unit cameras had additional infrared cameras on either side to capture extra information to help ILM build a 3D version of the face based on the photograph, useful for then extracting how the light was hitting that face .
“We also shot the lighting references for each individual high dynamic range camera setup,” said Whitehurst. “We scanned every single set so we knew exactly where everything was, including the lights. But often what you need comes down to using all these references but then going through the artists to use their skill and judgment to mark this light up a bit because it’s going to add this best bit of modeling to the face or whatever And.”
Mangold was so pleased with the early results that he added extra action beats to the opener, which begins at night and ends at dawn and revolves around Indy escaping the exploding castle and fighting bad guys on a motorcycle, in a in the car and on top of a moving train.” So we often found ourselves, frame by frame, going back to old footage to see exactly the idiosyncratic nature of Harrison’s particular performances,” Whitehurst said. “And the wry smile he has and the way he moves his face.”
The finishing process of each single shot has been defined as under painting. “The subtlety of likeness and performance can’t be tied to one look,” Whitehurst said. “You can’t say it’s just the physical location of one face at a time. Much depends on how it is lit. If you illuminate someone from different angles, you read the performance in a completely different way. We were able to generate low resolution versions of a face very quickly for editorial.
“So when they were editing, they were never editing exactly what we shot that day,” added Whitehurst. “There was a window around, but there were the beginnings of a period-appropriate performance. And that meant we got, I think, better notes from the cuts about the performance and how it all worked together. And that gave us a big step when we were working on improving it, adding fidelity and refinements to that performance.
The latest advances in animation allowed them to reproduce details as fine as pores and capillaries while “painting those maps that then make up the holistic look of our Indy hero,” Weaver said. “And there is the dispersion of the subsoil that you make up right in the areas. It’s never an approach where you just say you need to apply subsurface scattering. It’s more like if it works here, it doesn’t quite work here, and it refines them into a very specific set that you’re looking for.
“It would have been nice if we had a single recipe that we knew we had for one shot and now it’s going to be the holy grail for the rest of the shot’s work,” added Whitehurst, “but that’s not the case. Each shot is so individualistic and has required different feedback and artistic components to manipulate in the final performance.
.
