Virtual Production：Past, Present, and Future(1)

The digital environment in the film What we mentioned above are all about the production of CG sled dogs. There are many natural environments in the film, some of which are fully synthesized digital landscapes, and some are enhanced effects after real shots in Alberta, Canada. The two main scenes are Wheat Kinley Mountain shooting at Fortress Mountain, Alberta, and frozen Norton Bay shooting at Lake Abraham, Alberta. Real shot material 1. Shooting across a frozen lake in Alberta, relying on a lot of roto to complete The reason why I chose Lake Abraham in Alberta as the location for shooting the unstable frozen lake is that the blue ice here is very clean and clear, and the place is large. It's just that the weather on the day of the shooting was not very compatible. Two days later, the lake was covered with snow. Fortunately, the production team took a lot of reference photos within these two days. Nearly 95% of the shots are related to the sled dog and the owner Leonhard Seppala. To facilitate production, the entire sled team, sled, protective gear, fur, clothes, and Leonhard Seppala's hair need to be rooted separately, taking into account the sled dog body behind the team The cold atmosphere is stronger than the sled dogs in front, and the roto has to be created in layers. A real shot of sled dogs, replacing the environment and enhancing the atmosphere Under normal circumstances, it is sufficient to project the roto layer on the character's body trajectory or vehicle trajectory and then superimpose on it, but at this time the hair details are very complicated. You need to project the roto on the card, build a sled system as a card binding, and then add Each layer of particle FX, snow, atmosphere, etc. In addition, the snow and ice produced by the sled and sled dogs flying forward are simulated in Houdini and filled with some 2D elements. Real shot Final shot 2. Cracked ice The design of the ice-breaking lens is very complicated. The live shots were taken on a flat ice surface, so some ice flipped shots can only be done in full CG, while those non-CG shots used a method of destroying the lens track to make the ice surface look more" be more active, limit it to the range that does not destroy the parallax, use 2D techniques to constrain the fixed camera on a floating ice cube, or move the camera on the ice material, add 2D floating, and make the effect look less stable. Moreover, the material itself was shot on a flat ice surface with different light, and the lighting conditions are constantly changing with time. Therefore, the material of each version is different. It is necessary to keep these materials consistent and guide the ice surface. Fragmenting piece by piece is also a long and complicated process. 3. Simulate huge ice cubes In the sequence of the sled dog team crossing the frozen lake, huge blocks of ice will gradually rise as the ice surface breaks. The production team used procedural methods as much as possible to guide the shape of huge ice cubes. When there are a large number of huge ice cubes, there is no way to bind each piece individually, and it is impossible to carry out carving, texture, and appearance development processing on each piece. This will limit its shape and size. Once you want to make it If you modify it, you have to go back to the previous step and recreate it. So they created a new Cascade system that allows the layout department and the environment department to create huge ice cube layouts on a shot-by-shot basis. The Layout department created a very basic proxy shape in Maya and used Maya's curve tool to draw a huge ice shape, stretch it, place it in the scene, add binding constraints, and set up rough animations in the floating ocean. The environmental team has created a very practical toolset that can procedurally model huge ice blocks through basic geometry, generating broken edge details, internal bubbles, cracks, and faults in the ice layer. With the help of new tools, the work of the production team is basically all day shooting during the day and farm rendering at night. If you need to change the size or shape of the huge ice cube, go to the Layout link to redraw the curve and give it to the next process. The visual effects link will also simulate the interaction between huge ice cubes and water, including details such as bubbles and splashes, and finally, render in Clarisse. 4. The environment of other mountains In the film, the growth of Togo is described in the form of memories of Laosai, part of which takes place among the mountains. The production team took a lot of very beautiful mountain views and modified them on this basis. For example, when shooting in Fortress Mountain, Alberta, the director thought the environment was good but there were too many trees, so some processing was done in the later stage. For another example, in the shots of the settlers' houses, there are no Alaska mountains in the real shots, and they need to be added later. Real shot Final shot When designing the background, compared to the cumbersome work of drawing a large number of digital landscape maps, the production team adopted a 3D method that combined digital high-modulus, lidar scanning, and photogrammetry technology. In the end, the mountains that we saw close to the lens were sculpted, textured, and look-dev processes were completed. There were also some Clarisse renderings of trees, leaves, and rocks. In general, there was indeed a lot of background work that needed to be processed.

The dogfaces system for sled dog facial capture In order to produce the final version of the sled dog animation, DNEG also explored the dog's motion capture and spent time developing the Facial Action Coding System (FACS) system specifically for the face of the sled dog. 1. Research and development of sled dog motion capture In the early stages of production, DNEG used well-trained sled dogs for motion capture in Animatrix Studios. We usually see actors wearing professional clothing for motion capture. This time the same process happened on sled dogs. The way the equipment is worn and how to set tracking marks on the fur is very technically challenging. Although these motion capture data were not used in actual production, it was still a good learning experience for the production team in the future. 2. Research and development: dogFACS system In order to have a detailed understanding of the facial muscles of sled dogs, and to clarify the working direction of facial binding as soon as possible, DNEG also started dogFACS research very early. Since there is a human face-catching system, there can also be sled dogs. The researchers categorized all sled dog expressions, and drew facial expressions based on the dogfaces system, including "mouth raised" and "wrinkled nose". For example, a growl is "upper lip lift", "wrinkle nose", "wrinkle lips" "The expression of these three actions combined together. Facial control schemes based on these expressions can enable animators to activate or counteract individual expressions, thereby creating more detailed and believable animations. DNEG’s binding department has re-developed the entire four-legged binding system to simulate higher standards of real performance, including a new front leg module, which has the function of “fixing limbs” so that animators can imitate most animals. The fixed state of the front leg when the weight is on the leg; the reconstructed spine setting is also included to improve the realism and function of the animation. In general, the binding process requires a lot of effort. It is necessary to work closely with the animation department to develop and improve the four-legged binding standard, and always draw inspiration and direction from research references. 3. Roaring time There is a scene where a little partner named Ilsa in the sled team roars at Togo, and Togo immediately subdues Ilsa. Please note that the seemingly real performance here is actually a CG shot. In the actual shooting, the dog handler used a "snarling device", which uses a rubber band and a prosthesis to open the sled dog's mouth. It is said that the sled dog who plays Ilsa is very docile and very coordinating with the whole process, but its eyes and constantly wagging tail reveals its extremely happy state, and it doesn't look angry at all. Real shot material The effect after replacing the CG head The production team decided to use a real shot of the body + CG head to solve it, and designed some conceptual images of the sled dogs being angry and fierce. This production process is very challenging for riggers, modelers, and animators. They need to accurately grasp the subtle facial animations of sled dogs, and Ilsa's head production is more detailed and precise than Togo. After all, the lens is mainly In performance Ilsa. Real shot material Final shot The most useful part of the dogfaces system is this close-up shot of Ilsa's rant. FACS expressions and mixed expressions are linear, while the sled dog's mouth and nose can be moved at will. In order to have the authenticity of proper micro-motion, the production team added many small details next to its nose. It is said that when the roaring scene animation test was shown for the first time, it was played side by side with the real shot material, and most viewers could not see that it was fake. Real shot material Final shot 4. Sled dogs skating on ice There is a plot in the film. The sled dog team braved the snow and cold to cross the unstable frozen lake under the leadership of Togo. When they returned, the ice surface was torn apart, and the sled dogs were moving forward quickly while their feet were slipping. The DNEG team also began to find some materials on the Internet about sled dogs slipping or falling, to provide a reference for finding the unstable feeling of sled dogs standing on the ice. These materials will be put into the rough cut of the animation processed with Time Editor to create a suitable effect for the overall shot like building blocks, and also help them determine the performance of the sled dog in some specific shots. From the perspective of animation quality and details, this method should belong to a relatively advanced blocking. 5. Animate multiple sled dogs The above-mentioned ice skiing. In addition, there are 11 sled dogs in Togo. The behavior of the sled dogs in the team is different from individual performance. For this reason, DNEG has developed a binding system to determine each The distance between the sled dogs in the team, if the distance is too far, part of the system will be displayed in red. Although this system is not super accurate, it can still provide the team with an approximate range. The production team carried out some layout development work when processing the animation, set up many different cycles and different speeds for the state of the sled dog action, and input them into the binding system so that the layout effect can be freely switched and loaded before loading The method of setting animation after binding is much more convenient. The Layout department can set up the sled animation according to the selected sled dog sport Cycles and speed while ensuring the work efficiency while achieving the authenticity of the effect as much as possible. It seems that this part of their work is also very convenient.

The visual effects of this film are produced by DNEG, Lola Visual Effects, and Soho VFX. The most impressive thing in the whole film is the loyal and brave sled dogs and the local weather. DNEG began production after shooting in October 2018. The cycle lasted for about one year. A total of 872 artists from 4 different branch teams completed 778 VFX shots. Next, we will divide the parts made by DNEG (CG Dog and Digital Environment) into three parts to introduce to you. 3D scanning of sled dogs The husky sled dogs we saw in the film are not necessarily real, some are full CG, and some are CG replacements of a dog head. The sled dogs produced by CG need to match the real shots of the sled dogs. It will involve 3D scanning of real sled dogs, collecting reference materials, redesigning hair tools, building muscles and bones, and reconstructing the roaring appearance of the sled dogs. Live shot 1. Testing DNEG has done a rocket raccoon with 800,000 hairs in "Avengers 4: Endgame", and has also tested wolves in previous projects, but has not done a dog, nor has it dealt with something like this Such a huge fur/hair production in the film. For the "Togo" project, they updated the internal hair tool Furball. Through some development and optimization work, they first created the largest amount of hair on a single dog as much as possible, and then met the hair of 11 dogs in 1 shot. In addition to the real appearance, it is necessary to simulate the state of hair with water, ice, and snow and its rendering effect. 2. 3D scanning of sled dogs During the shooting, DNEG performed two photogrammetric scans of 30-40 sled dogs in the studio using Clear Angle, one wearing summer clothes and the other wearing winter clothes. The dog handler introduces a single sled dog into the studio, first familiarizes himself with the environment, and then takes him to the designated C position. The scanning process must be completed all at once, otherwise, there will be no chance to do it again if the dog is scared away. There is a separate shed next to the Clear Angle shed, which is equipped with an animation reference camera, which can capture the detailed dynamics and characteristics of the sled dogs outside of the actual shooting, and provide reference materials for creating CG character bindings and assets. 3. From data to sled dogs A 3D scan of a sled dog can get feet, legs, head, and rough body volume data, but not including fur/hair data. Without hair data, it is impossible to analyze the muscle mass and the tissues under the fur. The solution adopted by DNEG is to manually measure the fur on the sled dog’s neck, back, tail, and other specific locations with a measuring tape in the small shed mentioned above. However, this method of creation is more based on the dogs involved in ideal, anatomy textbooks, not necessarily data-driven, and some specific details of the sled dogs need to be added on this basis. Real shot Final shot The final creation is to customize the muscular system and the skeletal system, use Ziva Dynamics to create the muscle and fat system, Maya's nCloth to create the skin, the fur tool Furball to handle grooming, and Houdini vellum to handle the hair follicle dynamics. The production team found that a lot of fur movement actually comes from fat. The fur itself is very stiff and will not twist left and right. Instead, fat and muscles move around underneath, forming a feeling that the fur is moving. Of course, it also needs to simulate the effect of pulling the fur after putting on the protective gear for the sled dog. Real shot Final shot