Sony Music Entertainment released a special version of the music video of the artist idiom, who sings the theme song "GLOW" for the Fuji TV drama "Koisei no keeper".


In fact, part of this video includes the world's first attempt to fuse two cutting-edge imaging technologies owned by the Sony Group. In this scene, a gigantic translucent "alter ego" appears behind himself and performs with him (Fig. 1 ). At a glance, you can confirm that the image of the other person's alter ego is real, different from ordinary CG (computer graphics).

Barjees lifestyle

Figure 1 Co-starring with a real alter ego
A scene from a music video that combines two cutting-edge video technologies, "virtual production" and "volumetric capture" (source: Sony Music Labels.Inc)


In fact, this scene was produced by fusing two video technologies, "virtual production" and "volumetric capture".


In virtual production, background images such as 3DCG (three-dimensional computer graphics) are projected onto a super-large LED display called an "LED wall", and the subject acting in the real space in front of it is filmed to make it look as if it were real. It is a method of producing images that look like they were shot on location. Recently, it is also used for filming movies and commercials.


Volumetric capture, on the other hand, is a technology that captures subjects and spaces with a large number of cameras surrounding a studio and converts them into 3D digital data. The advantage is that the subject can be converted into 3DCG as it is.


The scene in idom's music video was created using volumetric capture, and the background video including his 3DCG was played back on an LED display . 2 ).

Barjees Lifestyle

Fig. 2 Shooting in the virtual production studio
The photo is dark and hard to see, but there is a camera in front, and Mr. idom is singing passionately in front of the image of his alter ego projected on the LED display (Source: Sony Music Entertainment)


Of course, it's not impossible to use CG to create characters that look like they're real, just like in Hollywood movies. However, this requires a huge amount of labor and cost, and ``CG is fine if it is a still image, but it is extremely difficult to reproduce the natural movement of humans. Mr. Yasushi Ikeda, General Manager, v-Tech Section, Business Exploration Group, Business Exploration and Technology Strategy Division, Group R&D Center). Volumetric capture, on the other hand, does not have these problems because it is based on live action.


There is another reason for using volumetric capture. In virtual production, we use a technology called "in-camera VFX (visual effects)" that changes the background image displayed on the LED display in conjunction with the movement of the camera. This makes it possible to capture images that look as if they were actually shot at the scene.


However, in virtual production so far, still images have often been used for background CG, and there have been cases where people have pointed out that the resulting images are ``not realistic'' or ``no movement''. With volumetric capture, the movement of the subject can be converted into 3D 360° video CG as it is, so it is said that it is possible to create a background that perfectly matches the angle of view of the camera that moves in three dimensions at the shooting site.