Expanding entertainment horizons by extending and connecting space-time with 5G Haruyuki Moroishi, CEO and CCO, IMAGICA EEX
-5GMF Millimeter Wave Dissemination Promotion Workshop / Japan's Industrial Technology Frontline Report

How are technologies such as those represented by 5G being used in the fields of entertainment and art in real events? The 5th Generation Mobile Promotion Forum (5GMF) held its third workshop on "Japan's Industrial Technology Frontiers" in March 2024 to promote the spread of millimeter wave. The aim was to provide an opportunity to interact with a wide range of people outside the telecommunications industry, and the speakers for the third session were IMAGICA EEX and Mori Building, with Haruyuki Moroishi, CEO and CCO of IMAGICA EEX, and Tadashi Sugiyama of the TOKYO NODE Management Office, Mori Building Co. In the first part of this report, we report on Mr. Moroishi's presentation, "New Horizons of Entertainment Fusing Real and Digital.

I have been the president of IMAGICA EEX since 2020. I like new things myself, so I am in charge of producing projects that combine creativity and cutting-edge technology, and designing new experiences that combine ultra-high definition video, digital twin, spatial projection mapping, and so on, by keeping an eye on technology. I am also in charge of new experience design that combines ultra-high definition images, digital twin, and spatial projection mapping.

The concept of IMAGICA EEX puts Experience Design (XD), or experience design in the middle, which installs new experiences in society by thinking about the future, future thinking and technology. Our mission is to solve social issues and design a prosperous future by running with partners from various prototyping to social implementation. Our three business areas are Entertainment, Communication, and Society. We are promoting XD and experience design in the middle of these areas.

When creating either entertainment or services, I believe the most important thing is imagination, human creativity. When you think about it, the evolution of technology has made the canvas very large and expanded the range of expression. We, too, believe that we can design rich and free experiences that are free from frames, not just 16:9 or content on a movie theater screen, in a world where video and communication technologies are fused together.

5G/ 6G, IOWN changes the representation of space

Among these advances in imaging and communication technologies, we are now entering an era of advanced communication technologies and environments, represented by 5G/6G and NTT's next-generation communication infrastructure concept, IOWN. What kind of entertainment and experiences will be created in this environment?

In the era of ultra-high-speed, high-capacity communications, ultra-low latency, and the creation of new value through interactive situations created by 5G and 6G, NTT and the IMAGICA Group have begun a comprehensive study on the expression and staging of real cyber fusion spaces in the IOWN era from December 2022. In December 2022, NTT and the IMAGICA Group began a comprehensive study of the expression and staging of real cyber fusion spaces in the IOWN era. This is an effort to determine what kind of experiences, such as expressions, performances, contents, and services, will be presented to society as the telecommunication environment is updated.

I will introduce our efforts with a few videos. The first is live viewing with real-time transmission of 12K video and lighting. 12K video, lighting, and decorations are all synchronized from one place to another in a project that becomes a transportation system. By transmitting not only the video but also the lighting in synchronization with the live venue, a sense of unity can be achieved even in remote locations. In the project, the actual artists were in Yamagata and the 12K video and lighting were transmitted to the live viewing venue at the International Forum in Tokyo.

The other is Geo-Cosmos at the National Museum of Emerging Science and Innovation (Miraikan). Geo-Cosmos is a sphere that reflects the earth as it really is, expressed digitally. There is no set, and the space itself becomes the media.

Based on our collaboration with NTT, we are presenting a new UI/UX presentation for the IOWN era, and we are challenging ourselves to express the future using various data at the NTT R&D Forum. This is a project called "TENGUN Ogijima Project," in which we are creating a metaverse space by scanning all of Ogijima Island in the Seto Inland Sea with a point cloud. By entering the metaverse space as an avatar, visitors can relive Ogijima. You can experience a photo-realistic metaverse experience while preserving the realistic shape of the island. Currently, the point cloud is in a static state, but in the future, as the communication environment and the processing power of PCs and smartphones improve, we believe that a new digital twin world will be connected, such as this leaf moving and time changing.

Usually, for example, in dome concerts or halls, there is a stage in front of you and the audience behind you, and you are enjoying a live performance from one direction.

4 possibilities with EXDX, where the experience itself changes

In the possibilities of communication, we believe that it is not just a matter of technological evolution, but the very experience of oneself is changing. We feel that "EXDX" can digitally transform that experience design. I believe that the two key words for a world where communications are combined with entertainment are "time-space extension" and "time-space connection. We believe that such entertainment and a world in which people can access various things beyond time and space can be realized. I will explain the possibilities from four aspects.

One is the Live Experience. Until now, video has had technical limitations, and the format of live performances, for example, has been formatted. We believe that there is a possibility to free it from these formats. We believe that expressions and staging that were thought to be unusable for live performances because the data is too large will be possible in real-time sessions without delay by connecting to another venue.

This means that it will be possible to have sessions that transcend time and space, not only between the present artist and the audience, but also, for example, with someone who existed in the past or someone from the past. I believe that an era will come when it will be possible to stage real-time sessions with one's avatar or a performer at another venue. There will also be new ways of performing together that will connect places around the world. We believe that live performances and events connecting Tokyo and Dubai will become possible using communication technology.

The second is "Immersive Live Viewing," which, as we introduced live viewing of 12K video and lighting, builds a synchronized immersive space by transmitting not only video but also all production data, including lighting and decorations. Even if you want to go to a live concert but it is difficult to go to New York, and even if it is difficult to go to an actual live concert, we believe that immersive live viewing can provide value to the experience.

I believe that high-speed, high-capacity, ultra-low latency makes it effective not only for music but also for sports and other activities. Especially in the MLB in the U.S., it is difficult for home fans to go away to cheer for their favorite team, so if such facilities are permanently installed in their home locations, they will have more options for such experiences, such as cheering with their friends in their hometowns, in addition to going to their hometowns.

The third is Ability Transportation. This is a departure from entertainment, but it is about transmitting one's abilities, skills, and experiences to others with low latency. For example, I think that training, learning from sports professionals, will be updated in a way that allows people to experience it simultaneously from remote locations.

The fourth is "Digital Twin," an application called "XR City" by NTT ConoQ, the company behind XR, that allows users to collect extinct paleontology and restore realistic paleontology using AR, called "LOST ANIMAL PLANET. We worked with NTT Docomo to create an event to update this content using millimeter waves. We created a theme park by bringing back to life animals that had become extinct, using video technology and communication technology. The event was held at Tokyo Skytree in October 2023 as an event called "XR Extinct Zoo. Children in particular experienced dinosaurs and other animals with sparkling eyes.

What used to be a finite experience is now unlimited through communication technology.

We have introduced four telecommunications x video initiatives and worlds. We hope you have been able to understand that the world of entertainment is undergoing a dramatic transformation. Finally, we have summarized how communication technology is changing society and the world.

Earlier I mentioned the key words "space-time extension" and "space-time connection". When we think about connecting across time and across space, we are now quite physically constrained. We are dependent on the time and place beyond which we can connect. In other words, within the limited amount of time we have to live, our experiences are finite. I believe that communication technology will expand such experiences infinitely; in other words, communication will make it possible for us to encounter worlds that we have not yet seen or that we cannot easily encounter in reality, thereby creating an era in which the very experience itself will be transformed.

It is a very rich and free experience that will lead to a sustainable society. I have a sense of hope that this new future can be realized. I believe that this new world where the real and digital worlds are fused together is difficult to achieve alone. In a competitive relationship, people from various fields will connect with each other and work together in creative work to design a new society. We believe that a new future is expanding as a result.

The key word is our imagination. The human imagination is limitless, and when it connects with others, it creates endless possibilities.