​Immersive live experience systems and services

(Continuation of Question 8/16)​

Motivation
Recently, some of huge sport events and music concerts are not only broadcasted, but also delivered to remote sites for public viewing or live viewing in order to share emotion by audiences in remote sites as if they were in main event venues. In order to provide high-realistic sensations to audiences at remote sites, immersive live experience (ILE) needs to be implemented to reconstruct event sites virtually with presentation of real-sized objects and sound direction by transmitting environmental information together with audio and video streams.
Implementing ILE needs several technologies such as real-time objects extraction technologies at event sites, spatial location sensing technologies for objects, sound direction identify technologies, media transport technologies for extracted objects including spatial location information, presentation technologies including 3D projection at remote sites, synchronous technologies with video, sound and lighting, and so on. Although some of them are already established, there are some conditions and/or limitations such as specific content and pre-arrangement of remote sites. Pre-arrangement of remote sites includes 3D projection mapping, and takes much time for adjustment terminal devices. In addition, these technologies have not systemized, and most of them are not standardized yet.

In order to share enthusiasm at event venues with large audiences even if they are in remote sites far from event venue, implementing immersive live experience services based on standardized designs is desired. By standardizing ILE in ITU‑T, it is expected that audiences anywhere in the world can cheer their favourite sport teams or artists at remote sites even if they are not in the event venue, and they can feel a sense of togetherness and get passionate as if they were in the event venue. Most of these technologies are related to the multimedia studies in Study Group 16, thus this Question will progress standardization activities of ILE.

Globally interoperable standards will activate a market for the ILE systems and services. This Question will cover all relevant work items on multimedia aspects of immersive live experience systems and services.

Study items
Study items to be considered include, but are not limited to:

• domain of immersive live experience services;
• use cases and requirements for immersive live experience systems and services;
• architectural aspects of immersive live experience systems to support requirements and various use cases;
• presentation equipment profiles for supporting various kinds of immersive live experience applications;
• provision of content including spatial information from content source to presentation equipment for immersive live experience;
  
• multimedia application frameworks for immersive live experience, including five sensory (vibration, smell, humid, temperature and so on) information transmission;
  
• usage of cloud computing technologies for efficient deployment and operation, and for effective service offering;
• presentation aspects of immersive live experience services such as combination of multiple displays, multiple speakers and lighting equipment;
• specifications on metadata and media format for immersive live experience content for fitting with the use cases;
• management and operational aspects of immersive live experience systems;
  
• definition and evaluation/measurement methods of quality of ILE (immersiveness, live experiences, and others);
  
• consideration on providing emergency information including warning messages in the disaster environment;
• consideration on providing accessibility for disabled people, elder people, and foreign visitors;
• review and analysis of existing Recommendations and relevant specifications to find any reusable materials for immersive live experience systems and services;
• considerations on how to help measure and mitigate climate change.

Tasks
Tasks include, but are not limited to:

• identification of the use cases and requirements;
• definition of functional architectures and its components to support use cases and requirements for immersive live experience systems and services;
• definition of immersive live experience presentation equipment profiles based on capabilities;
• definition of a mechanisms and protocols to provide content delivery function;
• definition of interface specifications amongst functional components of immersive live experience systems;
• definition of procedures and methods to interact between immersive live experience systems and audiences' devices such as smart phone and tablet PC;
• definition of multimedia application frameworks, metadata and media formats for providing immersive live experience services;
• definition of control functions for synchronous/asynchronous presentation of multiple displays and other presentation equipment;
  
• definition of quality of ILE (immersiveness, live experiences, and others);​
  
• modification and/or extension of existing Recommendations under the ITU‑T Study Group 16 responsibility to provide immersive live experience services;
  
• maintain deliverables under the responsibility of the Question, including: ITU-T H.430.x series;
  
• collaboration and harmonization with other standardization bodies, forums and consortia to develop Recommendations to support immersive live experience service.

An up-to-date status of work under this Question is found in the SG16 work programme
(https://www.itu.int/ITU-T/workprog/wp_search.aspx?sp=16&q=8/16).

Relationships

Recommendations

• ITU‑T Study Group 16 Recommendations, in particular telepresence system Recommendations ITU‑T F.734, ITU‑T H.420
  

Questions

• All Questions of Study Group 16

Study groups

• ITU‑T SGs 9, 11, 12, 13 and 17
• ITU‑R SG6
  
• ITU-D SG2
  

Other bodies

• ISO, IEC, ISO/IEC JTC1
• ETSI SIG MEC (Mobile Edge Computing)
• W3C, IETF (e.g. CLUE), IEEE
  
• 3GPP SA4​​