Special issue The future of video and immersive media
Foreword
     Houlin Zhao Secretary-General International Telecommunication Union
Foreword
     Chaesub Lee Director ITU Telecommunication Standardization Bureau
Editor-in-Chief’s message
     Jian Song
Guest Editors’ message
EDITORIAL BOARD
Table of Contents
List of Abstracts
SELECTED PAPERS
     DELIVERING OBJECT-BASED IMMERSIVE MEDIA EXPERIENCES IN SPORTS
          1. INTRODUCTION
          2. Immersive media for sports
               2.1 Intel Sports immersive media platform
                    2.1.1 Intel® True View for volumetric video
                    2.1.2 Intel® True VR for panaromic video
                    2.1.3 Immersive media processing and experiences pipeline
          3. MPEG IMMERSIVE MEDIA STANDARDS
               3.1 Video-based point-cloud coding (V-PCC)
               3.2 MPEG immersive video (MIV)
          4. Delivering Object-based IMMERSIVE MEDIA experiEnce
               4.1 Objects indexing input
               4.2 Implementation in immersive standards
               4.3 Object-based immersive media platform
          5. Applications and Services
               5.1 Priority objects rendering
               5.2 Objects filtering
               5.3 Background rendering
               5.4 Object-based scalability
               5.5 Personalized 6DoF user experience
               5.6 Object of interest
          6. Conclusion
          REFERENCES
     IMMERSIVE MEDIA AND ACCESSIBILITY: HAND IN HAND TO THE FUTURE
          1. INTRODUCTION
          2. SOCIETAL IMPACT
          3. STATE-OF-THE-ART
               3.1 Related projects in the broadcast sector
               3.2 Research on accessibility in immersive media
          4. IMAc: end-to-end platform for immersive accessibility
               4.1 Content production / authoring
               4.2 Service provider
               4.3 Content preparation / distribution
               4.4 Content consumption
          5. CONCLUSION
          ACKNOWLEDGEMENT
          REFERENCES
     MULTI-VIEWPOINT AND OVERLAYS IN THE MPEG OMAF STANDARD
          1. INTRODUCTION
          2. MPEG OMAF System Architecture
          3. Overlays in the MPEG OMAF Standard
               3.1 Overlay features in OMAFv2
                    3.1.1 Spatial property
                    3.1.2 Temporal property
                    3.1.3 Interactivity property
                    3.1.4 Inherent property
          4. Multi-Viewpoint in the MPEG OMAF Standard
               4.1 Spatial relationship between multiple viewpoints
               4.2 Viewpoint switching
                    4.2.1 Content creator specified switching
                    4.2.2 User-preference-driven switching
                    4.2.3  Fully automatic viewpoint switching
          5. CONCLUSION
          ACKNOWLEDGEMENT
          REFERENCES
     DATA-DRIVEN INTRA-PREDICTION MODES IN THE DEVELOPMENT OF THE VERSATILE VIDEO
CODING STANDARD
          1. INTRODUCTION
          2. DATA-DRIVEN DESIGN OF INTRAPREDICTIONMODES
          3. NEURAL-NETWORK-BASED INTRA PREDICTORS
          4. PREDICTION INTO THE TRANSFORM DOMAIN
          5. MATRIX-BASED INTRA-PREDICTIONMODES
          6. CONCLUSION
          REFERENCES
     ARENA-STYLE IMMERSIVE LIVE EXPERIENCE (ILE) SERVICES AND SYSTEMS:
HIGHLY REALISTIC SENSATIONS FOR EVERYONE IN THE WORLD
          1. INTRODUCTION
          2. STANDARDIZATION ACTIVITY of ILE and related workS
               2.1 ITU-T H.430 standard series
               2.2 Related works
          3. Technology Suite for ILE
               3.1 Kirari!
               3.2 Immersive events by Kirari!
          4. Kirari! for Arena
               4.1 Overview
               4.2 System configuration
               4.3 Capture & Extraction function
               4.4 Measurement & Tracking function
               4.5 Information Integration & Transport function
               4.6 Depth Expression & Presentation function
          5. Evaluation and Discussion
               5.1 Conformance to ITU-T H.430 series (ILE) of standards
               5.2 Discussion on further study
          6. CONCLUSION
          REFERENCES
     BEYOND THE FRONTIERS OF EDUCATION:
HOW IMMERSIVE MEDIA CHANGES THE WAY WE LEARN
          1. INTRODUCTION
          2. SPACEBUZZ
          3. SERIOUS GAMES
          4. VIRTUAL REALITY
          5. INTELLIGENT TUTORING SYSTEMS
          6. LEARNING ANALYTICS
          7. CONCLUSION
          ACKNOWLEDGEMENT
          REFERENCES
     TRENDS AND ADVANCEMENTS IN
DEEP NEURAL NETWORK COMMUNICATION
          1. Introduction
          2. Survey on Neural Network Communication
               2.1 On-device Inference
               2.2 Federated Learning
               2.3 Peer-to-Peer Learning
               2.4 Distributed Training in the Data Center
          3 Related Challenges in Embedded ML
          4. Conclusion
     A STUDY OF THE EXTENDED PERCEPTUALLY WEIGHTED PEAK SIGNAL-TO-NOISE RATIO
(XPSNR) FOR VIDEO COMPRESSION WITH DIFFERENT RESOLUTIONS AND BIT DEPTHS
          Abstract
          1. INTRODUCTION
               1.1 Prior work by the authors
               1.2 Outline of this paper
          2. REVIEW OF BLOCK-BASED WPSNR
          3. EXTENSION FOR MOVING PICTURES
          4. TEMPORALLY VARYING VIDEO QUALITY
          5. VARYING INPUT OR OUTPUT BIT DEPTH
          6. HIGH AND LOW-RESOLUTION VIDEOS
          7. EXPERIMENTAL EVALUATION
          7.1 Correlation with subjective MOS data
          7.2 Comparison of computational complexity
          8. DISCUSSION AND CONCLUSION
          9. ACKNOWLEDGMENT
          REFERENCES
     THE VIDEO CODEC LANDSCAPE IN 2020
          1. INTRODUCTION
          2. Overview of the video coding designs
               2.1 Partitioning
               2.2 Intra coding
               2.3 Inter coding
               2.4 Transforms
               2.5 In-loop filters
               2.6 Entropy coding
               2.7 Screen Content Coding
               2.8 Other tools
          3. Test conditions
          4. Video Coding configurations
               4.1 HEVC, VVC, and EVC
               4.2 AV1
                    4.2.1 Two-pass encoding
                    4.2.2 Quantization control
                    4.2.3 Encoding parameters
          5. Results
               5.1 Objective quality
                    5.1.1 Broadcast
                    5.1.2 Streaming
               5.2 Subjective quality
               5.3 Processing time
          6. Conclusion
          REFERENCES
     JPEG PLENO: STANDARDIZING A CODING FRAMEWORK AND TOOLS
FOR PLENOPTIC IMAGING MODALITIES
          1. Introduction
          2. Use cases
          3. Requirements
               3.1 Generic requirements
               3.2 Light field specific requirements
               3.3 Point cloud specific requirements
               3.4 Holography specific requirements
          4. JPEG Pleno
               4.1 JPEG Pleno Part 1: Framework
               4.2 JPEG Pleno Part 2: Light field coding
                    4.2.1 4D-Transform Mode (4DTM)
                    4.2.2 4D-Prediction Mode (4DPM)
                    4.2.3 Part 2 Performance Evaluation
               4.3 JPEG Pleno Part 3: Conformance testing
               4.4 JPEG Pleno Part 4: Reference software
               4.5 JPEG Pleno point cloud
                    4.5.1 Current status
                    4.5.2 Objective quality assessment
                    4.5.3 Subjective testing protocols
               4.6 JPEG Pleno Holography
                    4.6.1 Current status
                    4.6.2 Numerical reconstruction software
                    4.6.3 Objective quality assessment
                    4.6.4 Subjective testing protocols
          5. Conclusions and future plans
     A NEW SET OF DIRECTIONAL WEIGHTS FOR ITU-R BS.1770 LOUDNESS
MEASUREMENT OF MULTICHANNEL AUDIO
          1. INTRODUCTION
               1.1 Directional weighting estimation in ITU-RBS.1770-4
          2. LISTENING TEST
               2.1 Design
               2.2 Methodology
               2.3 Results
          3. GAIN ESTIMATION
               3.1 Optimization problem
               3.2 Regression problem
          4. TESTS WITH MULTICHANNEL AUDIOCONTENT
          5. CONCLUSION
          ACKNOWLEDGMENT
          REFERENCES
     OVERVIEW OF MPEG-5 PART 2 –
LOW COMPLEXITY ENHANCEMENT VIDEO CODING (LCEVC)
          1. Introduction
          2. Commercial Reasons for the Standard
          3. Key Technical Features
               3.1 Sparse residual data processing
               3.2 Efficient use of existing codecs
               3.3 Resilient and adaptive coding process
               3.4 Agnostic base enhancement
               3.5 Parallelization
          4. Bitstream Structure
          5. Coding Structure
               5.1 Encoder
                    5.1.1 Base codec
                    5.1.2 Enhancement sub-layer 1
                    5.1.3 Enhancement sub-layer 2
               5.2 Decoder
                    5.2.1 Base codec
                    5.2.2 Enhancement sub-layer 1
                    5.2.3 Enhancement sub-layer 2
          6. Coding Tools
               6.1 Down- and upscaler
               6.2 Transform
               6.3 Quantization
               6.4 L-1 filter
               6.5 Temporal prediction
               6.6 Entropy encoding
          7. Performance Results
               7.1 Experimental set-up
                    7.1.1 Reference implementation
                    7.1.2 Commercial implementation
               7.2 Objective and subjective metrics results
               7.3 Processing time performances
          8. Conclusion
          References
     HISTORY REVIEW PAPER
     SOME LITTLE-KNOWN ASPECTS OF THE HISTORY OF THE JPEG STILL PICTURE-CODINGSTANDARD, ITU-T T.81 | ISO/IEC 10918-1 (1986-1993)
          Introduction
          1. The JPEG-1 project (ITU-T T.81 | ISO/IEC 10918-1) of CCITT SGVIII and ISO/IEC JTC1 SC29 and its working rules and procedures
          2. Requirements of ITU-T T.81 (1992) |ISO/IEC 10918-1:1993 [1] and its toolbox nature
               2.1 JPEG: An architecture for image compression
               2.2 JPEG baseline and extended systems
               2.3 JPEG patent policy
          3. JPEG and the Independent JPEG Group contacts
               3.1 Conclusions and take aways on JPEG and the IJG OSS process
          4. Planned and not directly planned (but successful) JPEG applications
               4.1  ITU-T JPEG applications
                    4.1.1  Colour facsimile group 3: Annex E of ITU-T T.30 [14]; Annex E of ITU-T T.4 [15]
                    4.1.2 Colour group 4 facsimile: ITU-T T.563 [16]; ITU-T T.42 [17]
                    4.1.3  Videotex: Annex F of ITU-T T.101 [18]
                    4.1.4  Multimedia conferencing: ITU-T T.120 [13]; ITU-T T.126 [20]
                    4.1.5  Open document architecture (ODA) raster graphics content architecture: ITU-T T.417 | ISO/IEC 8613-7 [21]
                    4.1.6  ISDN Videophone still image transmission: ITU-T H.261 [10]
               4.2  JPEG file interchange format
               4.3  M-JPEG
               4.4  Exchangeable image file format
               4.5  Web HTML
               4.6  Standards in medical image communications
               4.7  JPEG as a military standard
          5. Conclusions
          Acknowledgements
          References
          INDEX OF AUTHORS