Title
Delivering object-based immersive media experiences in sports
Abstract
Immersive media technology in sports enables fans to experience interactive, personalized content. A fan can experience the action in six degrees of freedom (6DoF), through the eyes of a player or from any desired perspective. Intel Sports makes deploying these immersive media experiences a reality by transforming the captured content from the cameras installed in the stadium into preferred volumetric formats used for compressing and streaming the video feed, as well as for decoding and rendering desired viewports to fans’ many devices. Object-based immersive coding enables innovative use cases where the streaming bandwidth can be better allocated to the objects of interest. The Moving Picture Experts Group (MPEG) is developing immersive codecs for streaming immersive video and point clouds. In this paper, we explain how to implement object-based coding in MPEG metadata for immersive video (MIV) and video-based point-cloud coding (V-PCC) along with the enabled experiences.
Keywords
Immersive media, MPEG-I, point-cloud, six degrees of freedom (6DoF), volumetric video
Authors
Fai Yeung
Intel Corporation, USA
Dr. Fai Yeung is
responsible for the architectural design of managed cloud services for virtual
reality sports broadcasting in Intel Sports. His technical interests include
immersive media, computer vision, and artificial intelligence. He holds a B.S.
and PHD in Electrical Engineering from University of Rochester, and MBA from UC
Berkeley. Angus owns 30+ pending and granted patents.
Basel Salahieh
Intel Corporation, USA
Basel Salahieh is immersive media algorithims and standards architect at Intel Corporation joined in 2015, holding a Ph.D. degree in electrical and computer engineering and M.Sc. in optical science from University of Arizona-USA, M.Sc. in electrical engineering from University of Oklahoma-USA, and B.Sc. in communication engineering from Aleppo University-Syria. His research interests are related to light fields, 6DoF, point clouds, VR/AR & immersive video systems (including capturing, coding and transmitting, and rendering). Basel is an active participant in the development of the MPEG Immersive Video standard. He has more than 20 published papers, 15 patents and patent applications, and is a senior member of OSA, and member of IEEE, SPIE, and SySSR.
Kimberly Loza
Intel Corporation, USA
Kimberly Loza is a
Software Engineer who works on creating immersive media experiences for the
Immersive Media Processing & Experiences Team in Intel Sports. She is an
active community board member for Latino Tech groups that work to encourage and
support underrepresented minorities in pursuing opportunities in tech. Her
technical interests include immersive media, artificial intelligence, and
contributing to open source initiatives. She holds a B.S. in Computer
Engineering from San Francisco State University and has 5+ pending patents on
methods for implementing point-cloud content.
Sankar Jayaram
Intel Corporation, USA
Dr. S. (Jay)
Jayaram has driven the digitization and personalization of sports for fans
through immersive technologies. Through his influential work in virtual reality
over the past 25 years, he has brought VR to a wide spectrum of domains - from
live events in sports and concerts to VR for engineering applications and training.
He has brought to market highly innovative solutions and products combining
virtual reality, immersive environments, powerful user controllable media
experiences, and social networks. The work done by his team in Virtual Assembly
and Virtual Prototyping in the 90s continues to be widely referenced by groups
around the world. Dr. Jay has co-founded several companies including VOKE
(acquired by Intel in 2016), Integrated Engineering Solutions, and Translation
Technologies. He was also a Professor at Washington State University and
co-founded the WSU Virtual Reality Laboratory in 1994. Most recently he was
Chief Technology Officer and Chief Product Officer at Intel Sports. He is
currently the CEO of a startup, QuintAR, Inc.
Jill Boyce
Intel Corporation, USA
Jill M. Boyce is an
Intel Fellow and Chief Media Architect at Intel, responsible for defining media
architectures for Intel’s video hardware designs. She represents Intel at the
Joint Collaborative Team on Video Coding (JCT-VC) and Joint Video Exploration
Team (JVET) of ITU-T SG16 and ISO/IEC MPEG, and in the Alliance for Open Media.
She serves as Associate Rapporteur of ITU-T VCEG (Study Group 16 Question 6),
is an editor of the MPEG-I Immersive Video draft standard, and was an editor of
the Scalability High Efficiency Video Coding extension (SHVC). She was named an
IEEE Fellow in 2019 “for contributions to video coding.”
