Title
JPEG Pleno: Standardizing a coding framework and tools for plenoptic imaging modalities
Abstract
JPEG Pleno is an upcoming standard from the ISO/IEC JTC 1/SC 29/WG 1 (JPEG) Committee. It aims to provide a standard framework for coding new imaging modalities derived from representations inspired by the plenoptic function. The image modalities addressed by the current standardization activities are light field, holography, and point clouds, where these image modalities describe different sampled representations of the plenoptic function. The applications that may benefit from these emerging image modalities range from supporting varying capture platforms, interactive content viewing, cultural environments exploration and medical imaging to more immersive browsing with novel special effects and more realistic images. These use cases come with a set of requirements addressed by the JPEG Pleno standard. Main requirements envision high compression efficiency, random access, scalability, error-resilience, low complexity, and metadata support. This paper presents a synopsis of the status of the standardization process and provides technical insights as well as the latest performance evaluation results.
Keywords
Holography, JPEG Pleno, light field, plenoptic, point cloud, standardization
Authors
Pekka Astola
Tampere University, Finland
Pekka Astola received the M.Sc. degree in signal processing and communications engineering from the Tampere University of Technology, Tampere, Finland, in 2013. He is currently pursuing the Ph.D. degree with the Computing Sciences Unit, Tampere University, under the supervision of Prof. I. Tabus. Since 2017, he has been actively participating in the working group ISO/IEC JTC1/SC29/WG1 developing the JPEG Pleno light field compression standard. He is the author of the JPEG Pleno Light Field Verification Model Software versions 1.0 and 1.1. His research interest includes light field image compression and processing. He was a corecipient of the 2016 3DTV Best Paper Award, the 2016 ISMVL Outstanding Paper Award, and the ICIP 2017 Light Field Image Coding Challenge Award.
Luis A. da Silva Cruz
Universidade de Coimbra & Instituto de Telecomunicações, Portugal
Luis A. da Silva Cruz (M'12) received the Licenciado and M.Sc. degrees in Electrical Engineering from the University of Coimbra, Portugal, in 1989 and 1993 respectively. He also holds an MSc degree in Mathematics and a Ph.D. degree in Electrical Computer and Systems Engineering from Rensselaer Polytechnic Institute (RPI), Troy, NY, US granted in 1997 and 2000 respectively. He has been with the Department of Electrical and Computer Engineering of the University of Coimbra in Portugal since 1990 first as a Teaching Assistant and as an Assistant Professor since 2000. He is a researcher of the Institute for Telecommunications of Coimbra where he works on image and video processing and coding, medical image processing and wireless communications. He is a member of the EURASIP, SPIE and IEEE technical societies.
Eduardo A. B. da Silva
Universidade Federal do Rio de Janeiro, Brazil
Eduardo A. B. da Silva was born in Rio de Janeiro, Brazil. He received the Electronics Engineering degree from Instituto Militar de Engenharia (IME), Brazil, in 1984, the M.Sc. degree in Electrical Engineering from Universidade Federal do Rio de Janeiro (COPPE/UFRJ) in 1990, and the Ph.D. degree in Electronics from the University of Essex, England, in 1995. He is a professor at Universidade Federal do Rio de Janeiro since 1989. He is co-author of the book ”Digital Signal Processing - System Analysis and Design”, published by Cambridge University Press. He published more than 70 papers in international journals. His research interests lie in the fields of signal and image processing, signal compression, 3D videos, computer vision, light fields and machine learning, together with its applications to telecommunications and the oil and gas industry. He is co-editor of the future standard ISO/IEC CD 21794-2, JPEG Pleno Plenoptic Image Coding System.
Touradj Ebrahimi
Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland
Touradj Ebrahimi is a professor at the Swiss Federal Institute of Technology in Lausanne (EPFL), working in teaching and research in multimedia signal processing. He is also Convener of the JPEG standardization committee, which has produced a family of standards that have revolutionized the world of imaging. He was also an adjunct professor at NTNU and has founded two companies: Emitall SA which is a high-tech company active in the field of digital media innovations, and Emitall Surveillance, which is active in the field of intelligent video surveillance systems. He represents Switzerland as the head of its delegation to SC29 (the body that oversees MPEG and JPEG standardization committees), and he acted as the chairman of the SC29 Advisory Group on Management for many years until 2014.
Pedro Garcia Freitas
Samsung, Brazil
Pedro Garcia Freitas is a researcher at Samsung R&D Institute Brazil. He received his Ph.D. and MSc degrees in Computer Science from the University of Brasília (UnB), Brazil, in 2017 and 2010, respectively. He received his BSc degree in physics, also from UnB, in 2010. Prior to his Ph.D., Pedro worked for more than 12 years as a software developer in multi-disciplinary projects in the areas of data science, smart grids, information sciences, image processing, and more. He has interests in computer vision, machine learning, and quality of experience.
Antonin Gilles
Institute of Research & Technology b-com, France
Antonin Gilles received his MS in electronics and computer engineering and his PhD in signal and image processing from the National Institute of Applied Sciences, Rennes, France, in 2013 and 2016, respectively. He is now working as a researcher on Computer-Generated Holography at the Institute of Research and Technology b<>com, Rennes, France. His main research interests include realistic full-color hologram synthesis, real-time hologram calculation and holographic data compression. He is involved in the ISO/IEC JTC1/SC29/WG1 (JPEG) standardization committee.
Kwan-Jung Oh
Electronics and Telecommunications Research Institute (ETRI), Korea
Kwan-Jung Oh received his M.S. and Ph.D. degrees in Information and Communications Engineering from the Gwangju Institute of Science and Technology (GIST), Gwangju, Rep. of Korea, in 2005 and 2010, respectively. From 2010 to 2013, he was with the Samsung Advanced Institute of Technology (SAIT), where he was involved in standardization activities of 3D video coding. He joined the Electronics and Telecommunications Research Institute (ETRI), Daejeon, Rep. of Korea in 2013, where he is currently a Senior Researcher with the Telecommunications & Media Research Laboratory. His research interests include 3-D image and video coding, immersive media, and holography.
Carla Pagliari
Instituto Militar de Engenharia, Brazil
Carla L. Pagliari (Senior Member, IEEE) received the Ph.D. degree in electronic systems Engineering from the University of Essex, Colchester, U.K., in 2000. She was with TV Globo, Rio de Janeiro, Brazil, from 1983 to 1985. From 1986 to 1992, she was a Researcher with the Brazilian Army Research and Development Institute, Rio de Janeiro. Since 1993, she has been with the Department of Electrical Engineering, Military Institute of Engineering, Rio de Janeiro. She has experience in television engineering and image processing, acting on the following topics: image/video coding, stereo/multiview video coding, video streaming, digital TV, and light field coding. She participated in the definition of the Brazilian Digital Television System, and is a member of the Educational Board of the Brazilian Society of Television Engineering. She is an expert member of the International Standard Organization ISO/IEC SC29 and Working Group WG1 (JPEG).
Fernando Pereira
Instituto Superior Técnico, Universidade de Lisboa & Instituto de
Telecomunicações, Portugal
Fernando Pereira received the B.S., M.Sc., and Ph.D. degrees in electrical and computer engineering from the Instituto Superior Técnico (IST), Universidade de Lisboa, Lisboa, Portugal, in 1985, 1988, and 1991, respectively. He is currently with the Electrical and Computer Engineering Department, IST, and Instituto de Telecomunicações. He has authored or coauthored more than 300 papers. His areas of research interest include visual information analysis, processing, coding and description, and interactive multimedia services. He has been a member of the Scientific and Program Committees of many international conferences. He has been participating in the work of ISO/IEC MPEG and ISO/IEC JPEG for many years, notably as the Head of the Portuguese delegation, and the Chair of the MPEG and JPEG Requirements Subgroups.
Cristian Perra
University of Cagliari, Italy
Cristian Perra is an assistant professor of Communications at the Department of Electrical and Electronic Engineering, University of Cagliari, Italy. He is a research associate at the Multimedia Communications Laboratory of CNIT (Consorzio Nazionale Interuniversitario per le Telecomunicazioni) in Cagliari, Italy. In 2003, he received his PhD in electronic engineering and computer science from the University of Cagliari. He is a member of the ISO/IEC JTC1/SC29/WG1 (JPEG) standardization committee. His main research interests include multimedia communications, image/video/point cloud/light field/holography signal processing, virtual reality and augmented reality applications, and quality of experience for multimedia communications.
Stuart Perry
University of Technology Sydney, Australia
Stuart Perry gained his PhD in Electrical Engineering in 1999 from the University of Sydney, Australia. He previously worked for the Defence Science and Technology Organisation and Canon Information Systems Research Australia (CiSRA). He is currently an Associate Professor at the School of Electrical and Data Engineering, Faculty of Engineering and IT, University of Technology Sydney and co-director of the Perceptual Imaging Laboratory (PILab) conducting research into colour and perceptual quality in 3D environments. He is an Australian Representative for ISO standards for still image compression as well as chair of the JPEG Pleno Ad Hoc Group on Point Cloud Compression. He has more than 30 international publications including two books and 20 patents.
Antonio M. G. Pinheiro
Universidade da Beira Interior & Instituto de Telecomunicacoes, Portugal
Antonio M. G. Pinheiro is a professor at U.B.I. (Universidade da Beira Interior), Covilhã, Portugal and a researcher at I.T. (Instituto de Telecomunicações), Portugal. He received his B.S. (Licenciatura) from I.S.T., Lisbon in 1988 and his Ph.D. in electronic systems engineering from the University of Essex in 2002. Currently, his research interests are on image processing, namely on for multimedia quality evaluation, emerging 3D image technologies and medical image analysis. He is a Portuguese delegate to ISO/IEC JTC 1/SC 29, where he is currently involved with JPEG standardisation committee, currently the JPEG Communication Subgroup chair and involved with the JPEG Pleno standard development. He is a senior member of IEEE.
Peter Schelkens
Vrije Universiteit Brussel & Imec, Belgium
Peter Schelkens holds a professorship with the Department of Electronics and Informatics, Vrije Universiteit Brussel, Belgium and is a research group leader at imec, Belgium. In 2013, he received an EU ERC Consolidator Grant focusing on digital holography. His research interests are situated in the field of multidimensional signal processing encompassing the representation, communication, security, rendering and quality assessment of these signals while especially focussing on multidisciplinary research. He is the co-editor of the books The JPEG 2000 Suite (Wiley, 2009) and Optical and Digital Image Processing (Wiley, 2011). He chairs the Coding, Test and Quality subgroup of the ISO/IEC JTC1/SC29/WG1 (JPEG) standardization committee and is associate editor of Signal Processing: Image Communication and AOP JPhys Photonics.
Ismael Seidel
Samsung, Brazil
Ismael Seidel received his Ph.D., M.Sc., and B.Sc. in Computer Science from the Federal University of Santa Catarina (UFSC), Brazil, in 2019, 2014, and 2011, respectively. He is currently a research scientist at the Samsung R&D Institute Brazil (SRBR), where he develops research on video/image processing. Dr. Seidel is a member of the Brazilian delegation for the ISO/IEC JTC1/SC29/WG1 (JPEG) and ISO/IEC JTC1/SC29/WG11 (MPEG) activities. He is also a member of the Embedded Computing Laboratory (ECL) from UFSC. His main areas of interest are visual signal coding and related low-power hardware architectures.
Ioan Tabus
Tampere University, Finland
Ioan Tabus received the Ph.D. degree (with honors) from Tampere University of Technology, Finland, in 1995. Since January 2000, he has been a Professor in the Department of Signal Processing at Tampere University of Technology. His research interests are in light field image processing, plenoptic image compression, points clouds compression, and statistical signal processing. He is coauthor of more than 250 publications in the fields of signal compression, image processing, bioinformatics, and system identification. He served as an Associate Editor of the IEEE Transactions on Signal Processing and as a guest editor of special issues for IEEE Signal Processing Magazine and for IEEE Journal of Selected Topics in Signal Processing and was the Editor-in-Chief of the EURASIP Journal on Bioinformatics and Systems Biology. Dr. Tabus is the co-recipient of 1991 "Train Vuia" Award of Romanian Academy, the 2016 3DTv Best Paper Award, and the ICIP 2017 Light Field Image Coding Challenge Award.
