High fidelity imaging [Elektronische Ressource] : the computational models of the human visual system in high dynamic range video compression, visible difference prediction and image processing / vorgelegt von Rafał Mantiuk

Rafal Mantiuk

Informations

• Publié par: universitat_des_saarlandes
• Publié le: 01 janvier 2007
• Nombre de lectures: 7
• Langue: English
• Poids de l'ouvrage: 6 Mo

Extrait

?HIGH-FIDELITY IMAGING?
THE COMPUTATIONAL MODELS OF
THE HUMAN VISUAL SYSTEM IN
HIGH DYNAMIC RANGE VIDEO COMPRESSION,
VISIBLE DIFFERENCE PREDICTION AND
IMAGE PROCESSING
DISSERTATION
ZUR ERLANGUNG DES GRADES DES
DOKTORS DER INGENIEURWISSENSCHAFTEN (DR.-ING.)
¤DER NATURWISSENSCHAFTLICH-TECHNISCHEN FAKULTATEN
¤DER UNIVERSITAT DES SAARLANDES
VORGELEGT VON
RAFA MANTIUK
¤EINGEREICHT AM 10. JULI 2006 IN SAARBRUCKENDatum des Kolloqiums: 14.12.2006
Betreuender Hochschullehrer ? Supervisor:
Dr.-Ing. habil. Karol Myszkowski, MPI fur¤ Informatik, Saarbruk¤ en, Germany
Gutachter ? Reviewers:
¤ ¤Dr.-Ing. habil. Karol Myszkowski, MPI fur Informatik, Saarbruken, Germany
Prof. Dr. Hans-Peter Seidel, MPI fur¤ Informatik, Saarbruk¤ en, Germany
Prof. Dr. Sumanta N. Pattanaik, University of Central Florida, USA
Dekan ? Dean:
Prof. Dr. Thorsten Herfet, Universitat¤ des Saarlandes, Saarbruk¤ en, Germany3
Abstract
As new displays and cameras offer enhanced color capabilities, there is a need to extend
the precision of digital content. High Dynamic Range (HDR) imaging encodes images
and video with higher than normal bit-depth precision, enabling representation of the
complete color gamut and the full visible range of luminance.
This thesis addresses three problems of HDR imaging: the measurement of visible dis-
tortions in HDR images, lossy compression for HDR video, and artifact-free image
processing. To measure distortions in HDR images, we develop a visual difference pre-
dictor for HDR images that is based on a computational model of the human visual
system. To address the problem of HDR image encoding and compression, we derive
a perceptually motivated color space for HDR pixels that can ef ciently encode all
perceivable colors and distinguishable shades of brightness. We use the derived color
space to extend the MPEG-4 video compression standard for encoding HDR movie
sequences. We also propose a backward-compatible HDR MPEG compression algo-
rithm that encodes both a low-dynamic range and an HDR video sequence into a single
MPEG stream. Finally, we propose a framework for image processing in the contrast
domain. The framework transforms an image into multi-resolution physical
images (maps), which are then rescaled in just-noticeable-difference (JND) units. The
application of the framework is demonstrated with a contrast-enhancing tone mapping
and a color to gray conversion that preserves color saliency.
Kurzfassung
Aktuelle Innovationen in der Farbverarbeitung bei Bildschirmen und Kameras erzwin-
gen eine Prazisionserweiterung¤ bei digitalen Medien. High Dynamic Range (HDR) ko-
dieren Bilder und Video mit einer grosseren¤ Bittiefe pro Pixel, und ermoglichen¤ damit
die Darstellung des kompletten Farbraums und aller sichtbaren Helligkeitswerte.
Diese Arbeit konzentriert sich auf drei Probleme in der HDR-Verarbeitung: Messung
von fur¤ den Menschen storenden¤ Fehlern in HDR-Bildern, verlustbehaftete Kompres-
sion von HDR-Video, und visuell verlustfreie HDR-Bildverarbeitung. Die Messung
von HDR-Bildfehlern geschieht mittels einer Vorhersage von sichtbaren Unterschieden
zweier HDR-Bilder. Die Vorhersage basiert dabei auf einer Modellierung der menschli-
chen Sehens. Wir addressieren die Kompression und Kodierung von HDR-Bildern mit
der Ableitung eines perzeptuellen Farbraums fur¤ HDR-Pixel, der alle wahrnehmba-
ren Farben und deren unterscheidbaren Helligkeitsnuancen ef zient abbildet. Danach
verwenden wir diesen Farbraum fur¤ die Erweiterung des MPEG-4 Videokompressi-
onsstandards, welcher sich hinfort auch fur¤ die Kodierung von HDR-Videosequenzen
eignet. Wir unterbreiten weiters eine ruckw¤ arts-k¤ ompatible MPEG-Kompression von
¤HDR-Material, welche die ubliche YUV-Bildsequenz zusammen mit dessen HDR-
¤Version in einen gemeinsamen MPEG-Strom bettet. Abschliessend erklaren wir un-
ser Framework zur Bildverarbeitung in der Kontrastdomane.¤ Das Framework trans-
formiert Bilder in mehrere physikalische Kontrastau osungen,¤ um sie danach in Ein-
heiten von just-noticeable-difference (JND, noch erkennbarem Unterschied) zu res-
kalieren. Wir demonstrieren den Nutzen dieses Frameworks anhand von einem kon-
trastverstark¤ enden Tone Mapping-Verfahren und einer Graukonvertierung, die die ur-
sprunglichen¤ Farbkontraste bestmoglich¤ beibehalt.¤4
Summary
As new displays and cameras offer enhanced color capabilities, there is a need to extend
the precision of digital content, speci cally images and video. High Dynamic Range
Imaging (HDRI) encodes images and video with higher bit-depth precision, enabling
representation of the complete color gamut and the full visible range of luminance,
which makes this technology a successor to traditional 8-bit-per-color-channel imag-
ing. However, to realize transition from the to HDR imaging, it is necessay
to develop imaging algorithms that work with the high-precision data. To make such
algorithms effective and usable in practice, it is necessary to take advantage of the limi-
tations of the human visual system by reducing the storage and processing precision so
that it matches the performance of the human eye. Therefore, human visual perception
is the key component in the solutions we present in this dissertation. We address three
important problems in this dissertation: the measurement of visible distortions in HDR
images, lossy compression for HDR video, and an HDR image processing framework,
suitable for contrast compression.
To facilitate assessment of the visual quality of HDR content, we develop a visual
difference predictor for HDR images. Given two images, the predictor can detect dif-
ferences that would be noticeable to the human observer. The metric is based on a
computational model of the human visual system, which we extend and adapt for HDR
content. We included several aspects that are important in the perception of high con-
trast images, such as distortions of the eye’s optics, photoreceptor response under a
broad range of luminance adaptation conditions, and contrast sensitivity in the pres-
ence of the local adaptation. The metric is calibrated for natural images in a subjective
experiment.
The key component of an imaging pipeline is standardized and effective image and
video encoding. To address the problem of HDR image encoding and compression, we
derive a color space for HDR pixels from perceptual measurements. The color space
can ef ciently encode all perceivable colors and distinguishable shades of brightness
that are visible under all illumination conditions. The proposed color space, which
requires only twelve bits to encode luminance and two eight-bit channels to encode
chrominance, offers a straightforward extension of existing image and video compres-
sion standards.
We use the derived color space for HDR pixels to extend the MPEG-4 video compres-
sion standard for encoding HDR movie sequences. The extended encoder offers a spe-
cial treatment of sharp contrast edges, which can have higher contrast than traditional
video material. The proposed compression method proves to be an effective as well as
novel extension to the existing MPEG standard (ISO/IEC 14496-2 and 14496-10).
To facilitate a smooth transition from traditional to HDR content, we propose a back-
ward-compatible HDR MPEG compression algorithm. Within a single MPEG stream,
the algorithm encodes two video sequences, one low-dynamic range (LDR ? traditional
video) and the other HDR, into a single MPEG stream. Naive applications recognize
this stream as an ordinary MPEG video, however advanced software or hardware can
decode HDR video. The algorithm requires only 8-bit software or hardware MPEG
coders. The LDR and HDR video sequences are decorrelated to achieve the best com-
pression performance. To further improve compression, invisible noise is removed
from the HDR data stream using a multi-band perceptual lter. The lter estimates5
visibility thresholds, taking into account luminance masking, the contrast sensitivity
function, phase uncertainty and contrast masking.
The multi-resolution representations of images, such as wavelets, pyramids or band-
pass channels, offer an attractive tool for image processing and editing. However, these
representations often lead to unwanted artifacts and arti cial looking resulting images,
especially when each band or resolution is modi ed separately. To avoid such artifacts
while bene ting from the advantages of the multi-resolution representation, we propose
a contrast-domain image processing framework. The framework transforms an image
into several resolutions of physical contrast. The contrast is then rescaled using a spe-
cially derived transducer function in perceptually plausible just-noticeable-difference
(JND) units. The resulting image is constructed from the modi ed contrast by solv-
ing an optimization problem. All components of the framework are designed to work
with high contrast HDR images. We demonstrate the application of the framework
on a contrast-enhancing tone mapping and a color to gray conversion that preserves
color saliency. The framework is especially effective for operations that heavily distort
contrast, such as extreme sharpening of images.
The proposed solutions constitute the c