EVALUATION OF IMAGE QUALITY METRICS

Chief investigator: Martin Čadík, email: cadikm@centrum.cz, homepage: http://cadik.posvete.cz/

Project web page: http://cadik.posvete.cz/iqm/

Project description:

The goal of the project is the evaluation of state-of the art image quality metrics. We conduct both the subjective and objective comparisons of metrics.


FULL-REFERENCE IMAGE QUALITY METRICS

Full-reference image quality metrics


IMPLEMENTED IMAGE QUALITY METRICS

Original image Distorted image Map of differences Method
       

S. Daly

The Visible Differences Predictor: An Algorithm for the Assessment of Image Fidelity

       

Z. Wang, A. C. Bovik

A Universal Image Quality Index

http://www.cns.nyu.edu/~zwang/files/papers/uqi.html
Matlab code available
       

Z. Wang, A. C. Bovik, H. R. Seikh, E. P. Simoncelli

Image quality assessment: From error visibility to structural similarity

Matlab code available
       

A. Shnayderman, A. Gusev, A. M. Eskicioglu

An SVD-Based Gray-Scale Image Quality Measure for Local and Global Assessment

http://www.sci.brooklyn.cuny.edu/~eskicioglu/papers
Published also in [An SVD-Based Gray-Scale Image Quality Measure for Local and Global Assessment]
 DONE  DONE  DONE  

Tunc Ozan Aydin, Rafal Mantiuk, Karol Myszkowski, Hans-Peter Seidel

Dynamic Range Independent Image Quality Assessment


3 types of distortion: loss of visible contrast, amplification of invisible contrast, reversal of visible contrast, project page (online implementation)
 DONE  DONE  DONE  

Rafal Mantiuk, Kil Joong Kim, Allan G. Rempel, Wolfgang Heidrich

HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions


 DONE  DONE  DONE  

R. Mantiuk, S. Daly, K. Myszkowski, H.-P. Seidel

Predicting Visible Differences in High Dynamic Range Images - Model and its Calibration


Extends VDP for HDR, also published in [Visible Difference Predicator for High Dynamic Range Images]
http://www.mpi-inf.mpg.de/resources/hdr/vdp/
 DONE  DONE  DONE  

Kuang, J., Johnson, G.M., Fairchild M.D.

iCAM06: A refined image appearance model for HDR image rendering


Extends iCAM, MatLab code available


METHODS TO IMPLEMENT (i.e. possible student projects/master's thesis topics)

 TODO  

J. L. Mannos, D. J. Sakrison

The Effects of a visual fidelity criterion on the encoding of images


 TODO  

H. Rushmeier, G. Ward, C. Piatko, P. Sanders

Comparing Real and Synthetic Images: Some Ideas About Metrics


Model after [Gervais84]
 TODO  

Patrick C. Teo, David J. Heeger

Perceptual image distortion


 TODO  

J. Lubin

A visual discrimination model for imaging system design and evaluation


 TODO  

C.C. Taylor, Z. Pizlo, J.P. Allebach, C.A. Bouman

Image Quality Assessment with a Gabor Pyramid Model of the Human Visual System


 TODO  

M.R.M. Nijenhuis, F.J.J. Blommaert

Perceptual-error measure and its application to sampled and interpolated single-edged images


 TODO  

L. Neumann, K. Matkovic, W. Purgathofer

Perception based color image difference


 TODO  

A. P. Bradley

A Wavelet Visible Difference Predictor


 TODO  

J. Oh, S. I. Wooley, T. N. Arvanitis, J. N. Townend

A multistage perceptual quality assessment for compressed digital angiogram images


 TODO  

X. Tong, D. Heeger, C. J. van den B. Lambrecht

Video quality evaluation using ST-CIELAB




OUR PUBLICATIONS

Author Title Publishing details
Čadík, M., Herzog, R., Mantiuk, R., Mantiuk, R., Myszkowski, K., Seidel, H.P. Learning to Predict Localized Distortions in Rendered Images

[Paper (pdf)]
[Supplementary Material (html)]
[Presentation slides (pdf)]
Computer Graphics Forum Vol. 32, Num. 7 (proc. of Pacific Graphics'13), pp. 401-410, 2013.
Čadík, M., Herzog, R., Mantiuk, R., Myszkowski, K., Seidel, H.P. New Measurements Reveal Weaknesses of Image Quality Metrics in Evaluating Graphics Artifacts

[Paper (pdf)]
[Supplementary materials (pdf)]
ACM Transactions on Graphics (SIGGRAPH Asia 2012), vol. 31, no. 6, DOI: 10.1145/2366145.2366161, ACM, 2012.
Herzog, R., Čadík, M., Aydin, T. O., Kwang I. K., Myszkowski, K., Seidel, H.P. NoRM: No-Reference Image Quality Metric for Realistic Image Synthesis

[Paper (pdf)]
[Supplementary materials (pdf)]
[Dataset and User Study]
Computer Graphics Forum, vol. 31, issue 2 (2012), (Proceedings of Eurographics'12, Cagliari / Italy, 13–18 May), 2012.
Čadík, M., Aydin, T.O., Myszkowski, K., Seidel, H.P. On Evaluation of Video Quality Metrics

[Paper (pdf)]
[Presentation slides (pdf)]
[bibTeX entry (bib)]
[HDR Dataset (905MB zip file)]
[Browse HDR Dataset]
Proc. of IS&T/SPIE's Human Vision and Electronic Imaging, 2011.

Copyright 2011 Society of Photo-Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.
Aydin, T.O., Čadík, M., Myszkowski, K., Seidel, H.P. Video Quality Assessment for Computer Graphics Applications

[Video Quality Metric Online]
[Paper (pdf)]
[Video (mp4)]
[Validation experiment (pdf)]
[Formulae (pdf)]
[bibTeX entry (bib)]
ACM Transactions on Graphics (SIGGRAPH Asia 2010), vol. 29, no. 5, ISSN 0730-0301, ACM, 2010.
Čadík, M. Perceptually Based Image Quality Assessment and Image Transformations


[bibTeX entry (bib)]
Ph.D. Thesis, Czech Technical University in Prague, 2008.

Outstanding dissertation award, President of CTU in Prague, December 2009.
Čadík, M., Slavík, P. Evaluation of Two Principal Approaches to Objective Image Quality Assessment

[bibTeX entry (bib)]
8th International Conference on Information Visualization. Los Alamitos: IEEE Computer Society Press, 2004.
Čadík, M. Perceptual Image Quality Assessment Metrics CODATA Information Visualization Workshop 2004. Prague, 2004.
Čadík, M. Human Perception and Computer Graphics
Postgraduate Study Report. Czech Technical University in Prague, 2004.
Čadík, M., Slavík, P. Comparing Image-Processing Operators by Means of the Visible Differences Predictor WSCG 2004. Pilsen: University of West Bohemia, 2004.


RESOURCES

Our experimental data and input images
Our HDR video quality dataset
Our LOCCG dataset (localized distortions in computer graphics) for evaluation of image quality metrics
Our LOCCG visual saliency dataset
Our CLFM (contrast-luminance-frequency-masking) dataset for low-level evaluation of IQMs


Image and Video Quality Assessment at LIVE
Video Quality Experts Group (VQEG)