L0-Regularized Intensity and Gradient Prior for Deblurring Text Images and Beyond

AN EXTENSION METHOD OF OUR TEXT DEBLURRING ALGORITHM 

Jinshan Pan    Zhe Hu    Zhixun Su     Ming-Hsuan Yang

Abstract

We propose a simple yet effective L0-regularized prior based on intensity and gradient for text image deblurring. The proposed image prior is motivated by observing distinct properties of text images. Based on this prior, we develop an efficient optimization algorithm to generate reliable intermediate results for kernel estimation. The proposed algorithm does not require any heuristic edge selection strategies which are critical to state-of-the-art edge-based deblurring methods. We discuss the relationship with other edge-based deblurring methods and discuss how to select salient edges more principally. For the final latent image restoration step, we present a effective method to remove artifacts and render better deblurred images. We show the proposed algorithm can be extended to deblur natural images with complex scenes and low illumination. In addition, we discuss the application of the proposed algorithm for non-uniform image deblurring. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art image deblurring methods.


Uniform Blind Image Deblurring

Blurred image Cho et al. ECCV 2012 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 IL0RIG
Blurred image Xu and Jia ECCV 2010 IL0RIG

Non-Uniform Blind Image Deblurring

Blurred image Xu et al. CVPR 2013 IL0RIG

Figure 1. Some challenging examples including text image, low-illumination image, and natural images. The L0RIG and IL0RIG in this material denote our text deblurring method and its extension.



Quantitative Evaluation on Text Image Dataset

We have created a dataset containing 15 images and 8 blur kernels extracted from Levin et al. CVPR 2009. Similar to this work, we can generate 120 different blurred images. The 15 ground truth images and 8 blur kernels are shown in below.

im01 im02 im03 im04 im05
im05 im06 im07 im08 im09
im11 im12 im13 im14 im15
kernel 1 kernel 2 kernel 3 kernel 4 kernel 5 kernel 6 kernel 7 kernel 8

Figure 2. Ground truth text images and blur kernels


Figure 3. Quantitative comparison on the proposed text image dataset. The x-axis denotes the image index and the average PSNR values of all the images are shown on the rightmost column.



Quantitative Evaluation on Low-Illumination Image Dataset

We have created a dataset containing 6 low-illumination images and 8 blur kernels extracted from Levin et al. CVPR 2009 (See Figure 2). Similar to the work [Hu et al. CVPR 2014], we stretch the pixel intensities of each image into [0, 2.2] and then apply 8 different blur kernels. Finally we clip the pixel intensities into dynamic range [0, 1]. The 6 ground truth images are shown in Figure 4.

im01 im02 im03 im04 im05 im06

Figure 4. Ground truth low-illumination images



Figure 5. Quantitative comparison on the low-illumination dataset. The x-axis denotes the image index and the average PSNR values of all the images are shown on the rightmost column.



Quantitative Evaluation on Natural Image Deblurring Datasets by Levin et al. CVPR 2009, Köhler et al. ECCV 2012, and Sun et al., ICCP 2013

To verify the validity of the proposed method on the natural images, we test our method on the dataset of Levin et al. CVPR 2009, which contains 4 ground truth images and 8 blur kernels. Figure 6(a) shows the cumulative histogram of the deconvolution error ratio across test examples. The proposed method achieves 100% of the results under error ratio 2.2.

(a) Results on the dataset by Levin et al. CVPR 2009 (b) Results on the dataset by Köhler et al. ECCV 2012 (c) Results on the dataset by Sun et al. ICCP 2013
Figure 6. Quantitative comparison on the natural image datasets by Levin et al. CVPR 2009, Köhler et al. ECCV 2012, and Sun et al., ICCP 2013. The numbers below the horizontal axis in (a) and (c) denote the error ratio values.



Visual Comparisons

    Results (including recovered latent images and blur kernels)     

          The followings are some visual comparison results with state-of-the-arts. For the estimated blur kernels, please click the above link to view the PDF file.

 

Visualization Results from the Proposed Text Image Dataset

Blurred image (PSNR: 14.74) Cho and Lee Siggraph Asia 2009 (PSNR: 21.63) Xu and Jia ECCV 2010 (PSNR: 18.65) Krishnan et al. CVPR 2011 (PSNR: 14.95)
Levin et al. CVPR 2011 (PSNR: 17.67) Xu et al. CVPR 2013 (PSNR: 17.46) L0RIG (PSNR: 18.95) IL0RIG (PSNR: 20.75)
Blurred image (PSNR: 13.92) Cho and Lee Siggraph Asia 2009 (PSNR: 14.57) Xu and Jia ECCV 2010 (PSNR: 14.79) Krishnan et al. CVPR 2011 (PSNR: 15.45)
Levin et al. CVPR 2011 (PSNR: 24.76) Xu et al. CVPR 2013 (PSNR: 27.10) L0RIG (PSNR: 30.62) IL0RIG (PSNR: 31.95)


Visualization Results from the Proposed Low-Illumination Image Dataset

Blurred image (PSNR: 23.57) Cho and Lee Siggraph Asia 2009 (PSNR: 23.88) Xu and Jia ECCV 2010 (PSNR: 24.17) Krishnan et al. CVPR 2011 (PSNR: 23.90)
Xu et al. CVPR 2013 (PSNR: 24.07) Hu et al. CVPR 2014 (PSNR: 22.62) L0RIG (PSNR: 25.95) IL0RIG (PSNR: 26.53)
Blurred image (PSNR: 22.68) Cho and Lee Siggraph Asia 2009 (PSNR: 23.24) Xu and Jia ECCV 2010 (PSNR: 23.43) Krishnan et al. CVPR 2011 (PSNR: 23.72)
Xu et al. CVPR 2013 (PSNR: 23.26) Hu et al. CVPR 2014 (PSNR: 21.48) L0RIG (PSNR: 24.75) IL0RIG (PSNR: 25.26)


Visualization Results from the Dataset of Levin et al. CVPR 2009

Blurred image Shan et al. Siggraph 2008 Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010
Levin et al. CVPR 2011 Xu et al. CVPR 2013 L0RIG IL0RIG


Visualization Results from the Dataset of Köhler et al. ECCV 2012

Blurred image Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010 Krishnan et al. CVPR 2011
Hirsch et al. ICCV 2011 Whyte et al. IJCV 2012 L0RIG IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010 Krishnan et al. CVPR 2011
Hirsch et al. ICCV 2011 Whyte et al. IJCV 2012 L0RIG IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010 Krishnan et al. CVPR 2011
Hirsch et al. ICCV 2011 Whyte et al. IJCV 2012 L0RIG IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010 Krishnan et al. CVPR 2011
Hirsch et al. ICCV 2011 Whyte et al. IJCV 2012 L0RIG IL0RIG


Other Synthetic Images

Blurred image Cho and Lee Siggraph Asia 2009 Xu et al. CVPR 2013 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Levin et al. CVPR 2011 IL0RIG


Real Captured Images

Blurred image Cho and Lee Siggraph Asia 2009 Cho et al. ECCV 2012 IL0RIG
Blurred image Xu and Jia ECCV 2010 Cho et al. ECCV 2012 IL0RIG
Blurred image Chen et al. CVPR 2011 Cho et al. ECCV 2012 IL0RIG
Blurred image Xu et al. CVPR 2013 Hu et al. CVPR 2014 IL0RIG
Blurred image Xu et al. CVPR 2013 Hu et al. CVPR 2014 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010 IL0RIG
Blurred image Result from DeblurFamousPhoto Xu and Jia ECCV 2010 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Xu et al. CVPR 2013 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010 IL0RIG
L0RIG_deblur/roma_krishnan.png
Blurred image Xu and Jia ECCV 2010 Krishnan et al. CVPR 2011 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Xu and Jia ECCV 2010 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Zhong et al. CVPR 2013 IL0RIG
Blurred image Cho and Lee Siggraph Asia 2009 Zhong et al. CVPR 2013 IL0RIG
Blurred image Xu et al. CVPR 2013 Pan et al. ECCV 2014 IL0RIG
Blurred image Gupta et al. ECCV 2010 Xu et al. CVPR 2013 IL0RIG
Blurred image Gupta et al. ECCV 2010 Xu et al. CVPR 2013 IL0RIG
Blurred image Whyte et al. CVPR 2010 Xu et al. CVPR 2013 IL0RIG
Blurred image Hirsch et al. ICCV 2011 Xu et al. CVPR 2013 IL0RIG
Blurred image Hirsch et al. ICCV 2011 Xu et al. CVPR 2013 IL0RIG
Blurred image Whyte et al. IJCV 2012 Xu et al. CVPR 2013 IL0RIG



Technical Papers, Codes, and Datasets

Jinshan Pan, Zhe Hu, Zhixun Su, and Ming-Hsuan Yang, "Deblurring Text Images via L0-Regularized Intensity and Gradient Prior", IEEE Conference on Computer Vision and Pattern Recognition ( CVPR), 2014

    Paper      

    MATLAB code   

    Datasets 

Jinshan Pan, Zhe Hu, Zhixun Su, and Ming-Hsuan Yang, " L0-Regularized Intensity and Gradient Prior for Deblurring Text Images and Beyond", IEEE Transactions on Pattern Analysis and Machine Intelligence ( TPAMI), 2016

    Paper       

  MATLAB code         

    Datasets 

References

[1] R. Fergus, B. Singh, A. Hertzmann, S.T. Roweis, and W.T. Freeman. “Removing camera shake from a single photograph”, SIGGRAPH 2006.

[2] Q. Shan, J. Jia, and A. Agarwala. “High-quality motion deblurring from a single image”, SIGGRAPH 2008.

[3] H. Cho, J. Wang, and S. Lee. “Text image deblurring using text-specific properties,” ECCV 2012.

[4] S. Cho and S. Lee. “Fast motion deblurring”, SIGGRAPH ASIA 2009.

[5] L. Xu and J. Jia. “Two-phase kernel estimation for robust motion deblurring”, ECCV 2010.

[6] L. Xu, S. Zheng, and J. Jia. “Unnatural L0 sparse representation for natural image deblurring”, CVPR 2013.

[7] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman, “Understanding and evaluating blind deconvolution algorithms,” CVPR 2009.

[8] D. Krishnan, T. Tay and R. Fergus. “Blind deconvolution using a normalized sparsity measure”, CVPR 2011.

[9] R. Kohler, M. Hirsch, B. Mohler and B. Scholkopf. “Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database”, ECCV 2012.

[10] O. Whyte, J. Sivic, A. Zisserman, and J. Ponce. “Non-uniform deblurring for shaken images”, CVPR 2010.

[11] A. Gupta, N. Joshi, L. Zitnick, M. Cohen, and B. Curless. “Single image deblurring using motion density functions”, ECCV 2010.

[12] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman. "Efficient marginal likelihood optimization in blind deconvolution", CVPR 2011.

[13] D. Krishnan, T. Tay, and R. Fergus. “Blind deconvolution using a normalized sparsity measure”, CVPR 2011.

[14] M. Hirsch, C. Schuler, S. Harmeling, and B. Scholkopf. “Fast removal of non-uniform camera shake”, ICCV 2011.

[15] Z. Hu, S. Cho, J. Wang, and M.-H. Yang. “Deblurring low-light images with light streaks”, CVPR 2014.

[16] O. Whyte, J. Sivic, A. Zisserman, and J. Ponce. “Non-uniform deblurring for shaken images”, IJCV 2012.

[17] X. Chen, X. He, J. Yang, and Q. Wu. “An effective document image deblurring algorithm”, CVPR 2011.

[18] J. Pan, Z. Hu, Z. Su, and M.-H. Yang. “Deblurring face images with exemplars”, ECCV 2014.