Abstract
Image restoration is a long-standing problem in image processing and low-level computer vision. Recently, discriminative convolutional neural network (CNN)-based approaches have attracted considerable attention due to their superior performance. However, most of these frameworks are designed for one specific image restoration task; hence, they seldom show high performance on other image restoration tasks. To address this issue, we propose a flexible deep CNN framework that exploits the frequency characteristics of different types of artifacts. Hence, the same approach can be employed for a variety of image restoration tasks by adjusting the architecture. For reducing the artifacts with similar frequency characteristics, a quality enhancement network that adopts residual and recursive learning is proposed. Residual learning is utilized to speed up the training process and boost the performance; recursive learning is adopted to significantly reduce the number of training parameters as well as boost the performance. Moreover, lateral connections transmit the extracted features between different frequency streams via multiple paths. One aggregation network combines the outputs of these streams to further enhance the restored images. We demonstrate the capabilities of the proposed framework with three representative applications: image compression artifacts reduction (CAR), image denoising, and single image super-resolution (SISR). Extensive experiments confirm that the proposed framework outperforms the state-of-the-art approaches to benchmark datasets for these applications.
Author Keywords
- Image restoration,
- flexible CNN framework,
- image decomposition,
- recursive learning,
- residual learning
IEEE Keywords
- Training,
- Image restoration,
- Image coding,
- Automobiles,
- Task analysis,
- Transform coding,
- Image denoising
Introduction
Image restoration (IR), as one of the most fundamental tasks in image processing and low-level computer vision, aims to reconstruct the latent high-quality (HQ) image from its distorted observation [1]. Degradation can arise from coding artifacts, resolution limitations, transmission noise, object motion, and camera movement or a combination of them. Accordingly, IR includes image compression artifacts reduction (CAR), image denoising, single image super-resolution (SISR), deblurring, dehazing, etc. Due to the capability of providing state-of-the-art performance for high-level computer vision problems, deep learning (DL)-based methods have become a more recent trend to solve the IR problem [3]. Moreover, by focusing on learning a nonlinear mapping between the distorted images and their corresponding HQ images, regression-type neural networks have demonstrated impressive results on inverse problems with exact models [4]. The first DL-based solutions to SISR and image CAR tasks were the SRCNN [5] and ARCNN [6], respectively. They have demonstrated the effectiveness of CNNs by solely adopting shallow networks. However, due to the limited representation capacity of shallow networks, they still suffer from over smoothing the reconstructed images. As a deep network, although DnCNN [7] is designed for image denoising, it shows promising performance on image CAR and SR as well. The end-to-end deep networks RED30 [8], ARN [9], MemNet [10], and MWCNN [11] target solving the IR problem; however, all of them treat all types of artifacts equally. Therefore, the different characteristics of various artifacts are not taken into account. Unfortunately, without specific consideration of various artifacts, one can observe the following issue. Specifically, the reduction of one type of artifact can lead to an unintentional increase in other types of artifacts [3, 12]. In addition, while seeking to fulfill “the deeper the better” premise, most of the deep networks suffer from high computational cost due to an enormous number of training parameters. For example, RED30 and ARN have 4,131k and 1,145k training parameters, respectively, while the number of training parameters of our network for image SR is only 594k.
Conclusion
In this work, we propose a flexible deep CNN framework for image restoration, which exploits the frequency characteristics of different types of artifacts. For specific IR tasks, the artifacts are first decomposed into a high-frequency or low-frequency group based on their characteristics. Then, according to the decomposition outcomes, the proposed framework can be efficiently adjusted to suppress these artifacts separately by the proposed quality enhancement network. In the proposed network, the multipath design helps the gradient flow and the transmission of the low-level features. Residual learning eases the training process. Recursive learning allows reducing the number of training parameters. In this way, the proposed flexible framework can significantly handle different artifacts while requiring fewer training parameters and less running time than the state-of-the-art approaches. Our future work will address an extension of the proposed framework from the quality enhancement of still images to video sequences by exploring the temporal relations.
About KSRA
The Kavian Scientific Research Association (KSRA) is a non-profit research organization to provide research / educational services in December 2013. The members of the community had formed a virtual group on the Viber social network. The core of the Kavian Scientific Association was formed with these members as founders. These individuals, led by Professor Siavosh Kaviani, decided to launch a scientific / research association with an emphasis on education.
KSRA research association, as a non-profit research firm, is committed to providing research services in the field of knowledge. The main beneficiaries of this association are public or private knowledge-based companies, students, researchers, researchers, professors, universities, and industrial and semi-industrial centers around the world.
Our main services Based on Education for all Spectrum people in the world. We want to make an integration between researches and educations. We believe education is the main right of Human beings. So our services should be concentrated on inclusive education.
The KSRA team partners with local under-served communities around the world to improve the access to and quality of knowledge based on education, amplify and augment learning programs where they exist, and create new opportunities for e-learning where traditional education systems are lacking or non-existent.
FULL Paper PDF file:
A Flexible Deep CNN Framework for Image RestorationBibliography
author
Year
2020
Title
A Flexible Deep CNN Framework for Image Restoration
Publish in
in IEEE Transactions on Multimedia, vol. 22, no. 4, pp. 1055-1068, April 2020,
Doi
10.1109/TMM.2019.2938340
PDF reference and original file: Click here
Somayeh Nosrati was born in 1982 in Tehran. She holds a Master's degree in artificial intelligence from Khatam University of Tehran.
