ICCV ’17 Paper Accepted!

Our paper on Deep Generative Adversarial Compression Artifact Removal, has been accepted for publication at ICCV 2017.  In the following figure we can see how our GAN can recover details in a compressed image (left). Note how texture and edges are better looking and blocking, ringing and color quantization artifacts are removed.

We have shown that it is possible to remove compression artifacts by transforming images with deep convolutional residual networks. We have trained a generative network using SSIM loss obtaining state of the art results according to standard image similarity metrics. Nonetheless, images reconstructed as such appear blurry and missing details at higher frequencies. These details make images look less similar to the original ones for human viewers and harder to understand for object detectors. We therefore propose a conditional Generative Adversarial framework which we train alternating full size patch generation with sub-patch discrimination. Human evaluation and quantitative experiments in object detection show that our GAN generates images with finer consistent details and these details make a difference both for machines and humans.

We developed a simple demo to show how our GAN applied to compressed images is able to generate pleasant and semantically correct images.

Artifact Removal Lens

Hover your mouse to see the reconstruction of the image.


More results are available here!

Interestingly, our GAN can improve image quality for object detectors. We tested Faster R-CNN on compressed and restored images obtaining the following results on PASCAL VOC 07.

Method
JPEG 20 .587 .692 .516 .434 .350 .673 .710 .559 .334 .559 .579
AR-CNN .641 .686 .523 .413 .367 .702 .742 .530 .363 .574 .607
Our GAN .666 .753 .565 .475 .395 .727 .770 .725 .403 .684 .602
Original .698 .788 .692 .559 .488 .769 .798 .858 .487 .762 .637
mAP
JPEG 20 .532 .691 .665 .638 .260 .482 .434 .707 .570 .549
AR-CNN .581 .724 .661 .658 .313 .499 .526 .712 .578 .570
Our GAN .718 .753 .707 .670 .303 .625 .586 .712 .611 .623
Original .790 .802 .757 .763 .376 .683 .672 .777 .667 .691

Note how the most improvement happens for cat (+16.6), cow (+12.5), dog (+18.6) and sheep (+14.3), which are classes where the object is highly articulated and texture is the most informative cue.

This entry was posted in Uncategorized. Bookmark the permalink.

Comments are closed.