Duke University researchers have developed an artificial intelligence tool that can transform blurry and unrecognizable images of people's faces into compelling computer-generated portraits, with finer details than ever.
Methods prior to Pulse GAN could detail an image with blurry pixels up to eight times its original resolution. But Duke's team figured out a way to take a handful of pixels and create realistic-looking faces at up to 64 times the resolution, "imagining" features like fine lines, eyelashes and wrinkles that weren't there in the first image .
No images with this resolution have ever been created before
Cynthia Rudin, computer scientist, Duke University
It is not an identikit
The Pulse GAN system cannot be used to identify people, say the researchers: it won't turn a blurry, unrecognizable photo from a security camera into a crystal-clear image of a real person. Rather, it is capable of generating new faces that do not exist, but seem plausibly real.
The same technique could in theory take low-resolution photos of almost anything and create sharp, lifelike images, with applications ranging from medicine and microscopy to astronomy and satellite imagery, the co-author said Sachit Menon, double specialization in mathematics and computer science.
The researchers will present their method, called PULSE GAN, from tomorrow to June 19 at the 2020 Computer Vision and Pattern Recognition (CVPR) conference.
Traditional approaches start by taking a low-resolution image with blurry pixels and “guess” which extra pixels are needed by trying to make them match, on average, the corresponding pixels in high-resolution images that the computer has seen before. As a result of this averaging, textured areas in hair and skin that may not line up perfectly from pixel to pixel may appear blurry and indistinct.
The Duke team has taken a different approach
Instead of taking a low-resolution image and slowly adding new details, the system searches for AI-generated examples of high-resolution faces (now become very good at this), looking for ones that resemble the input image as closely as possible when reduced to the same size.
The team used a machine learning tool called GAN, or “generative adversarial network.” I talked about it in more depth in this article and in others on this site. GANs are neural networks trained on the same photo dataset. One network presents AI-created human faces that mimic those it was trained on, while the other takes this result and decides whether it is convincing enough to be mistaken for a real photo. The first network gets better and better with experience, until the second can't tell the difference. They compete with each other, in other words, and by competing they improve.
PULSE can create realistic-looking images from noisy, low-quality inputs. From a single blurry image of a face he can emit an unlimited number of realistic possibilities, each of which appears subtly different.
Even given pixelated photos where the eyes and mouth are barely recognizable, “our algorithm can do something with it. Something that traditional approaches cannot do." Word of the co-author Alex Damian, mathematician at Duke.
Pulse GAN, “fantasy” in power
The system can convert a blurry pixel, or 16x16 pixel image into a 1024 x 1024 pixel image in seconds, adding more than a million pixels, similar to HD resolution. Details such as pores, wrinkles and strands of hair, imperceptible in low resolution photos, become sharp and clear in the computer generated versions.
The researchers asked 40 people to evaluate 1.440 images generated via PULSE GAN and five other resizing methods. Their judgment? A number on a scale of one to five. and Pulse GAN has done best of all. What's more, he scored almost equal to high-quality photos of real people.
See the results for yourself https://pulse.cs.duke.edu/.
