Never forget a face

Do you have a forgettable face? Many of us go to great lengths to make our faces more memorable, using makeup and hairstyles to give ourselves a more distinctive look.

Now your face could be instantly transformed into a more memorable one without the need for an expensive makeover, thanks to an algorithm developed by researchers in MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).

The algorithm, which makes subtle changes to various points on the face to make it more memorable without changing a person’s overall appearance, was unveiled earlier this month at the International Conference on Computer Vision in Sydney.

“We want to modify the extent to which people will actually remember a face,” says lead author Aditya Khosla, a graduate student in the Computer Vision group within CSAIL. “This is a very subtle quality, because we don’t want to take your face and replace it with the most memorable one in our database, we want your face to still look like you.”

More memorable — or less

The system could ultimately be used in a smartphone app to allow people to modify a digital image of their face before uploading it to their social networking pages. It could also be used for job applications, to create a digital version of an applicant’s face that will more readily stick in the minds of potential employers, says Khosla, who developed the algorithm with CSAIL principal research scientist Aude Oliva, the senior author of the paper, Antonio Torralba, an associate professor of electrical engineering and computer science, and graduate student Wilma Bainbridge.

Conversely, it could also be used to make faces appear less memorable, so that actors in the background of a television program or film do not distract viewers’ attention from the main actors, for example.

To develop the memorability algorithm, the team first fed the software a database of more than 2,000 images. Each of these images had been awarded a “memorability score,” based on the ability of human volunteers to remember the pictures. In this way the software was able to analyze the information to detect subtle trends in the features of these faces that made them more or less memorable to people.

The researchers then programmed the algorithm with a set of objectives — to make the face as memorable as possible, but without changing the identity of the person or altering their facial attributes, such as their age, gender, or overall attractiveness. Changing the width of a nose may make a face look much more distinctive, for example, but it could also completely alter how attractive the person is, and so would fail to meet the algorithm’s objectives.

When the system has a new face to modify, it first takes the image and generates thousands of copies, known as samples. Each of these samples contains tiny modifications to different parts of the face. The algorithm then analyzes how well each of these samples meets its objectives.

Once the algorithm finds a sample that succeeds in making the face look more memorable without significantly altering the person’s appearance, it makes yet more copies of this new image, with each containing further alterations. It then keeps repeating this process until it finds a version that best meets its objectives.

“It’s really like applying an elastic mesh onto the photograph that slightly modifies the face,” Oliva says. “So the face still looks like you, but maybe with a bit of lifting.”

The team then selected photographs of 500 people and modified them to produce both a memorable and forgettable version of each. When they tested these images on a group of volunteers, they found that the algorithm succeeded in making the faces more or less memorable, as required, in around 75 percent of cases.

Familiarity breeds likability

Making a face appear familiar can also make it seem more likable, Oliva says. She and Bainbridge have published a complementary paper in the Journal of Experimental Psychology: General on the attributes that make a face memorable. The first time we see a face, we tend to “tag” it with attributes based on appearance, such as intelligence, kindness, or coldness. “If we tag a person with familiarity, because we think this is a face we have seen before, we have a tendency to like it more, and for instance to think the person is more trustworthy,” she says.

The team is now investigating the possibility of adding other attributes to their model, so that it could modify faces to be both more memorable and to appear more intelligent or trustworthy, for example. “So you could imagine having a system that would be able to change the features of your face to make you whatever you would wish for, but always in a very subtle way,” Oliva says.

We all wish to use a photo that makes us more visible to our audience, says Aleix Martinez, an associate professor of electrical and computer engineering at Ohio State University. “Painters of the Renaissance knew how to make portraits memorable, but we are clueless on how to take that picture that will give us an edge over others or, at a minimum, show the best side of us,” Martinez says.

Now Oliva and her team have developed a computational algorithm that can do this for us, he says. “Input your preferred picture of your face and it will make it even better,” Martinez says. “This will allow us to gain that advantage we were looking for and hopefully make people remember us more.”

The research was funded by grants from Xerox, Google, Facebook, and the Office of Naval Research.