Californian university project will use facial recognition software to identify subjects of paintings
A Californian university has won funding to use advanced facial recognition technology to try to solve the mysteries of some of the world's most famous works of art.
Professor Conrad Rudolph said the idea for the experiment came from watching news and detective shows such as CSI which had a constant theme of using advanced computers to recognise unknown faces from murder victims to wanted criminals.
Rudolph, professor of medieval art history at the University of California at Riverside, realised he might be able to apply that cutting-edge forensic science to some of the oldest mysteries in art: identifying the real people in paintings such as Vermeer's Girl with a Pearl Earring, Hals's The Laughing Cavalier or thousands of other portraits and busts where the identity of the subject has been lost. Work on the project should begin within a month or so.
Police and forensic scientists can use facial recognition software that identifies individuals by measuring certain key features. For example, it might measure the distance between someone's eyes or the gap between their mouth and their nose. In real life such measurements should be almost as unique as a fingerprint. Rudolph is hoping that the same might be true of portraiture, whether it is a sculpted bust or a painting.
To start with, his team will use facial recognition software on death masks of known individuals and then compare them to busts and portraits. If the software can find a match where Rudolph and his team know one exists, then it shows the technique works and can be used on unknown subjects to see if it can match them up with known identities.
The identity of the subjects of some of the most famous pictures in the world are unknown, including Girl with a Pearl Earring, the 17th-century portrait that inspired a film starring Scarlett Johansson. The Imagined Lives exhibition now running at London's National Portrait Gallery features portraits of 14 unknown subjects. Many of those paintings were once thought to be of historical figures such as Elizabeth I, but the identities are now disputed. The truth behind several paintings of Shakespeare – such as the Chandos portrait and the Cobbe portrait – has also been much disputed. It is possible facial recognition software could help solve these mysteries.
To be identified, the subject of a portrait would need to be matched to the identity of another named person in a separate picture. But Rudolph has some tricks up his sleeve. He believes that another forensic technique – whereby an "ageing" programme is run on a subject – could also help solve art mysteries. In fighting crime the software is usually used to produce "adult" pictures of children who have been missing for many years. But it could see if the Girl with a Pearl Earring had been painted again as a much older woman, whose identity might be known.
Away from the high-profile cases there are a legion of other unknown subjects that might be more easily identified. In many works from before the 19th century wealthy patrons often inserted themselves, their families or friends and business associates into crowd scenes.
Facial recognition technology could be used to identify some of these people from already known works and thus provide insight into personal, political and business relationships of the day. In other cases families in wealthy homes commissioned busts of relatives that were often sold when estates went bankrupt or families declined.
The new technique could identify many of these people by linking the busts to known portraits. "These are historical documents and they can teach us things. Works of art can show us political connections or business links. It opens up a whole new window into the past," Rudolph said.
In order to transfer the process to analysing faces in works of art, some technical issues will need to be overcome. Portraits are in two dimensions and are also an artistic interpretation rather than a definitive likeness. In some cases, the painter might have simply not been very accurate, or attempted to flatter a subject, which would make recognition more difficult.
"It is different using this on art rather than an actual human," said Rudolph, "But we are trying to test the limits of the technology now and then who knows what advances may happen in the future? This is a fast-moving field."