New AI notices invisible similarities between works of art

A new algorithm has emerged that can study works of art and find similarities between them. The model trains itself and gets better with each iteration.

The new system, developed by researchers at MIT, finds subtle similarities between works of art. The MosAIc model scans images and then uses deep networks to find similarities in them – it could be cultural similarities, similar working methods, or details that even art historians cannot notice.

To use MosAIc, the user uploads images there, and the algorithm finds similar works of art. In one example, MosAIc linked Francisco de Zurbaran’s The Martyrdom of Saint Serapion and Jan Asselin’s The Frightened Swan. The researchers explained that the two artists never met each other, did not correspond, but the model was able to find several plots that underlie the two works.

A particularly challenging aspect of MosAIc’s development was creating an algorithm that can find not only similarities in color or style but also plots in works of art. The researchers examined a deep network of connections that art historians had already noticed, and the algorithm studied the logic of how some works of art relate to others.

The researchers also used a new data structure for searching images – the KNN Tree, which combines images into a tree structure. To find the closest match of one image to another, the algorithm starts with the trunk of the links and then follows the nearest perspective branch. Thus, the data structure improves on its own.

Scientists hope that their development can be useful in other areas – the humanities, social sciences, and medicine. “These areas are rich in information that has never been processed through our methods. They can become a source of inspiration for both scientists and people who are simply interested”.