MIT PhD Student Uses AI to Restore Ancient Paintings in Just 3.5 Hours

An MIT doctoral student has published a groundbreaking paper as the sole author in Nature, introducing an AI-based method to restore damaged paintings. The entire restoration process can now be completed in just 3.5 hours, significantly outpacing traditional methods. To conduct the restoration, Alex used high-resolution commercial printing equipment to overlay a bi-layer mask onto the artwork. This mask, created using soluble materials, can be removed with specialized solvents if necessary, ensuring that the original piece remains unharmed. Additionally, digital files of the masked areas are preserved, providing a permanent record of the restoration work. Alex noted that restoring a painting with damage similar to that of a severely weathered Monet took about nine months with traditional techniques. "The more severe the damage, the more pronounced the advantages of this new technology," he said. The efficiency gain is estimated to be about 66 times faster compared to manual restoration by human artists. One of the key innovations in this research is the approach taken for repairing damage in uniformly colored areas. For such regions, like those affected by stains or small spots, a 15-resolution repair brush tool was used to digitally reconstruct the missing parts based on surrounding color patterns. This technique analyzes the color and texture of the surrounding area, generating natural-looking fill-ins to mend the damaged sections. For areas with complex visual patterns, such as intricate landscapes, a combination of partial convolution algorithms and repair brushes is employed. Using undamaged sections of the wooden panel as reference, this method selects top-quality wood grain samples for patching up and stretching the damaged areas. When reference images are available, the process relies on the restoration artist’s manual use of the brush tool, guided by the color and texture at the edges of the damaged region. When dealing with high-complexity losses, such as facial features, the system uses feature migration technology. In this case, a well-preserved face from a related work serves as a foundational model, which is then color-corrected and texture-matched before being transferred to the damaged area. The research involved the restoration of a child’s face from a well-preserved sample in the "Seashell Still Life" at the National Gallery of Art in Washington, D.C. Alex emphasized that widespread adoption of this method will require input from art conservation experts to ensure that the restoration results align with the original artistic style and integrity. "Each step involves thoughtful considerations, and we must establish frameworks that respect the principles of conservation. Currently, this study provides a solid foundation for future technological development, and as more researchers contribute, we aim to refine the restoration system even further," he stated. This research, detailed in a paper published on the MIT News website, marks a significant advancement in the field of art restoration, promising to not only speed up the process but also to improve the precision and quality of repairs. The original link to the article can be found here: MIT News.