The last fifty years have seen an impressive development of mathematical methods for the analysis and processing of digital images, mostly in the context of photography, biomedical imaging and various forms of engineering. The arts have been mostly overlooked in this process, apart from a few exceptional works in the last ten years. With the rapid emergence of digitisation in the arts, however, the arts domain is becoming increasingly receptive to digital image processing methods and the importance of paying attention to this therefore increases.

Virtual image restoration, also called image inpainting, denotes the process whereby missing or occluded parts in images are filled in based on some a-priori information that is, e.g., provided by the intact parts of the image. In this talk I will sketch and motivate different mathematical principles that can guide a digital restoration attempt. Digital photographs of art pieces are essentially mathematical objects, and this puts the vast toolbox of mathematics at the restorers’ fingertips.

We will encounter the role of differential equations, patch-based methods and deep learning for virtually restoring structure, texture and colour in images. In particular, we will show examples from the restoration of the Neidhart frescoes (Tuchlauben, Vienna), the restoration of a painting by Sebastiano Del Piombo (the Hamilton Kerr Institute, The Fitzwilliam Museum), and the unveiling of hidden structures in illuminated manuscripts revealed by infrared imaging (part of the MACH project). After a critical discussion of restoration results I will conclude by pointing out the capabilities and limitations of digital restoration methods, and provide some hints towards other applications of mathematics in cultural heritage, including paint layer and pottery classification.