Inpainting: Difference between revisions

Content deleted Content added

Inline

Latest revision as of 12:08, 23 June 2024

Inpainting is a conservation process where damaged, deteriorated, or missing parts of an artwork are filled in to present a complete image.^[1] This process is commonly used in image restoration. It can be applied to both physical and digital art mediums such as oil or acrylic paintings, chemical photographic prints, sculptures, or digital images and video.

With its roots in physical artwork, such as painting and sculpture, traditional inpainting is performed by a trained art conservator who has carefully studied the artwork to determine the mediums and techniques used in the piece, potential risks of treatments, and ethical appropriateness of treatment.

History[edit]

The modern use of inpainting can be traced back to Pietro Edwards (1744–1821), Director of the Restoration of the Public Pictures in Venice, Italy. Using a scientific approach, Edwards focused his restoration efforts on the intentions of the artist.^[2]

It was during the 1930 International Conference for the Study of Scientific Methods for the Examination and Preservation of Works of Art, that the modern approach to inpainting was established. Helmut Ruhemann (1891–1973), a German restorer and conservator, led the discussions on the use of inpainting in conservation. Helmut Ruhemann was a leading figure in modernizing restoration and conservation.^[3] His greatest contribution to the field of conservation "was his insistence on following the methods of the original painter exactly, and on understanding the painter's artistic intention".^[4] After his career of over 40 years as a conservator, Ruhemann published his treatise The Cleaning of Paintings: Problems & Potentialities in 1968. In describing his method, Ruhemann states that "The surface [of the fill] should be slightly lower than that of the surrounding paint to allow for the thickness of the inpainting...Inpainting medium should look and behave like the original medium, but must not darken with age."^[5] Cesare Brandi (1906–1988) developed the teoria del restauro, the inpainting approach combining aesthetics and psychology. However, this approach was used primarily by Italian restorers and conservators, with the terminology becoming widespread in the 1990s.^[6]

Technological advancements led to new applications of inpainting. Widespread use of digital techniques range from entirely automatic computerized inpainting to tools used to simulate the process manually.^[7] Since the mid-1990s, the process of inpainting has evolved to include digital media. More commonly known as image or video interpolation, a form of estimation, digital inpainting includes the use of computer software that relies on sophisticated algorithms to replace lost or corrupted parts of the image data.

Ethics[edit]

In order to preserve the integrity of an original artwork, any inpainting technique or treatment applied to physical or digital work should be reversible or distinguishable from the original content of the artwork.^[8] Prior to any treatments, conservators proceed according to the American Institute of Conservation of Historical and Artistic Works.^[9]

There are several ethic considerations before Inpainting can be justified. Various deliberation decisions over the ethical appropriateness of the amount and type of inpainting done, resides on many factors. As most conservation treatments, inpainting's ethical questions rest mainly with authenticity, reversibility and documentation.

Any intervention to compensate for loss should be documented in treatment records and reports and should be detectable by common examination methods. Such compensation should be reversible and should not falsely modify the known aesthetic, conceptual, and physical characteristics of the cultural property, especially by removing or obscuring original material.^[10]

New technologies and the aesthetic demand for perfect images without imperfections challenge conservators' ethical practices to protect the integrity of originals.^[11]

Methods[edit]

Inpainting methods and techniques depend on the desired goal and type of image being treated. Treatments to fill in the gaps are different between physical and digital art. In inpainting, detailed records of the initial state of the images can help with the treatment and replicate the original closer.

Physical inpainting[edit]

Inpainting is rooted in the conservation and restoration of paintings. Inpainting can aim to make a visual improvement to the artwork as a whole by repairing missing or damaged parts using methods and materials equivalent to the original artist's work.^[12]

Application techniques[edit]

By studying the painting methods of various artists and the composition of paints used historically, conservators are able to restore works very closely to their original visual appearance. The picture as a whole determines how to fill in the gap.

Helmut Ruhemann's inpainting techniques by Jessell have procedures to "preserve" the quality of oil and tempera paintings.^[13]

Digital inpainting[edit]

Many programs are able to reconstruct missing or damaged areas of digital photographs and videos. Most widely known for use with digital images is Adobe Photoshop. Given the various abilities of the digital camera and the digitization of old photos, inpainting has become an automatic process that can be performed on digital images. The inpainting techniques can be applied to object removal, text removal, and other automatic modifications of images and videos.

In video special effects, inpainting is usually performed after video matting. They can also be observed in applications like image compression and super-resolution. In photography and cinema, it is used for film restoration to reverse, repair, or mitigate deterioration (e.g., physical damage such as cracks in photographs, scratches and dust spots in film, or chemical damage resulting in image loss; performed infrared cleaning). It can also be used for removing red-eye, the stamped date from photographs, and objects for creative effect. This technique can be used to replace any lost blocks in the coding and transmission of images, for example, in a streaming video. It can also be used to remove logos or watermarks in videos.

Deep learning neural network-based inpainting can be used for decensoring images.^[14] Deep image prior-based techniques can be used for digital image inpainting, where a trained deep learning model is either unavailable or infeasible.

Three main groups of 2D image-inpainting algorithms can be found in the literature. The first one to be noted is structural (or geometric) inpainting, the second one is texture inpainting, the last one is a combination of these two techniques. They use the information of the known or non-destroyed image areas in order to fill the gap, similar to how physical images are restored.

Structural[edit]

Structural or geometric inpainting is used for smooth images that have strong, defined borders.^[15] There are many different approaches to geometric inpainting, but they all come from the idea that geometry can be recovered from similar areas or domains. Bertalmio^[15] proposed a method of structural inpainting that mimics how conservators address painting restoration. Bertalmio proposed that by progressively transferring similar information from the borders of an inpainting domain inwards, the gap can be filled.^[16]

Textural[edit]

While structural/geometric inpainting works to repair smooth images, textural inpainting works best with images that are heavily textured.^[17] Texture has a repetitive pattern which means that a missing portion cannot be restored by continuing the level lines into the gap; level lines provide a complete, stable representation of an image.^[16] To repair texture in an image, one can combine frequency and spatial domain information to fill in a selected area with a desired texture. This method, while the most simple and very effective, works well when selecting a texture to be in-painted. For a texture that covers a wider area or a larger frame one would have to go through the image segmenting the areas to be in-painted and selecting the corresponding textures from throughout the image; there are programs that can help find the corresponding areas that work in a similar way as 'find and replace' works in a word processor.^[18]

Combined structural and textural[edit]

Combined structural and textural inpainting approaches simultaneously try to perform texture- and structure-filling in regions of missing image information. Most parts of an image consist of texture and structure and the boundaries between image regions contain a large amount of structural information. This is the result when blending different textures together. That is why some state-of-the-art methods attempt to combine structural and textural inpainting.

A more traditional method is to use differential equations (such as Laplace's equation) with Dirichlet boundary conditions for continuity so as to create a seemingly seamless fit. This works well if missing information lies within the homogeneous portion of an object area.^[19]

Other methods follow isophote directions (in an image, a contour of equal luminance), to do the inpainting.^[20]

Model based inpainting follows the Bayesian approach for which missing information is best fitted or estimated from the combination of the models of the underlying images, as well as the image data actually being observed. In deterministic language, this has led to various variational inpainting models.^[21]

Manual computer methods include using a clone tool to copy existing parts of the image to restore a damaged texture. Texture synthesis may also be used.^[22]

Exemplar-based image inpainting attempts to automate the clone tool process. It fills "holes" in the image by searching for similar patches in a nearby source region of the image, and copying the pixels from the most similar patch into the hole. By performing the fill at the patch level as opposed to the pixel level, the algorithm reduces blurring artifacts caused by prior techniques.^[23]^[24]

References[edit]

^ Newman, Richard (2011). Conservation and care of museum collections (1st ed.). MFA publications. p. 29. ISBN 978-0-87846-729-7.
^ Darrow, E.J. "Pietro Edwards and the restoration of the public pictures of Venice, 1778–1819: necessity introduced these arts". Research Works Archive. University of Washington. Retrieved 29 March 2020.
^ "Helmut Ruhemann Papers (1930-1979) | Archive | National Gallery, London". www.nationalgallery.org.uk. Retrieved 2024-03-20.
^ Jessell, Bettina (1977). "HELMUT RUHEMANN'S INPAINTING TECHNIQUES". cool.culturalheritage.org. Retrieved 2024-03-20.
^ Garland, Patricia, 2011, Chapter 3, Tradition of retouching Practices in America, pp 34–52, in Painting Conservation Catalog, Volume III Inpainting, The Paintings Specialty Group of the American Institute for Conservation, https://www.culturalheritage.org/docs/default-source/resource-guides/painting-conservation-catalog-volume-3-(inpainting).pdf, Retrieved November 2, 2019.
^ Idelson, Antonia Iaccarino; Severini, Leonardo (18 June 2018). "Inpainting". The Encyclopedia of Archaeological Sciences: 1–4. doi:10.1002/9781119188230.saseas0330. ISBN 9780470674611.
^ Bertalmio, Marcelo; Sapiro, Guillermo. "Image Inpainting" (PDF). University of Minnesota. Retrieved 29 March 2020.^{[dead link]}
^ "Inpainting | Museu Nacional d'Art de Catalunya". www.museunacional.cat. 25 March 2014. Retrieved 29 March 2020.
^ (1994) “AIC Code of Ethics and Guidelines for Practice”[1] https://www.culturalheritage.org/docs/default-source/administration/governance/code-of-ethics-and-guidelines-for-practice.pdf. Retrieved November 3, 2019.
^ American Institute of Conservation of Historical and Artistic Works. (1994). "AIC Code of Ethics and Guidelines for Practice". https://www.nps.gov/training/tel/Guides/HPS1022_AIC_Code_of_Ethics.pdf. Retrieved March 27, 2020.
^ Antonio laccarino Idelson. (28 June 2018). "Inpainting". The Encyclopedia of Archaeological Sciences. https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330 Retrieved March 26, 2020.
^ Idelson, Antonio; Severini, Leonardo (28 June 2018). "Inpainting". The Encyclopedia of Archeological Sciences: 1–4. https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330. Retrieved October 30, 2019.
^ Bettina Jessell. (1977). "Helmut Ruhemann's Inpainting Techniques". Journal of the American Institute for Conservation, JAIC 1977, Vol. 17, Number 1, Article 1 (pp. 08 - 08). Retrieved March 26, 2020.
^ "This Researcher Created 'DeepCreamPy,' a Machine Learning Algorithm That Uncensors Hentai – Motherboard". 2018-10-31.
^ ^a ^b Bugeau, Aurelie & Marcelo Bertalmio(2011).Combining Texture Synthesis and Diffusion for Image Inpainting.Hal Archives https://hal.archives-ouvertes.fr/hal-00551587/ Retrieved November 1 2019.
^ ^a ^b Cao, Frederic et al. (2011) "Geometrically Guided Exemplar-Based Inpainting." SIAM Journal of Imaging Sciences Vol 4. no.4 pp. 1143–1179. https://perso.telecom-paristech.fr/gousseau/inpaintingSIAM.pdf reterieved November 2 2019.
^ Bugeau, Aurelie & Marcelo Bertalmio(2011).Combining Texture Synthesis and Diffusion for Image Inpainting.Hal Archives https://hal.archives-ouvertes.fr/hal-00551587/
^ Bertalmio, Marcelo & Guillermo Saprio."Image Inpainting" https://conservancy.umn.edu/bitstream/handle/11299/3365/1/1655.pdf
^ Peterson, Ivars (11 May 2002). "Filling in Blanks". Science News. 161 (19): 299–300. doi:10.2307/4013521. JSTOR 4013521.
^ Bertalmío, M.; Sapiro, G.; Caselles, V.; Ballester, C. (2000). "Image Inpainting". Proceedings of the 27th annual conference on Computer graphics and interactive techniques – SIGGRAPH '00. pp. 417–424. doi:10.1145/344779.344972. ISBN 1581132085. S2CID 308278.
^ Chan, T. F.; Shen, J. (2001). "Mathematical Models for Local Nontexture Inpainting". SIAM J. Appl. Math. 62 (3): 1019–1043. doi:10.1137/S0036139900368844.
^ Igehy, Homan; Pereira, Lucas (1997). "Image Replacement through Texture Synthesis". Proceedings of International Conference on Image Processing. Vol. 3. pp. 186–189. doi:10.1109/ICIP.1997.632049. ISBN 0-8186-8183-7. S2CID 5895502.
^ Criminisi, A.; Perez, P.; Toyama, K. (2003). "Object Removal by Exemplar-Based Inpainting". 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. Vol. 2. pp. II-721–II-728. doi:10.1109/CVPR.2003.1211538. ISBN 0-7695-1900-8. S2CID 315138.
^ Lorenzi, L.; Melgani, F.; Mercier, G. (2011). "Inpainting Strategies for Reconstruction of Missing Data in VHR Images". IEEE Geoscience and Remote Sensing Letters. 8 (5): 914–918. Bibcode:2011IGRSL...8..914L. doi:10.1109/LGRS.2011.2141112. S2CID 2119517.

Bugeau Aurelie & Bertalmio, Marcelo (2011) "Combining Texture Synthesis and Diffusion for Image Inpainting." Hal Archives. Combining Texture Synthesis and Diffusion for Image Inpainting.

External links[edit]

Interactive Point-and-Click Segmentation for Object Removal in Digital Images, ICCV-HCI 2005
Image and Video Inpainting (webarchive) by Guillermo Sapiro
Mathematica Inpaint function.
Inpainting of images on meshes or point clouds, TIP 2014
Image Completion using Planar Structure Guidance, SIGGRAPH 2014

[1] Newman, Richard (2011). Conservation and care of museum collections (1st ed.). MFA publications. p. 29. ISBN 978-0-87846-729-7.

[2] Darrow, E.J. "Pietro Edwards and the restoration of the public pictures of Venice, 1778–1819: necessity introduced these arts". Research Works Archive. University of Washington. Retrieved 29 March 2020.

[3] "Helmut Ruhemann Papers (1930-1979) | Archive | National Gallery, London". www.nationalgallery.org.uk. Retrieved 2024-03-20.

[4] Jessell, Bettina (1977). "HELMUT RUHEMANN'S INPAINTING TECHNIQUES". cool.culturalheritage.org. Retrieved 2024-03-20.

[5] Garland, Patricia, 2011, Chapter 3, Tradition of retouching Practices in America, pp 34–52, in Painting Conservation Catalog, Volume III Inpainting, The Paintings Specialty Group of the American Institute for Conservation, https://www.culturalheritage.org/docs/default-source/resource-guides/painting-conservation-catalog-volume-3-(inpainting).pdf, Retrieved November 2, 2019.

[6] Idelson, Antonia Iaccarino; Severini, Leonardo (18 June 2018). "Inpainting". The Encyclopedia of Archaeological Sciences: 1–4. doi:10.1002/9781119188230.saseas0330. ISBN 9780470674611.

[7] Bertalmio, Marcelo; Sapiro, Guillermo. "Image Inpainting" (PDF). University of Minnesota. Retrieved 29 March 2020.^{[dead link]}

[8] "Inpainting | Museu Nacional d'Art de Catalunya". www.museunacional.cat. 25 March 2014. Retrieved 29 March 2020.

[9] (1994) “AIC Code of Ethics and Guidelines for Practice”[1] https://www.culturalheritage.org/docs/default-source/administration/governance/code-of-ethics-and-guidelines-for-practice.pdf. Retrieved November 3, 2019.

[10] American Institute of Conservation of Historical and Artistic Works. (1994). "AIC Code of Ethics and Guidelines for Practice". https://www.nps.gov/training/tel/Guides/HPS1022_AIC_Code_of_Ethics.pdf. Retrieved March 27, 2020.

[11] Antonio laccarino Idelson. (28 June 2018). "Inpainting". The Encyclopedia of Archaeological Sciences. https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330 Retrieved March 26, 2020.

[12] Idelson, Antonio; Severini, Leonardo (28 June 2018). "Inpainting". The Encyclopedia of Archeological Sciences: 1–4. https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330. Retrieved October 30, 2019.

[13] Bettina Jessell. (1977). "Helmut Ruhemann's Inpainting Techniques". Journal of the American Institute for Conservation, JAIC 1977, Vol. 17, Number 1, Article 1 (pp. 08 - 08). Retrieved March 26, 2020.

[url_This_Researcher_Created_‘DeepCreamPy,’_a_Machine_Learning_Algorithm_That_Uncensors_Hentai_-_Motherboard-14] "This Researcher Created 'DeepCreamPy,' a Machine Learning Algorithm That Uncensors Hentai – Motherboard". 2018-10-31.

[:0-15] Bugeau, Aurelie & Marcelo Bertalmio(2011).Combining Texture Synthesis and Diffusion for Image Inpainting.Hal Archives https://hal.archives-ouvertes.fr/hal-00551587/ Retrieved November 1 2019.

[exemplar-16] Cao, Frederic et al. (2011) "Geometrically Guided Exemplar-Based Inpainting." SIAM Journal of Imaging Sciences Vol 4. no.4 pp. 1143–1179. https://perso.telecom-paristech.fr/gousseau/inpaintingSIAM.pdf reterieved November 2 2019.

[17] Bugeau, Aurelie & Marcelo Bertalmio(2011).Combining Texture Synthesis and Diffusion for Image Inpainting.Hal Archives https://hal.archives-ouvertes.fr/hal-00551587/

[18] Bertalmio, Marcelo & Guillermo Saprio."Image Inpainting" https://conservancy.umn.edu/bitstream/handle/11299/3365/1/1655.pdf

[19] Peterson, Ivars (11 May 2002). "Filling in Blanks". Science News. 161 (19): 299–300. doi:10.2307/4013521. JSTOR 4013521.

[20] Bertalmío, M.; Sapiro, G.; Caselles, V.; Ballester, C. (2000). "Image Inpainting". Proceedings of the 27th annual conference on Computer graphics and interactive techniques – SIGGRAPH '00. pp. 417–424. doi:10.1145/344779.344972. ISBN 1581132085. S2CID 308278.

[21] Chan, T. F.; Shen, J. (2001). "Mathematical Models for Local Nontexture Inpainting". SIAM J. Appl. Math. 62 (3): 1019–1043. doi:10.1137/S0036139900368844.

[22] Igehy, Homan; Pereira, Lucas (1997). "Image Replacement through Texture Synthesis". Proceedings of International Conference on Image Processing. Vol. 3. pp. 186–189. doi:10.1109/ICIP.1997.632049. ISBN 0-8186-8183-7. S2CID 5895502.

[23] Criminisi, A.; Perez, P.; Toyama, K. (2003). "Object Removal by Exemplar-Based Inpainting". 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. Vol. 2. pp. II-721–II-728. doi:10.1109/CVPR.2003.1211538. ISBN 0-7695-1900-8. S2CID 315138.

[24] Lorenzi, L.; Melgani, F.; Mercier, G. (2011). "Inpainting Strategies for Reconstruction of Missing Data in VHR Images". IEEE Geoscience and Remote Sensing Letters. 8 (5): 914–918. Bibcode:2011IGRSL...8..914L. doi:10.1109/LGRS.2011.2141112. S2CID 2119517.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

@@ Line 1: / Line 1: @@
+{{update|date=January 2024}}
-{{main|Image editing}}
+{{Short description|Image conservation process}}
-[[Image:Restoration.jpg|350px|thumb|right|Original and restored image]]
+[[File:Restoration using Artificial intelligence.jpg|thumb|Image restoration and recolorization using [[artificial intelligence]]]]'''Inpainting''' is a [[Art conservation|conservation]] process where damaged, deteriorated, or missing parts of an artwork are filled in to present a complete image.<ref>{{cite book |last1=Newman |first1=Richard |title=Conservation and care of museum collections |date=2011 |publisher=MFA publications |isbn=978-0-87846-729-7 |pages=29 |edition=1st}}</ref> This process is commonly used in [[image restoration]]. It can be applied to both physical and digital art [[List of art media|mediums]] such as [[Oil painting|oil]] or [[Acrylic paint|acrylic]] paintings, [[Photographic printing|chemical photographic prints]], [[sculpture]]s, or digital [[Digital imaging|images]] and [[Digital video|video]].
+With its roots in physical artwork, such as painting and sculpture, traditional inpainting is performed by a trained [[Conservator-restorer|art conservator]] who has carefully studied the artwork to determine the mediums and techniques used in the piece, potential risks of treatments, and ethical appropriateness of treatment.
-'''Inpainting''' is a [[Art conservation|conservation]] process where damaged, deteriorating, or missing parts of an artwork are filled in to present a complete image<ref>{{cite book |last1=Newman |first1=Richard |title=Conservation and care of museum collections |date=2011 |publisher=MFA publications |isbn=978-0-87846-729-7 |pages=29 |edition=1st}}</ref>. This process can be applied to both physical and digital art [[List of art media|mediums]] such as [[Oil painting|oil]] or [[Acrylic paint|acrylic]] paintings, [[Photographic printing|chemical photographic prints]], [[Sculpture|3-dimensional sculptures]], or digital [[Digital imaging|images]] and [[Digital video|video]].
-With its roots in physical artwork, such as painting and sculpture, traditional inpainting is performed by a trained [[Conservator-restorer|art conservator]] who has carefully studied the artwork to determine the mediums and techniques used in the piece, potential risks of treatments, and [https://www.conservation-wiki.com/wiki/BPG_Inpainting#Ethical_Considerations ethical appropriateness] of treatment.
 == History ==
-The modern use of inpainting can be traced back to [https://it.chped.com/wiki/Pietro_Edwards Pietro Edwards] (1744 - 1821), Director of the Restoration of the Public Pictures in [[Venice]], [[Italy]]. Using a scientific approach, Edwards focused his restoration efforts on the intentions of the artist.<ref>{{cite web |last1=Darrow |first1=E.J. |title=Pietro Edwards and the restoration of the public pictures of Venice, 1778-1819: necessity introduced these arts |url=https://digital.lib.washington.edu/researchworks/handle/1773/6223 |website=Research Works Archive |publisher=University of Washington |accessdate=29 March 2020}}</ref>
+The modern use of inpainting can be traced back to [[:it:Pietro Edwards|Pietro Edwards]] (1744–1821), Director of the Restoration of the Public Pictures in [[Venice]], [[Italy]]. Using a scientific approach, Edwards focused his restoration efforts on the intentions of the artist.<ref>{{cite web |last1=Darrow |first1=E.J. |title=Pietro Edwards and the restoration of the public pictures of Venice, 1778–1819: necessity introduced these arts |url=https://digital.lib.washington.edu/researchworks/handle/1773/6223 |website=Research Works Archive |publisher=University of Washington |access-date=29 March 2020}}</ref>
-It was during the 1930 International Conference for the Study of Scientific Methods for the Examination and Preservation of Works of Art, that the modern approach to inpainting was established. [[Helmut Ruhemann]] (1891-1973), a German restorer and conservator, led the discussions on the use of inpainting in conservation. Helmut Ruhemann was a leading figure in modernizing restoration  and conservation <ref>National Gallery,(2019), Helmut Ruhemann Papers:  1939-1979, https://www.nationalgallery.org.uk/archive/record/NG29.  Retrieved November 2. 2019.</ref>. His greatest contribution to the field of conservation "was his insistence on following the methods of the original painter exactly, and on understanding the painter's artistic intention"<ref>Jessell, Bettina, (1977), Helmut Ruhemann's Inpainting Techniques, Journal of the American Institute for Conservation, Volume 17, Number 1, Article 1 (pp. 01 to 08), http://cool.conservation-us.org/jaic/articles/jaic17-01-001.html, Retrieved November 2, 2019.</ref>. After his career of over 40 years as a conservator, Ruhemann published his treatise ''The Cleaning of Paintings:  Problems & Potentialities'' in 1968.  In describing his method, Ruhemann states that “The surface [of the fill] should be slightly lower than that of the surrounding paint to allow for the thickness of the inpainting...Inpainting medium should look and behave like the original medium, but must not darken with age.”<ref>Garland, Patricia, 2011, Chapter3, Tradition of retouching Practices in America, pp 34-52, in Painting Conservation Catalog, Volume III Inpainting,The Paintings Specialty Group of the American Institute for Conservation, https://www.culturalheritage.org/docs/default-source/resource-guides/painting-conservation-catalog-volume-3-(inpainting).pdf?sfvrsn=6, Retrieved November 2, 2019.</ref> [[Cesare Brandi]] (1906 - 1988) developed the ''teoria del restauro'', the inpainting approach combining aesthetics and psychology. However, this approach was used primarily by Italian restorers and conservators, with the terminology becoming widespread in the 1990s. <ref>{{cite journal |last1=Idelson |first1=Antonia Iaccarino |last2=Severini |first2=Leonardo |title=Inpainting |journal=The Encyclopedia of Archaeological Sciences |date=18 June 2018 |pages=1-4 |url=https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330 |accessdate=29 March 2020}}</ref>
+It was during the 1930 International Conference for the Study of Scientific Methods for the Examination and Preservation of Works of Art, that the modern approach to inpainting was established. [[Helmut Ruhemann]] (1891–1973), a German restorer and conservator, led the discussions on the use of inpainting in conservation. Helmut Ruhemann was a leading figure in modernizing restoration and conservation.<ref>{{Cite web |title=Helmut Ruhemann Papers (1930-1979) {{!}} Archive {{!}} National Gallery, London |url=https://www.nationalgallery.org.uk/research/research-centre/archive/record/NG29 |access-date=2024-03-20 |website=www.nationalgallery.org.uk}}</ref> His greatest contribution to the field of conservation "was his insistence on following the methods of the original painter exactly, and on understanding the painter's artistic intention".<ref>{{Cite web |last=Jessell |first=Bettina |date=1977 |title=HELMUT RUHEMANN'S INPAINTING TECHNIQUES |url=https://cool.culturalheritage.org/jaic/articles/jaic17-01-001.html |access-date=2024-03-20 |website=cool.culturalheritage.org}}</ref> After his career of over 40 years as a conservator, Ruhemann published his treatise ''The Cleaning of Paintings:  Problems & Potentialities'' in 1968.  In describing his method, Ruhemann states that "The surface [of the fill] should be slightly lower than that of the surrounding paint to allow for the thickness of the inpainting...Inpainting medium should look and behave like the original medium, but must not darken with age."<ref>Garland, Patricia, 2011, Chapter 3, ''Tradition of retouching Practices in America'', pp 34–52, in Painting Conservation Catalog, Volume III Inpainting, The Paintings Specialty Group of the American Institute for Conservation, https://www.culturalheritage.org/docs/default-source/resource-guides/painting-conservation-catalog-volume-3-(inpainting).pdf, Retrieved November 2, 2019.</ref> [[Cesare Brandi]] (1906–1988) developed the ''teoria del restauro'', the inpainting approach combining aesthetics and psychology. However, this approach was used primarily by Italian restorers and conservators, with the terminology becoming widespread in the 1990s.<ref>{{cite journal |last1=Idelson |first1=Antonia Iaccarino |last2=Severini |first2=Leonardo |title=Inpainting |journal=The Encyclopedia of Archaeological Sciences |date=18 June 2018 |pages=1–4 |doi=10.1002/9781119188230.saseas0330 |isbn=9780470674611 |doi-access=free }}</ref>
-Technological advancements led to new applications of inpainting. Widespread use of digital techniques range from entirely automatic computerized inpainting to tools used to simulate the process manually. <ref>{{cite web |last1=Bertalmio |first1=Marcelo |last2=Sapiro |first2=Guillermo |title=Image Inpainting |url=https://www.ima.umn.edu/sites/default/files/1655.pdf |website=University of Minnesota |publisher=University of Minnesota |accessdate=29 March 2020}}</ref> Since the mid-1990s, the process of inpainting has evolved to include [[Digital media|digital media]]. More commonly known as image or video interpolation, a form of estimation, digital inpainting includes the use of [[Software|computer software]] that relies on sophisticated [[Algorithm|algorithms]] to replace lost or corrupted parts of the image data.
+Technological advancements led to new applications of inpainting. Widespread use of digital techniques range from entirely automatic computerized inpainting to tools used to simulate the process manually.<ref>{{cite web |last1=Bertalmio |first1=Marcelo |last2=Sapiro |first2=Guillermo |title=Image Inpainting |url=https://www.ima.umn.edu/sites/default/files/1655.pdf |website=University of Minnesota |access-date=29 March 2020}}{{dead link|date=October 2023}}</ref> Since the mid-1990s, the process of inpainting has evolved to include [[digital media]]. More commonly known as image or video interpolation, a form of estimation, digital inpainting includes the use of [[Software|computer software]] that relies on sophisticated [[algorithm]]s to replace lost or corrupted parts of the image data.
 == Ethics ==
+{{worldwide|date=April 2021}}
-In order to preserve the integrity of an original artwork, any inpainting technique or treatment applied to physical or digital work should be reversible or distinguishable from the original content of the artwork<ref>{{cite web |title=Inpainting {{!}} Museu Nacional d'Art de Catalunya |url=https://www.museunacional.cat/en/inpainting |website=www.museunacional.cat |accessdate=29 March 2020}}</ref>. Prior to any treatments, conservators proceed according to the American Institute of Conservation of Historical and Artistic Works.<ref> (1994) “AIC Code of Ethics and Guidelines for Practice”[1] https://www.culturalheritage.org/docs/default-source/administration/governance/code-of-ethics-and-guidelines-for-practice.pdf?sfvrsn=21. Retrieved November 3, 2019.</ref>
+In order to preserve the integrity of an original artwork, any inpainting technique or treatment applied to physical or digital work should be reversible or distinguishable from the original content of the artwork.<ref>{{cite web |title=Inpainting {{!}} Museu Nacional d'Art de Catalunya |url=https://www.museunacional.cat/en/inpainting |website=www.museunacional.cat |date=25 March 2014 |access-date=29 March 2020}}</ref> Prior to any treatments, conservators proceed according to the American Institute of Conservation of Historical and Artistic Works.<ref> (1994) “AIC Code of Ethics and Guidelines for Practice”[1] https://www.culturalheritage.org/docs/default-source/administration/governance/code-of-ethics-and-guidelines-for-practice.pdf. Retrieved November 3, 2019.</ref>
-There are several ethic considerations before Inpainting can be [https://www.conservation-wiki.com/wiki/BPG_Inpainting#Ethical_Considerations justified]. Various deliberation decisions over the [https://www.conservation-wiki.com/wiki/BPG_Inpainting#Ethical_Considerations ethical appropriateness] of the amount and type of inpainting done, resides on many factors. As most conservation treatments, inpainting’s ethical questions rest mainly with [[Authenticity in Art|authenticity]], [[Conservation and restoration of cultural heritage|reversibility]] and [[documentation]].
+There are several ethic considerations before Inpainting can be justified. Various deliberation decisions over the ethical appropriateness of the amount and type of inpainting done, resides on many factors. As most conservation treatments, inpainting's ethical questions rest mainly with [[Authenticity in Art|authenticity]], [[Conservation and restoration of cultural heritage|reversibility]] and [[documentation]].<blockquote>Any intervention to compensate for loss should be documented in treatment records and reports and should be detectable by common examination methods. Such compensation should be reversible and should not falsely modify the known aesthetic, conceptual, and physical characteristics of the cultural property, especially by removing or obscuring original material.<ref>American Institute of Conservation of Historical and Artistic Works. (1994). "AIC Code of Ethics and Guidelines for Practice". https://www.nps.gov/training/tel/Guides/HPS1022_AIC_Code_of_Ethics.pdf. Retrieved March 27, 2020.</ref></blockquote>New technologies and the aesthetic demand for perfect images without imperfections challenge conservators' ethical practices to protect the integrity of originals.<ref>Antonio laccarino Idelson. (28 June 2018). "Inpainting". The Encyclopedia of Archaeological Sciences. https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330 Retrieved March 26, 2020.</ref>
-“Any intervention to compensate for loss should '''be documented''' in treatment records and reports and should '''be detectable''' by common examination methods. Such compensation should '''be reversible''' and should '''not falsely modify''' the known aesthetic, conceptual, and physical characteristics of the cultural property, especially by removing or obscuring original material.”<ref>American Institute of Conservation of Historical and Artistic Works. (1994). "AIC Code of Ethics and Guidelines for Practice". https://www.nps.gov/training/tel/Guides/HPS1022_AIC_Code_of_Ethics.pdf. Retrieved March 27, 2020.</ref>
-In an age of museum tourism, new technologies and aesthetic demand for perfect images without imperfections, continue to challenge conservators' ethical practices to protect the integrity of originals.<ref>Antonio laccarino Idelson. (28 June 2018). "Inpainting". The Encyclopedia of Archaeological Sciences. https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330 Retrieved March 26, 2020.</ref>
 == Methods ==
+Inpainting methods and techniques depend on the desired goal and type of image being treated. Treatments to fill in the gaps are different between physical and digital art. In inpainting, detailed records of the initial state of the images can help with the treatment and replicate the original closer.
+=== Physical inpainting ===
-Inpainting method techniques depend on the desired goal and type of image being treated. Treatments to fill in the gaps are very different between physical and digital art.
+[[File:Piero della Francesca - Scene after and before restoration - WGA17592.jpg|thumb|[[Piero della Francesca]] – Scene after and before restoration. (''The Queen of Sheba in'') ''Adoration of the'' (''Holy'') ''Wood''.]]
-With all applications of inpainting, it is important to keep detailed records of the initial state of the images, treatments done and justification for treatment, and the original copies when applicable (e.g. original digital images).
-=== Physical Inpainting ===
-[[File:Piero della Francesca - Scene after and before restoration - WGA17592.jpg|thumb|Piero della Francesca - Scene after and before restoration - WGA17592]]
-Inpainting is rooted in the restoration of painted images. In the [[conservation and restoration of paintings]], "the term inpainting refers to the compensation of paint losses—aiming at the recomposition of the missing parts of an image in order to improve its perception by making damages less visible".<ref> Idelson, Antonio; Severini, Leonardo (28 June 2018). "Inpainting". The Encyclopedia of Archeological Sciences: 1–4. https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330. Retrieved October 30, 2019.</ref> In other words, inpainting aims to make a visual improvement to the artwork as a whole by repairing missing or damaged parts using methods and materials equivalent to the original artist's work.
-==== Application Techniques ====
-By studying the painting methods of various artists, the composition of paints used historically, and taking the time to carefully study the medium one is working with, conservators are able to, using an array of methodology, restore works very closely to their original visual appearance.
-Other tips of inpainting:
+Inpainting is rooted in the [[conservation and restoration of paintings]]. Inpainting can aim to make a visual improvement to the artwork as a whole by repairing missing or damaged parts using methods and materials equivalent to the original artist's work.<ref> Idelson, Antonio; Severini, Leonardo (28 June 2018). "Inpainting". The Encyclopedia of Archeological Sciences: 1–4. https://onlinelibrary.wiley.com/doi/full/10.1002/9781119188230.saseas0330. Retrieved October 30, 2019.</ref>
-* The picture as a whole determines how to fill in the gap; the purpose of inpainting is to restore the unity of the work so it is crucial to know how the repaired piece will function within the rest of the image.
-* The structure of the area surrounding the gap ought to be continued into the gap. Contour lines that end at the gap boundary are to be carried on into the gap.
-* The different regions inside a gap, as defined by the contour lines, are filled with colors matching those of its boundary although the specific materials do not have to be identical. If alternate materials are to be used, it is important to test for potential reactivity.
-* The small details are painted, i.e. “texture” is added to ensure the eye will not be drawn first to the in-painted region.
+==== Application techniques ====
-Helmut Ruhemann's Inpainting Techniques by Jessell is filled with technique chioces and procedures to "preserve" the quality of Oil and early Tempera paintings.<ref>Bettina Jessell. (1977). "Helmut Ruhemann's Inpainting Techniques". Journal of the American Institute for Conservation, JAIC 1977, Vol. 17, Number 1, Article 1 (pp. 08 - 08). Retrieved March 26, 2020.</ref>
+By studying the painting methods of various artists and the composition of paints used historically, conservators are able to restore works very closely to their original visual appearance. The picture as a whole determines how to fill in the gap.
+Helmut Ruhemann's inpainting techniques by Jessell have procedures to "preserve" the quality of oil and [[Tempera|tempera paintings]].<ref>Bettina Jessell. (1977). "Helmut Ruhemann's Inpainting Techniques". Journal of the American Institute for Conservation, JAIC 1977, Vol. 17, Number 1, Article 1 (pp. 08 - 08). Retrieved March 26, 2020.</ref>
-=== Digital Inpainting ===
-There are currently many programs in use that are able to reconstruct missing or damaged areas of digital photographs and videos. Most widely known for use with digital images is [[Adobe Photoshop]]. Since the digital files are able to be duplicated, any restorative alterations should be made to the duplicate file, while maintaining the original files in an archive. Given the various abilities of the digital camera and the digitization of old photos, inpainting has become an automatic process that can be performed on digital images. More than mere scratch removal, the inpainting techniques can also be applied to object removal, text removal, and other automatic modifications of images and videos. Furthermore, they can also be observed in applications like [[image compression]] and [[super resolution]].
+=== Digital inpainting ===
-In [[photography]] and [[Film|cinema]], it is used for [[film restoration]] to reverse, repair, or mitigate deterioration (e.g. physical damage such as cracks in photographs or scratches and dust spots in film or chemical damage resulting in image loss; see [[infrared cleaning]]). It can also be used for removing [[red eye effect|red-eye]], the stamped date from photographs, and removing objects for creative effect.
+Many programs are able to reconstruct missing or damaged areas of digital photographs and videos. Most widely known for use with digital images is [[Adobe Photoshop]]. Given the various abilities of the digital camera and the digitization of old photos, inpainting has become an automatic process that can be performed on digital images. The inpainting techniques can be applied to object removal, text removal, and other automatic modifications of images and videos.
+In [[special effect|video special effects]], inpainting is usually performed after [[video matting]]. They can also be observed in applications like [[image compression]] and [[Super-resolution imaging|super-resolution]]. In [[photography]] and [[Film|cinema]], it is used for [[film restoration]] to reverse, repair, or mitigate deterioration (e.g., physical damage such as cracks in photographs, scratches and dust spots in film, or chemical damage resulting in image loss; performed [[infrared cleaning]]). It can also be used for removing [[red eye effect|red-eye]], the stamped date from photographs, and objects for creative effect. This technique can be used to replace any lost blocks in the coding and transmission of images, for example, in a [[streaming video]]. It can also be used to remove logos or watermarks in videos.
-[[File:Digital Image Restoration and Reconstraction.jpg|thumb|Digital Image Restoration and Reconstraction]]
+[[File:Digital Image Restoration and Reconstraction.jpg|thumb|Digital image restoration and reconstruction]]
-This technique can be used to replace any lost blocks in the coding and transmission of images, for example, in a [[streaming video]]. It can also be used to remove logos in videos.
-[[Deep learning]] [[neural network]] based inpainting can be used for decensoring images.<ref name="url This Researcher Created ‘DeepCreamPy,’ a Machine Learning Algorithm That Uncensors Hentai - Motherboard">{{cite web |url=https://motherboard.vice.com/en_us/article/43ekxg/machine-learning-algorithm-that-uncensors-hentai-porn-deepcreampy |title=This Researcher Created 'DeepCreamPy,' a Machine Learning Algorithm That Uncensors Hentai - Motherboard |format= |website= |accessdate=|date=2018-10-31 }}</ref>
+[[Deep learning]] [[neural network]]-based inpainting can be used for decensoring images.<ref name="url This Researcher Created ‘DeepCreamPy,’ a Machine Learning Algorithm That Uncensors Hentai - Motherboard">{{cite web |url=https://motherboard.vice.com/en_us/article/43ekxg/machine-learning-algorithm-that-uncensors-hentai-porn-deepcreampy |title=This Researcher Created 'DeepCreamPy,' a Machine Learning Algorithm That Uncensors Hentai – Motherboard |date=2018-10-31 }}</ref> [[Deep image prior]]-based techniques can be used for digital image inpainting, where a trained deep learning model is either unavailable or infeasible.
-Three main groups of 2D image inpainting algorithms can be found in literature. The first one to be noted is structural (or geometric) inpainting, the second one is texture inpainting, and the last one is a combination of these two techniques. All these inpainting methods have one thing in common: they use the information of the known or non-destroyed image areas in order to fill the gap, similar to how physical images are restored.
+Three main groups of 2D image-inpainting algorithms can be found in the literature. The first one to be noted is structural (or geometric) inpainting, the second one is texture inpainting, the last one is a combination of these two techniques. They use the information of the known or non-destroyed image areas in order to fill the gap, similar to how physical images are restored.
 ==== Structural ====
-Structural or geometric inpainting is used for smooth images that have strong, defined borders.<ref name=":0">Bugeau, Aurelie & Marcelo Bertalmio(2011).''Combining Texture Synthesis and Diffusion for Image Inpainting.''Hal Archives https://hal.archives-ouvertes.fr/hal-00551587/ Retrieved November 1 2019.</ref>. There are many different approaches to geometric inpainting, but they all stem from the same idea that geometry can be recovered from similar areas or domains. Bertalmio<ref name=":0" /> proposed a method of structural inpainting that mimics how conservators address painting restoration. Bertalmio proposed that by progressively transferring similar information from the borders of an inpainting domain inwards, the gap can be filled.<ref>Cao, Frederic et al. (2011) "Geometrically Guided Exemplar-Based Inpainting." ''SIAM Journal of Imaging Sciences'' Vol 4. no.4 pp 1143-1179. https://perso.telecom-paristech.fr/gousseau/inpaintingSIAM.pdf reterieved November 2 2019.</ref>
+Structural or geometric inpainting is used for smooth images that have strong, defined borders.<ref name=":0">Bugeau, Aurelie & Marcelo Bertalmio(2011).''Combining Texture Synthesis and Diffusion for Image Inpainting.''Hal Archives https://hal.archives-ouvertes.fr/hal-00551587/ Retrieved November 1 2019.</ref> There are many different approaches to geometric inpainting, but they all come from the idea that geometry can be recovered from similar areas or domains. Bertalmio<ref name=":0" /> proposed a method of structural inpainting that mimics how conservators address painting restoration. Bertalmio proposed that by progressively transferring similar information from the borders of an inpainting domain inwards, the gap can be filled.<ref name="exemplar">Cao, Frederic et al. (2011) "Geometrically Guided Exemplar-Based Inpainting." ''SIAM Journal of Imaging Sciences'' Vol 4. no.4 pp. 1143–1179. https://perso.telecom-paristech.fr/gousseau/inpaintingSIAM.pdf reterieved November 2 2019.</ref>
 ==== Textural ====
-While structural/geometric inpainting works to repair smooth images, textural inpainting works best with images that are heavily textured.<ref>Bugeau, Aurelie & Marcelo Bertalmio(2011).''Combining Texture Synthesis and Diffusion for Image Inpainting.''Hal Archives https://hal.archives-ouvertes.fr/hal-00551587/</ref> Texture has a repetitive pattern which means that a missing portion cannot be restored by continuing the level lines into the gap; level lines provide a complete, stable representation of an image.<ref>Cao, Frederic et al. (2011) "Geometrically Guided Exemplar-Based Inpainting." ''SIAM Journal of Imaging Sciences'' Vol 4. no.4 pp 1143-1179. https://perso.telecom-paristech.fr/gousseau/inpaintingSIAM.pdf retrieved November 2 2019. </ref> To repair texture in an image, one can combine frequency and spatial domain information to fill in a selected area with a desired texture. This method, while the most simple and very effective, works well when selecting a texture to be in-painted. For a texture that covers a wider area or a larger frame one would have to go through the image segmenting the areas to be in-painted and selecting the corresponding textures from throughout the image; there are programs that can help find the corresponding areas that work in a similar way as 'find and replace' works in a word processor.<ref>Bertalmio, Marcelo & Guillermo Saprio."Image Inpainting" https://conservancy.umn.edu/bitstream/handle/11299/3365/1/1655.pdf</ref>
+While structural/geometric inpainting works to repair smooth images, textural inpainting works best with images that are heavily textured.<ref>Bugeau, Aurelie & Marcelo Bertalmio(2011).''Combining Texture Synthesis and Diffusion for Image Inpainting.''Hal Archives https://hal.archives-ouvertes.fr/hal-00551587/</ref> Texture has a repetitive pattern which means that a missing portion cannot be restored by continuing the level lines into the gap; level lines provide a complete, stable representation of an image.<ref name="exemplar"></ref> To repair texture in an image, one can combine frequency and spatial domain information to fill in a selected area with a desired texture. This method, while the most simple and very effective, works well when selecting a texture to be in-painted. For a texture that covers a wider area or a larger frame one would have to go through the image segmenting the areas to be in-painted and selecting the corresponding textures from throughout the image; there are programs that can help find the corresponding areas that work in a similar way as 'find and replace' works in a word processor.<ref>Bertalmio, Marcelo & Guillermo Saprio."Image Inpainting" https://conservancy.umn.edu/bitstream/handle/11299/3365/1/1655.pdf</ref>
 ==== Combined structural and textural ====
-Combined structural and textural inpainting approaches simultaneously try to perform texture- and structure-filling in regions of missing image information.
+Combined structural and textural inpainting approaches simultaneously try to perform texture- and structure-filling in regions of missing image information. Most parts of an image consist of texture and structure and the boundaries between image regions contain a large amount of structural information. This is the result when blending different textures together. That is why some state-of-the-art methods attempt to combine structural and textural inpainting.
-Most parts of an image consist of texture and structure and the boundaries between image regions contain a large amount of structural information. This is the result when blending different textures together. That is why state of the art methods attempt to combine structural and textural inpainting.
-A more traditional method is to use [[differential equation]]s (such as the [[Laplace's equation]]) with [[Dirichlet boundary condition|Dirichlet boundary conditions]] for continuity so as to create a seemingly seamless fit. This works well if missing information lies within the homogeneous portion of an object area.<ref>{{cite journal
+A more traditional method is to use [[differential equation]]s (such as [[Laplace's equation]]) with [[Dirichlet boundary condition]]s for continuity so as to create a seemingly seamless fit. This works well if missing information lies within the homogeneous portion of an object area.<ref>{{cite journal
  | author = Peterson, Ivars
  |date=11 May 2002
@@ Line 82: / Line 64: @@
  }}</ref>
-Other methods follow [[Contour line|isophote]] directions (in an image, a contour of equal luminance), to do the inpainting.<ref>{{cite book |first=M. |last=Bertalmío |first2=G. |last2=Sapiro |first3=V. |last3=Caselles |first4=C. |last4=Ballester |chapter=Image Inpainting |work=Proceedings of SIGGRAPH 2000, New Orleans, USA, July 2000 |year=2000 |doi=10.1145/344779.344972 |title=Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 |pages=417–424 |isbn=1581132085 }}</ref>
+Other methods follow [[Contour line|isophote]] directions (in an image, a contour of equal luminance), to do the inpainting.<ref>{{cite book |first1=M. |last1=Bertalmío |first2=G. |last2=Sapiro |first3=V. |last3=Caselles |first4=C. |last4=Ballester |chapter=Image Inpainting |year=2000 |doi=10.1145/344779.344972 |title=Proceedings of the 27th annual conference on Computer graphics and interactive techniques – SIGGRAPH '00 |pages=417–424 |isbn=1581132085 |s2cid=308278 }}</ref>
-Recent investigations included the exploration of the wavelet transform properties to perform inpainting in the space-frequency domain, obtaining a better performance when compared to the frequency-based inpainting techniques.<ref>{{cite journal |last1=Penedo |first1=S. R. M. |last2=Cipparrone |first2=F. A. M. |last3=Justo |first3=J. F. |title=Digital Image Inpainting by Estimating Wavelet Coefficient Decays From Regularity Property and Besov Spaces |journal=IEEE Access |date=2019 |volume=7 |pages=3459–3471 |doi=10.1109/ACCESS.2018.2882364|doi-access=free }}</ref>
-Model based inpainting follows the [[Bayesian inference|Bayesian approach]] for which missing information is best fitted or estimated from the combination of the models of the underlying images, as well as the image data actually being observed. In deterministic language, this has led to various variational inpainting models.<ref>{{cite journal |first=T. F. |last=Chan |first2=J. |last2=Shen |title=Mathematical Models for Local Nontexture Inpainting |journal=SIAM J. Appl. Math. |volume=62 |issue=3 |year=2001 |pages=1019–1043 |doi=10.1137/S0036139900368844 }}</ref>
+Model based inpainting follows the [[Bayesian inference|Bayesian approach]] for which missing information is best fitted or estimated from the combination of the models of the underlying images, as well as the image data actually being observed. In deterministic language, this has led to various variational inpainting models.<ref>{{cite journal |first1=T. F. |last1=Chan |first2=J. |last2=Shen |title=Mathematical Models for Local Nontexture Inpainting |journal=SIAM J. Appl. Math. |volume=62 |issue=3 |year=2001 |pages=1019–1043 |doi=10.1137/S0036139900368844 }}</ref>
-Manual computer methods include using a [[clone tool]] to copy existing parts of the image to restore a damaged texture. [[Texture synthesis]] may also be used.<ref>{{cite book |chapter=Image Replacement through Texture Synthesis |first=Homan |last=Igehy |first2=Lucas |last2=Pereira |work=Proceedings of the 1997 IEEE International Conference on Image Processing |year=1997 |doi=10.1109/ICIP.1997.632049 |title=Proceedings of International Conference on Image Processing |volume=3 |pages=186–189 |isbn=0-8186-8183-7 }}</ref>
+Manual computer methods include using a [[clone tool]] to copy existing parts of the image to restore a damaged texture. [[Texture synthesis]] may also be used.<ref>{{cite book |chapter=Image Replacement through Texture Synthesis |first1=Homan |last1=Igehy |first2=Lucas |last2=Pereira |year=1997 |doi=10.1109/ICIP.1997.632049 |title=Proceedings of International Conference on Image Processing |volume=3 |pages=186–189 |isbn=0-8186-8183-7 |s2cid=5895502 }}</ref>
-Exemplar-based image inpainting attempts to automate the clone tool process.  It fills "holes" in the image by searching for similar patches in a nearby source region of the image, and copying the pixels from the most similar patch into the hole.  By performing the fill at the patch level as opposed to the pixel level, the algorithm reduces blurring artifacts caused by prior techniques.<ref>{{cite book |chapter=Object Removal by Exemplar-Based Inpainting |last=Criminisi |first=A. |last2=Perez |first2=P. |last3=Toyama |first3=K. |work=Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition |year=2003 |doi=10.1109/CVPR.2003.1211538 |title=2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings |volume=2 |pages=II-721-II-728 |isbn=0-7695-1900-8 }}</ref><ref>{{cite journal |title=Inpainting Strategies for Reconstruction of Missing Data in VHR Images |first=L. |last=Lorenzi |first2=F. |last2=Melgani |first3=G. |last3=Mercier |journal=IEEE Geoscience and Remote Sensing Letters |year=2011 |volume=8 |issue=5 |pages=914–918 |doi=10.1109/LGRS.2011.2141112 }}</ref>
+Exemplar-based image inpainting attempts to automate the clone tool process. It fills "holes" in the image by searching for similar patches in a nearby source region of the image, and copying the pixels from the most similar patch into the hole.  By performing the fill at the patch level as opposed to the pixel level, the algorithm reduces blurring artifacts caused by prior techniques.<ref>{{cite book |chapter=Object Removal by Exemplar-Based Inpainting |last1=Criminisi |first1=A. |last2=Perez |first2=P. |last3=Toyama |first3=K. |year=2003 |doi=10.1109/CVPR.2003.1211538 |title=2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings |volume=2 |pages=II-721–II-728 |isbn=0-7695-1900-8 |s2cid=315138 }}</ref><ref>{{cite journal |title=Inpainting Strategies for Reconstruction of Missing Data in VHR Images |first1=L. |last1=Lorenzi |first2=F. |last2=Melgani |first3=G. |last3=Mercier |journal=IEEE Geoscience and Remote Sensing Letters |year=2011 |volume=8 |issue=5 |pages=914–918 |doi=10.1109/LGRS.2011.2141112 |bibcode=2011IGRSL...8..914L |s2cid=2119517 }}</ref>
 ==See also==
+* [[Inpaint]]
 * [[Infrared cleaning]]
 * [[Noise reduction]]
@@ Line 105: / Line 87: @@
 * [[Conservation and restoration of parchment]]
 * [[Conservation and restoration of cultural heritage]]
+* [[Audio inpainting]]
 ==References==
-<references/>
+<references />
-Bugeau Aurelie & Bertalmio, Marcelo (2011) "Combining Texture Synthesis and Diffusion for Image Inpainting." Hal Archives. https://hal.archives-ouvertes.fr/hal-00551587/
+*Bugeau Aurelie & Bertalmio, Marcelo (2011) "Combining Texture Synthesis and Diffusion for Image Inpainting." Hal Archives. [https://hal.archives-ouvertes.fr/hal-00551587/ Combining Texture Synthesis and Diffusion for Image Inpainting.]
 == External links ==
 {{external links|date=April 2016}}
-*[https://link.springer.com/chapter/10.1007%2F11573425_13 Interactive Point-and-Click Segmentation for Object Removal in Digital Images], ICCV-HCI 2005
+* [https://link.springer.com/chapter/10.1007%2F11573425_13 Interactive Point-and-Click Segmentation for Object Removal in Digital Images], ICCV-HCI 2005
+* [https://web.archive.org/web/20120206062613/http://mountains.ece.umn.edu/~guille/inpainting.htm Image and Video Inpainting]  {{small|(webarchive)}} by [[Guillermo Sapiro]]
-*[http://www.siam.org/pdf/news/328.pdf Inpainting and the Fundamental Problem of Image Processing], [[Society for Industrial and Applied Mathematics|SIAM]]
+* Mathematica [http://reference.wolfram.com/mathematica/ref/Inpaint.html Inpaint] function.
-*[https://www.math.ucla.edu/~imagers/htmls/inp.html Image Inpainting], by the Image Processing Group at [[UCLA]].
-*[https://web.archive.org/web/20120206062613/http://mountains.ece.umn.edu/~guille/inpainting.htm Image and Video Inpainting]  {{small|(webarchive)}} by [[Guillermo Sapiro]]
+* [https://web.archive.org/web/20141127054103/https://lozes.users.greyc.fr/ressources/Lozes_TIP2336548.pdf Inpainting of images on meshes or point clouds], TIP 2014
+* [https://www.microsoft.com/en-us/research/wp-content/uploads/2017/01/structure_completion_small.pdf Image Completion using Planar Structure Guidance], SIGGRAPH 2014
-*Mathematica [http://reference.wolfram.com/mathematica/ref/Inpaint.html Inpaint] function.
-*[https://web.archive.org/web/20141127054103/https://lozes.users.greyc.fr/ressources/Lozes_TIP2336548.pdf Inpainting of images on meshes or point clouds], TIP 2014
-*[https://www.microsoft.com/en-us/research/wp-content/uploads/2017/01/structure_completion_small.pdf Image Completion using Planar Structure Guidance], SIGGRAPH 2014
 {{Cultural Conservation-Restoration |state=expanded}}
-[[Category:Conservation and restoration]]
+[[Category:Conservation and restoration of cultural heritage]]
 [[Category:Image processing]]

v t e Cultural heritage and historic preservation
Topics and issues	Agents of deterioration Archival processing Archaeological science Archaeology Archive Bioarchaeology Calendar (archives) Conservation and restoration of cultural property Conservation and restoration of immovable cultural property Conservation and restoration of movable cultural property Conservation science (cultural property) Collecting Collection (museum) Collection catalog Collections maintenance Collections management Collections management system Cultural heritage Cultural heritage management Cultural property Cultural property documentation Cultural property exhibition Cultural property imaging Cultural property storage Cultural resources management Database preservation Deaccessioning (museum) Digital library Digital photograph restoration Digital preservation Disaster preparedness (cultural property) Film preservation Finding aid Fonds Found in collection Heritage asset Heritage science Inherent vice Intangible cultural heritage Integrated pest management (cultural property) Inventory (library and archive) Inventory (museum) Media preservation Midden Mold control and prevention (library and archive) Museum Optical media preservation Preservation (library and archive) Preservation metadata Preservation survey Provenance Repatriation Ruins Sustainable preservation Treasure Web archiving
Roles and expertise	Archivist Art dealer Art handler Auctioneer Collection manager Conservator-restorer Conservation scientist Conservation technician Curator Exhibition designer Mount maker Objects conservator Paintings conservator Photograph conservator Preservationist Registrar (cultural property) Textile conservator
Methods and techniques	Aging (artwork) Anastylosis Arrested decay Cradling (paintings) Cultural property radiography Detachment of wall paintings Desmet method Display case Digital repository audit method based on risk assessment Historic paint analysis Inpainting Kintsugi Leafcasting Lining of paintings Mass deacidification Overpainting Paleo-inspiration Paper splitting Reconstruction (architecture) Rissverklebung Textile stabilization Transfer of panel paintings UVC-based preservation VisualAudio
Conservation and restoration of immovable cultural property by item type	Archaeological sites Frescos Heritage railways Historic gardens Outdoor artworks Outdoor bronze objects Outdoor murals
Conservation and restoration of movable cultural property by item type	Aircraft Ancient Greek pottery Bone, horn, and antler objects Books, manuscripts, documents and ephemera Ceramic objects Clocks Copper-based objects Feathers Film Flags and banners Fur objects Glass objects Herbaria Human remains Illuminated manuscripts Insect specimens Iron and steel objects Ivory objects Judaica Lacquerware Leather objects Lighthouses Metals Musical instruments Neon objects New media art Paintings Painting frames Panel paintings Papyrus Parchment Performance art Photographs Photographic plates Plastic objects Rail vehicles Road vehicles Shipwreck artifacts Silver objects South Asian household shrines Stained glass Taxidermy Textiles Tibetan thangkas Time-based media art Totem poles Vinyl discs Woodblock prints Wooden artifacts Wooden furniture
Intangible cultural heritage preservation	Ancient music Applied folklore Dance notation Early music Endangered language Ethnochoreology Ethnomusicology Ethnopoetics Family folklore Folklore Folk art Folk dance Folk etymology Folk instrument Folk medicine Folk music Folk process Folk play Foodways Folklore studies Heritage language Heritage language learning Indigenous intellectual property Indigenous culture Indigenous language Language death Language preservation Language revitalization Living history Oral history preservation Preservation of meaning Primitive music Tradition preservation Traditional knowledge
Notable projects	Conservation issues of Pompeii and Herculaneum Conservation-restoration of Ecce Homo by Elías García Martínez Conservation-restoration of The Gross Clinic by Thomas Eakins Conservation-restoration of Leonardo da Vinci's The Last Supper Pompeian frescoes Conservation-restoration of the Shroud of Turin Conservation-restoration of the Sistine Chapel frescoes Conservation-restoration of the Statue of Liberty Conservation-restoration of the H.L. Hunley Conservation response to flood of Arno, Florence Modern and Contemporary Art Research Initiative Preservation Metadata: Implementation Strategies