Revolutionizing Heritage: AI Transforms 135-Year-Old Photo into 3D Model

In a groundbreaking development for cultural preservation, a team of computer scientists has successfully employed artificial intelligence to reconstruct lost relief panels from the renowned Borobudur temple in Indonesia, a UNESCO World Heritage Site. This innovative approach utilizes a neural network capable of transforming a single two-dimensional image into a detailed three-dimensional model, effectively creating a modern stereoscope for the digital era. This remarkable achievement was presented at last month’s ACM Multimedia conference, showcasing how advanced technology can help revive lost cultural heritage.

The Borobudur temple is home to the largest collection of Buddhist reliefs in the world, with a staggering 2,672 bas reliefs adorning its walls. However, in the late 19th century, a portion of the temple was covered behind stone encasements during restoration work, hiding 156 of these intricate reliefs from view. Before their concealment, grayscale photographs of each relief were taken, allowing researchers to utilize these historical images for their AI model. Remarkably, the team was able to reconstruct one hidden relief using a black-and-white photograph taken 134 years ago.

Previous efforts to recreate these reliefs fell short due to a loss of intricate details caused by depth compression in the images. Specifically, the three-dimensional features of the reliefs, reflecting variances in depth, were flattened in earlier models. The researchers identified this problem, referring to the lack of detail as “soft edges.” In their study, they proposed a new mapping technique which captured these nuances, making it easier for their neural network to accurately estimate depth.

Satoshi Tanaka, a researcher at Ritsumeikan University and co-author of the study, explained, “Although we achieved 95% reconstruction accuracy, finer details such as human faces and decorations were still missing due to high compression of depth values in 2D relief images. Our new method enhances depth estimation, particularly along soft edges, using a novel edge-detection approach.” The results showcased an impressive edge map and semantic map related to the sample relief, revealing the model’s capacity to differentiate between various components in the image.

Significantly, this AI-driven technique also highlights the model’s ability to incorporate multi-modal understanding, where it processes multiple data channels to decipher the target object. In this case, the soft-edge detector utilized not only slight brightness variations but also the actual curves of the carvings. By intertwining these layers of information, the team achieved a sharper and more precise reconstruction than prior attempts.

Tanaka emphasized the potential of this technology, stating, “Our technology holds vast potential for preserving and sharing cultural heritage. It opens new opportunities not only for archaeologists but also for immersive virtual experiences through VR and metaverse technologies, preserving global heritage for future generations.”

The importance of cultural heritage preservation cannot be overstated, particularly as some artifacts face significant threats of deterioration or destruction. While AI-generated models may never fully replicate the original works, they hold profound promise for resurrecting lost heritage, even for artifacts like the Bamiyan Buddhas—iconic statues destroyed by the Taliban in 2001.

Moreover, this technology can also be pivotal for safeguarding heritage at imminent risk, such as ancient Aboriginal carvings located in Australia’s Tanami Desert. Cultural heritage plays an essential role in defining communities and identities over time, and the successful application of AI models could enable art historians and preservationists to save irreplaceable pieces of history. Nevertheless, it’s essential to remain aware of the environmental impact associated with AI technologies, which can consume vast amounts of energy.

Despite ongoing concerns about AI deployment, utilizing this technology for noble purposes—especially in the realm of anthropology and heritage conservation—suggests that there are ways to harness its potential for good. As the intersection of technology and history continues to evolve, the strides made by these researchers represent a hopeful step towards preserving our shared cultural legacy for future generations.