A Microservices Architecture based on a Deep-learning Approach for an Innovative Fruition of Art and Cultural Heritage

Abstract

Technological innovations have resulted in a digital transformation in a variety of fields, including culture and tourism. We propose an innovative and personalized solution to benefit art and cultural heritage in indoor and outdoor environments by combining Internet of Things-enabled technologies and deep learning-based approaches. A recent Convolutional Neural Network (CNN) architecture to jointly perform local feature detection and description has been adapted and exploited for the first time for image matching in the cultural heritage application context. The performance validation of the proposed system shows that the proposed modular architecture ensures a very low error rate and excellent response time up to 2000 user visits in 700 seconds. The validation of the computer vision module shows as the proposed CNN based feature extraction approach improves image matching performance, especially in poorly textured object areas reaching a F1-Score of 0.9907 (against the 0.9679 obtained by traditional gradient based approaches) on the challenging dataset of images taken from 4 different historical sites and a F1-Score of 0.9807 (against the 0.9798 obtained by traditional approaches) on a public benchmark dataset of artworks.

Keywords