Italy has the largest material and immaterial cultural heritage in the world that remains partially hidden in repositories for different reasons, as cataloguing, accessibility, logistic difficulties or communication. These issues are limiting the cultural and economic potential of the artistic heritage.
Meanwhile, the contemporary technological scenario is characterised by the progress of Artificial Intelligence (AI), in particular Machine Learning (ML) and Big Data. These can enable new tools for working with cultural heritage, from visual analysis to understanding behaviours, from historic research to narrative production.
The project aims at the development of an Expert System to support the curatorial, managing and exhibition work, while exploring the potential use of ML for the improvement of user experience in exhibition spaces (museums and other venues), with a special attention on promoting artworks in museum repositories and the management of post Covid-19.
The project involves 6 academic disciplines (SSD) with an approach of Design Driven Innovation.
Beyond the design and technical competences, the involvement of art history, psychology and marketing will help to achieve an innovative and sustainable model for cultural heritage promotion.
ML will be used for three main purposes: (1) to better understand visitor behaviour and artwork perception in the museum space; (2) to improve the cataloguing of artworks based on visual and semantic characteristics; (3) to facilitate the curatorial and exhibition design activity, involving the stored artworks.
The project is carried out in collaboration with the Ministry MiBAC and Lazio Innova, along with the scope of the regional centre of excellence DTC, relevant for the regional application.
Moreover, experimentation involves three museums (Galleria Spada, Manzù, Boncompagni Ludovisi).
The project is a valuable learning opportunity for Design and other disciplines that still have a limited experience with ML.
The expected result of the project is an Expert System developed by AI and ML supporting the museum curatorial and management activities, so leading to innovative, scalable and replicable models for the promotion of artistic heritage, which is often neglected in the Lazio Region and the Italian system of museums.
According to the state of the art, the use of digital technologies in museums is expanding and their importance is crucial. In particular, the study of solutions for the visitors augmented museum experience is an established field, integrating the physical and virtual experience before, during and after the visit. Moreover, the latest post-COVID-19 pandemic occurrence is forcing to re-think the overall cultural museum experience with new curatorial and organizational patterns. Hence, the research project is fitting into an established and growing field.
In particular, the research project is including AI and ML to design a dedicated Expert System, collecting data and profiling the users' behaviours for the development and the validation of a new approach to cultural museum experience and management, so progressing in the curatorial and art features, the marketing and the management of culture, as well the psychology of the user experience and the exhibition design. An additional innovation lies in providing objective, scientifically accurate tools for evaluating and implementing the effectiveness of exhibition design and curatorial work. Supposing an inter-museum network, the Expert System will help the selection from a variety of artworks that no single curator can handle, so featuring novel narratives and unexpected combinations. Through the partnership with partner institutions, the project aims to offer an innovative model of 'museum that learns': a sort of thinking mind to memorize the inputs from visitors and curators, in a fluid, cooperative and mutual exchange of information and training.
The nature and the innovation of the project isn't inherently technological: thanks to the skills in the field of Data Analysis and ML applied to museum experience and management, and thanks to the cooperation with the partner institutions, the Expert System is a new tool for users profiling and guidelines for art curators and museums, while collecting and processing data with an unprecedented accuracy about audience behaviors, reactions to artworks, contexts - and to the relationship between works and the exhibition space - opening to a dynamic configuration of the museum itself, its collections, the spaces and the visiting paths. The results can be exploited by the museum institutions as well as by the audience, without neglecting the context of training, lifelong learning, tourism and edutainment.
So far, the use of ML in the arts sector developed applications in the management and the fruition of the archives, but out of the museum spaces. In this regard should be mentioned experiments as Google Art Palette, which enables to search within a wide database, starting from a palette of colors of each artwork, rather than its author or its title. Also, LIFE tags is the photo archive of LIFE Magazine collecting 4 millions images taken in the last seventy years, which can be scanned, analyzed and tagged to help searching, thanks to machine vision algorithms. Another to be mentioned is the experiment of MoMA in New York, which mapped pictures of installations since the museum opening in 1929 by ML, so making them available and allowing further details about each artwork.
The research project is applying the capacity of ML not only to map the scattered heritage of small museums (which is a distinctive feature in Italy), but also to map the behavioral experience and the gradient of appreciation of visitors, so developing an additional tool of interpretation and enabling the museum curators and/or directors to widen and diversify the offer from a user-experience and a knowledge economy perspective.
In particular, the research project is including an AI/ML tool for analysing visitor behaviours, which is featuring two innovations with respect to the current state of the art. First, based on human body biometric representations (e.g., skeleton bones and joints), the temporal analysis of body emotions to study sentiment variations of visitors during a tour. Second, based on head and body attention mechanisms, the identification of interesting artworks for visitors. These procedures allow us to have dynamic maps of visitor sentiments related to specific artworks, thus providing an overview of the emotional impact of the environment in terms of sentiment and interest. The information, among other things, can be exploited to simulate alternative artworks settings including the path used to reach them, consequently enhancing the visitor experience.
The innovative results benefitting the different stakeholders can be found in the chapter about 'Other participants and role in the project'.