Sensing the Cultural Significance with AI for Social Inclusion: Computational Spatiotemporal Network-based Framework of Heritage Knowledge Documentation using User-Generated Content

This dissertation entitles “Sensing the Cultural Significance with AI for Social Inclusion: A Computational Spatiotemporal Network-based Framework of Heritage Knowledge Documentation using User-Generated Content”.

Core Premises

Social Inclusion has been growing as a goal in heritage management in the past decade. Whereas the 2011 UNESCO Recommendation on the Historic Urban Landscape (HUL) called for tools of knowledge documentation, social media already function as a resourceful platform for online communities to actively involve themselves in heritage-related discussions. Such discussions happen both in the baseline scenarios when people calmly share their experience of the cities they live in

or travel to, and in the activated scenarios when radical events trigger their emotions. Analyses have been recently performed on User-Generated Content (UGC) from social media platforms to actively collect opinions of the [online] public and to map cultural significance conveyed by various stakeholders in urban environments. Machine learning, deep learning, or more generally, Artificial Intelligence (AI) is shown to be indispensable for organizing, processing, and analysing the unstructured multi-modal massive data from social media efficiently andsystematically.

Research Aim

The aim of this research is to explore the use of AI in a methodological framework to include the contribution of a larger and more diverse group of participants and facilitate the knowledge documentation of cultural significance in cities with user-generated social media data. To reach the aim, five parts are used to elaborate the exploration process of the proposed methodological framework. PART A builds up a theoretical BASIS for the dissertation with a general introduction in Chapter 1 and a systematic literature review in Chapter 2. PART B develops the MODELLING process using AI to construct a machine replica of the authoritative view on cultural significance through UNESCO World Heritage Statements of Outstanding Universal Value in Chapter 3. Then the dissertation goes into two directions in PART C and PART D, respectively exploring the two variants of the methodological framework for knowledge documentation. PART C focuses on the CONTEXT of collective opinions in the everyday baseline scenarios with the data collection workflow in Chapter 4 and the mapping process of perceived cultural significance in Chapter 5. PART D focuses on the DYNAMICS of the discussions triggered by radical events by inspecting the spatiotemporal patterns of the content (especially emotions and proposed actions) and intensity of posting behaviours in Chapter 6. PART E concludes the dissertation.

Research Aim

The aim of this research is to explore the use of AI in a methodological framework to include the contribution of a larger and more diverse group of participants and facilitate the knowledge documentation of cultural significance in cities with user-generated social media data. To reach the aim, five parts are used to elaborate the exploration process of the proposed methodological framework. PART A builds up a theoretical BASIS for the dissertation with a general introduction in Chapter 1 and a systematic literature review in Chapter 2. PART B develops the MODELLING process using AI to construct a machine replica of the authoritative view on cultural significance through UNESCO World Heritage Statements of Outstanding Universal Value in Chapter 3. Then the dissertation goes into two directions in PART C and PART D, respectively exploring the two variants of the methodological framework for knowledge documentation. PART C focuses on the CONTEXT of collective opinions in the everyday baseline scenarios with the data collection workflow in Chapter 4 and the mapping process of perceived cultural significance in Chapter 5. PART D focuses on the DYNAMICS of the discussions triggered by radical events by inspecting the spatiotemporal patterns of the content (especially emotions and proposed actions) and intensity of posting behaviours in Chapter 6. PART E concludes the dissertation.

Research Aim

The aim of this research is to explore the use of AI in a methodological framework to include the contribution of a larger and more diverse group of participants and facilitate the knowledge documentation of cultural significance in cities with user-generated social media data. To reach the aim, five parts are used to elaborate the exploration process of the proposed methodological framework. PART A builds up a theoretical BASIS for the dissertation with a general introduction in Chapter 1 and

a systematic literature review in Chapter 2. PART B develops the MODELLING process using AI to construct a machine replica of the authoritative view on cultural significance through UNESCO World Heritage Statements of Outstanding Universal Value in Chapter 3. Then the dissertation goes into two directions in PART C and PART D, respectively exploring the two variants of the methodological framework for knowledge documentation. PART C focuses on the CONTEXT of collective opinions in the everyday baseline scenarios with the data collection workflow in Chapter 4 and the mapping process of perceived cultural significance in Chapter 5. PART D focuses on the DYNAMICS of the discussions triggered by radical events by inspecting the

spatiotemporal patterns of the content (especially emotions and proposed actions) and intensity of posting behaviours in Chapter 6. PART E concludes the dissertation in Chapter 7 and discusses on how the proposed methodological framework and the empirical findings can contribute to social INCLUSION in heritage management.

Methods Applied

It is an interdisciplinary study integrating the methods and knowledge from the broad fields of heritage studies, computer science, social sciences, network science, and spatial analysis. State-of-the-art methods from the AI communities were applied, nurtured, and tested within the research. The whole bundle of AI-based methods include ideas and models from Natural Language Processing, Computer Vision, Graph Neural Networks, Semi-Supervised Classification, Multi-modal Machine Learning, Topic Modelling, etc. Datasets of the UGC on social media platforms are collected and structured as networks/graphs, representing the spatial, temporal, and social connections among the posts. AI-based models are employed to help analyse the massive information content to derive the knowledge concerning cultural significance perceived and expressed by the online community in case study cities Venice, Paris, Suzhou, Amsterdam, and Rome. The results are further analysed and visualized with [spatial] statistics and mapping techniques as knowledge documentation.

Main Findings

Cultural significance perceived and conveyed by the online community to the cities was found to be strongly embedded in their spatiotemporal and social contexts. Tobler’s First Law of Geography was still shown as relevant for urban heritage on social media. In the baseline scenarios, cultural significance has been perceived and expressed by social media users at a broad variety of locations in cities with urban areas inscribed in the UNESCO World Heritage List, other than the conventional tourist destinations. In the activated scenarios, the triggered discussions reached places far beyond geographical boundaries during the event, forming a temporary global heritage community, where people mainly shared information about the event, expressed their emotions, and proposed or broadcast actions on how to help. Therefore, the AI-based methodological framework is shown to be able to collect information and map the knowledge of the community about the cultural significance of the cities, fulfilling the expectation and requirement of HUL, useful and informative for future socially inclusive heritage management processes.

Limitations and Drawbacks

The use of AI and social media data is never the “eternal solution” for mapping cultural significance, which could potentially create new challenges and opportunities compared to what it managed to solve. The AI models are always biased based on the available data and training methods, which can fall into sub-optimal solutions. Besides data privacy and ethical issues that need to be considered, the use of specific social media platforms as the data source implies that the people being included have been pre-defined, which may also have strong limitations to getting a comprehensive picture that may eventually result in systematic biases. The AI-based approach, therefore, needs to be accompanied by other sorts of qualitative and quantitative studies involving broader stakeholders. Nevertheless, this research makes the first steps to bridging the gaps towards collaborations between AI and heritage experts.