Integrating symbolic and subsymbolic AI for multimodal data fusion and enrichment of Building Digital Twins with material performance data

The aim of this research is to tackle the lack of reliable material information by enabling AI-based material information modelling (MIM) for Digital Twins (DT) and thereby improving design decision-making for building material related performance assessments. This research focuses on the development of reliable material data that adhere to the Findable, Accessible, Interoperable, and Reusable (FAIR) principles. FAIR material data is significant in supporting design decision-making, enabling comprehensive material-related performance assessments. Performance assessments include evaluating the reusability and recyclability of building products, along with key properties such as durability, thermal performance, and structural integrity.

The project leverages neuro-symbolic AI techniques to structure knowledge, predict new insights (data), and automate the semantic enrichment of Digital Twins (DTs) with material performance data. By integrating symbolic and sub-symbolic AI, the research seeks to enhance data sensitivity—ensuring responsiveness to dynamic data changes—and explainability, aligning with domain-specific knowledge structures. Key technologies employed include Building Information Modelling (BIM), semantic knowledge graphs, artificial neural networks, and natural language processing (NLP) methods.

Firstly, the system framework emphasises on structuring the material property and performance information along multi-scalar data models (i.e., material, element, building). The data prediction utilises neuro-symbolic AI techniques, including data prediction and knowledge graph inference to account for and tackle uncertainties related to the incompleteness of data. Semantically matching and enriching DTs with corresponding material information utilises the data structure from and the predicted material data to semantically map and fuse material information with the DT using NLP and large language models in combination with semantic knowledge graphs. Finally, an integration and validation of the developed system framework includes the testing of the system's capabilities with respect to material performance assessment against real-world demonstrator, all aimed at improving material data analysis and decision support in the context of the built environment.