If you are using our Multi-Modal ODP in your work, please consider citing the arXiv version of the paper:
@misc{apriceno2024patternalignallintegrating,
title={A Pattern to Align Them All: Integrating Different Modalities to Define Multi-Modal Entities},
author={Gianluca Apriceno and Valentina Tamma and Tania Bailoni and Jacopo de Berardinis and Mauro Dragoni},
year={2024},
eprint={2410.13803},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2410.13803},
}
The ability to reason with and integrate different sensory inputs is the foundation underpinning human intelligence and it is the reason for the growing interest in modelling multi-modal information within Knowledge Graphs. Multi-Modal Knowledge Graphs extend traditional Knowledge Graphs by assciating an entity with its possible modal representations, including text, images, audio, and videos, all of which are used to convey the semantics of the entity. Despite the increasing attention that Multi-Modal Knowledge Graphs have received, there is a lack of consensus about the definitions and modelling of modalities, whose definition is often determined by application domains.
We propose a Multi-Modal Ontology Design Pattern (ODP) that provides a domain-agnostic structural backbone for multi-modal representation by separating (i) a multi- modal entity as an information object, (ii) its concrete digital realisations as modal descriptors, and (iii) reusable modality specifications, including support for modality composition.
The ODP's goal is to represent information entities with their associated modalities of diverse nature, and the corresponding relationships between them while providing flexibility and extensibility.
Have you developed a Knowledge Graph using the Multi-Modal ODP pattern? Contribute your project to our repository and share it with the community!
Add Your KGThis work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
