Evaluation of Large Language Models on hierarchical entity matching for cultural heritage metadata

Data organization is a key challenge for many cultural heritage institutions. The size of current digital collections and the heterogeneity of metadata creates problems for interoperability, reliability and information retrieval. This work-in-progress paper aims to provide the groundwork for research into effectively and transparently performing automatic hierarchical entity matching within library metadata. We explore to what extent LLMs can detect whether an entity relationship for a pair of records (book descriptions) exists, based on their bibliographic metadata. We focus solely on the edge cases: those pairs of records that are potentially difficult to label with the correct entity relationship. We compare several different LLMs; we study the effect of the amount of detail in the metadata description and we also look at zero-shot versus few-shot prompting. Our results show that LLMs are a very promising technique for parts of this task of hierarchical entity matching of metadata book descriptions.