From Semantic to Thematic Analysis: Extracting Research Trends and Topics from Text Data

Abstract

Topic modelling and extraction is not a new field in synthesis-research, but with the current advancements in LLMs and Transformer-based models, such processes can as well be advanced with better accuracy and efficiency. In this paper, we explore the application of Transformer based methods, firstly to validate the domain experts based manual identification of topics and secondly, to create a benchmark method for machine-based extraction of Topics and Themes related to sustainable urban development. The research uses a corpus of documents, including Project proposals and Project profiles, from ten projects in the field of Sustainable Development. The process builds upon established methodology whereby the tokenized text is used to create embeddings with BERT (Bidirectional Encoder Representations from Transformers) to create a vector space. As part of the new proposed architecture, the embeddings are then clustered and labelled according to their cosine similarity with domain expert themes, which allows a comprehensive view of themes/topics and sub-topics.