ThreMoLIA: Threat Modeling of Large Language Model-Integrated Applications
2025 (English)In: Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering , EASE, 2025 edition, EASE 2025 / [ed] Babar M.A., Tosun A., Wagner S., Stray V., Association for Computing Machinery (ACM), 2025, p. 834-839Conference paper, Published paper (Refereed)
Abstract [en]
Large Language Models (LLMs) are currently being integrated into industrial software applications to help users perform more complex tasks in less time. However, these LLM-Integrated Applications (LIA) expand the attack surface and introduce new kinds of threats. Threat modeling is commonly used to identify these threats and suggest mitigations. However, it is a time-consuming practice that requires the involvement of a security practitioner. Our goals are to 1) provide a method for performing threat modeling for LIAs early in their lifecycle, (2) develop a threat modeling tool that integrates existing threat models, and (3) ensure high-quality threat modeling. To achieve the goals, we work in collaboration with our industry partner. Our proposed way of performing threat modeling will benefit industry by requiring fewer security experts' participation and reducing the time spent on this activity. Our proposed tool combines LLMs and Retrieval Augmented Generation (RAG) and uses sources such as existing threat models and application architecture repositories to continuously create and update threat models. We propose to evaluate the tool offline - i.e., using benchmarking - and online with practitioners in the field. We conducted an early evaluation using ChatGPT on a simple LIA and obtained results that encouraged us to proceed with our research efforts.
Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2025. p. 834-839
Keywords [en]
AI4SE, and SE4AI., LLM-integrated Applications, Secure Software Engineering, Threat Modeling, Artificial intelligence, Benchmarking, Human engineering, Information systems, Modeling languages, Complex task, Industrial software, Integrated applications, Language model, Large language model-integrated application, Software applications, Application programs
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-29120DOI: 10.1145/3756681.3757083ISI: 001668832700094Scopus ID: 2-s2.0-105026943348ISBN: 9798400713859 (print)OAI: oai:DiVA.org:bth-29120DiVA, id: diva2:2031466
Conference
29th International Conference on Evaluation and Assessment of Software Engineering, EASE 2025, Istanbul, June 17-20, 2025
Part of project
SERT- Software Engineering ReThought, Knowledge FoundationThreMoLIA - Threat Modeling for LLM-Integrated Applications
Funder
Knowledge Foundation, 20180010Vinnova, 2024-006592026-01-232026-01-232026-02-27Bibliographically approved