Experience with Large Language Model Applications for Information Retrieval from Enterprise Proprietary Data
2025 (English)In: Product-Focused Software Process Improvement / [ed] Dietmar Pfahl, Javier Gonzalez Huerta, Jil Klünder, Hina Anwar, Springer, 2025, Vol. 15452, p. 92-107Conference paper, Published paper (Refereed)
Abstract [en]
Large Language Models (LLMs) offer promising capabilities for information retrieval and processing. However, the LLM deployment for querying proprietary enterprise data poses unique challenges, particularly for companies with strict data security policies. This study shares our experience in setting up a secure LLM environment within a FinTech company and utilizing it for enterprise information retrieval while adhering to data privacy protocols.
We conducted three workshops and 30 interviews with industrial engineers to gather data and requirements. The interviews further enriched the insights collected from the workshops. We report the steps to deploy an LLM solution in an industrial sandboxed environment and lessons learned from the experience. These lessons contain LLM configuration (e.g., chunk_size and top_k settings), local document ingestion, and evaluating LLM outputs.
Our lessons learned serve as a practical guide for practitioners seeking to use private data with LLMs to achieve better usability, improve user experiences, or explore new business opportunities. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
Place, publisher, year, edition, pages
Springer, 2025. Vol. 15452, p. 92-107
Series
Lecture Notes in Computer Science (LNCS), ISSN 0302-9743, E-ISSN 1611-3349 ; 15452
Keywords [en]
AI, Artificial intelligence, Data security, Information retrieval, Large Language Model, LLM, Sandbox environment, Data privacy, Fintech, Enterprise data, Language model, Model application, Modeling environments, Privacy protocols, Security policy, Structured Query Language
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:bth-27326DOI: 10.1007/978-3-031-78386-9_7Scopus ID: 2-s2.0-85211960724ISBN: 9783031783852 (print)OAI: oai:DiVA.org:bth-27326DiVA, id: diva2:1923603
Conference
25th International Conference on Product-Focused Software Process Improvement, PROFES 2024, Tartu, Dec 2-4, 2024
Part of project
SERT- Software Engineering ReThought, Knowledge Foundation
Funder
Knowledge Foundation, 201800102024-12-282024-12-282024-12-28Bibliographically approved