Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Code Generation from Large API Specifications with Open Large Language Models: Increasing Relevance of Code Output in Initial Autonomic Code Generation from Large API Specifications with Open Large Language Models
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
Blekinge Institute of Technology, Faculty of Computing, Department of Software Engineering.
2024 (English)Independent thesis Basic level (degree of Bachelor), 180 HE creditsStudent thesis
Abstract [en]

Background. In software systems defined by extensive API specifications, auto- nomic code generation can streamline the coding process by replacing repetitive, manual tasks such as creating REST API endpoints. The use of large language models (LLMs) for generating source code comprehensively on the first try requires refined prompting strategies to ensure output relevancy, a challenge that grows as API specifications become larger. 

Objectives. This study aims to develop and validate a prompting orchestration solution for LLMs that generates more relevant, non-duplicated code compared to a single comprehensive prompt, without refactoring previous code. Additionally, the study evaluates the practical value of the generated code for developers at Ericsson familiar with the target application that uses the same API specification.

Methods. Employing a prototyping approach, we develop a solution that produces more relevant, non-duplicated code compared to a single prompt with local-hosted LLMs for the target API at Ericsson. We perform a controlled experiment running the developed solution and a single prompt to collect the outputs. Using the results, we conduct interviews with Ericsson developers about the value of the AI-generated code. 

Results. The study identified a prompting orchestration method that generated 427 relevant lines of code (LOC) on average in the best-case scenario compared to 66 LOC with a single comprehensive prompt. Additionally, 66% of the developers interviewed preferred using the AI-generated code as a starting point over starting from scratch when developing applications for Ericsson, and 66% preferred starting from the AI-generated code over code generated from the same API specification via Swagger CodeGen. 

Conclusions. Increasing the extent locally hosted LLMs can generate relevant code from large API specifications without refactoring the generated code in comparison to a single comprehensive prompt is possible with the right prompting orchestration method. The value of the generated code is that it can currently be used as a good starting point for further software development. 

Place, publisher, year, edition, pages
2024. , p. 37
Keywords [en]
Large Language Models, Autonomic Code Generation, API Specifica- tions, Artificial Intelligence, Prompt Engineering
National Category
Other Engineering and Technologies
Identifiers
URN: urn:nbn:se:bth-26582OAI: oai:DiVA.org:bth-26582DiVA, id: diva2:1877570
External cooperation
Ericsson
Subject / course
PA1445 Kandidatkurs i Programvaruteknik; PA1445 Kandidatkurs i Programvaruteknik
Educational program
PAGPT Software Engineering; PAGWE Web Programming
Supervisors
Examiners
Available from: 2024-07-01 Created: 2024-06-25 Last updated: 2025-02-10Bibliographically approved

Open Access in DiVA

fulltext(797 kB)265 downloads
File information
File name FULLTEXT01.pdfFile size 797 kBChecksum SHA-512
1b7c1a16edd397d24e83bf86989e6b5ade6156b7ee33b9537cde2cfb9b568d966eba6682c20a46f5cf8487d5c1cc2df5a25c343bdd24658e651ba0c51bb694ad
Type fulltextMimetype application/pdf

By organisation
Department of Software Engineering
Other Engineering and Technologies

Search outside of DiVA

GoogleGoogle Scholar
Total: 265 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 1401 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf