System disruptions
There are recurring service disruptions on some of the search portals due to high traffic. We are working to resolve the issue, you may temporarily encounter an error message.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Different Hallucinations calls for Different Solutions - A Categorisation of LLM Transcription Mistakes
Linnaeus University.
Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.ORCID iD: 0000-0001-9947-1088
2025 (English)In: CEUR Workshop Proceedings / [ed] Nowaczyk S., Vettoruzzo A., Technical University of Aachen , 2025, Vol. 4037, p. 39-52Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents a contribution to better interpretation of the results we get from GenAI models, more specifically, better interpretation of the mistakes that they make. We have conducted an analysis of 644 (from GPT-4o) + 4858 (from ARIA) mistakes made by two models on a key-value extraction task, and found that they may be categorised into three mutually exclusive groups. These groups are; i problems identifying the requested information, p problems presenting the correct information, and s skewed training data. These categories could be used to indicate which action a user could take to reduce the number of mistakes. Further, we have found a strong correlation between the suggested categories and the Ratcliff/Obershelp pattern recognition score between the generated result and the expected result; all faulty results containing minor mistakes are more than 60% similar to the expected result. Only mistakes based on lack of identifying what was requested had less than 60% similarity to the expected result. 

Place, publisher, year, edition, pages
Technical University of Aachen , 2025. Vol. 4037, p. 39-52
Series
CEUR Workshop Proceedings, E-ISSN 1613-0073
Keywords [en]
Document analysis, Generative AI, LLM, Verification, Artificial intelligence, Errors, Human engineering, Information retrieval systems, Pattern recognition, Documents analysis, Key values, Strong correlation, Training data
National Category
Artificial Intelligence
Identifiers
URN: urn:nbn:se:bth-28782Scopus ID: 2-s2.0-105017736795OAI: oai:DiVA.org:bth-28782DiVA, id: diva2:2007260
Conference
2025 Swedish AI Society Workshop, SAIS 2025, Halmstad, June 16-17, 2925
Available from: 2025-10-17 Created: 2025-10-17 Last updated: 2025-10-17Bibliographically approved

Open Access in DiVA

fulltext(274 kB)75 downloads
File information
File name FULLTEXT01.pdfFile size 274 kBChecksum SHA-512
471273bf931335c0d5d3a1cb0d7afc3766ff32258cd1ca71d97f90b0e40b46292a9524739b967004a8d0648b6a87c55b3cd572c6511bd2d7e67c1998688b973a
Type fulltextMimetype application/pdf

Scopus

Authority records

Grahn, Håkan

Search in DiVA

By author/editor
Grahn, Håkan
By organisation
Department of Computer Science
Artificial Intelligence

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 2500 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf