Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Performance benchmarks of lip-syncscripting in Maya using speechrecognition: Gender bias and speech recognition
Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science.
2022 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesis
Abstract [en]

Background: Automated lip sync is used in animation to make facial animations with a minimal interception from an animator. A lip-syncing script for Maya has been written in Python using the Vosk API to transcribe voice lines from audio files into instructions in Maya to automate the pipeline for speech animations. Previous studies have mentioned that some voice transcription and voice recognition API's have had a gender bias that does not read female voices as efficiently as male voices. Does gender affect this lip-syncing script's performance in creating animations?

Objectives: Benchmark the performance of a lip-syncing script that uses voice transcription by looking for a gender bias in a voice transcription API by comparing male and female voices as input. If there is a gender bias, how much does it affect the produced animations?

Methods: Evaluating the script's perceived performance by conducting a user study through a questionnaire. The Participants evaluate different animation attributes to build an image of a potentially perceived gender bias in the script. Analyzing the transcribed voice lines for an objective view of a possible gender bias.

Results: The transcribed voice lines were almost perfect on both male and female vocal lines, with just one transcription error for one word in one of the male voiced lines. The male and female voiced lines received very similar grading on their voice lines when analyzing the data from the questionnaire. On average, the male voice lines seemed to get a higher rating on most voice lines in the different criteria, but the score difference was minimal.

Conclusions: There is no gender bias in the lip syncing script. The accuracy experiment had a very similar accuracy rate between the male and female vocal lines. The female-voiced lines received a slightly higher accuracy than the male voice lines with the difference in one word. The male voice lines received a slightly higher score on the perceived scores through the questionnaire. The males had a higher score because of other factors than a possible gender bias.

Place, publisher, year, edition, pages
2022. , p. 44
Keywords [en]
Lip-syncing, speech recognition, Animation, Maya, Python
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:bth-23214OAI: oai:DiVA.org:bth-23214DiVA, id: diva2:1672076
Subject / course
UD1449 Bachelor´s Thesis in Digital Game Development
Educational program
UDGTA Technical artist for games
Presentation
2022-05-23, J1630, Valhallavägen 1, 371 41 Karlskrona, Karlskrona, 13:00 (English)
Supervisors
Examiners
Available from: 2022-06-22 Created: 2022-06-18 Last updated: 2022-06-22Bibliographically approved

Open Access in DiVA

Performance benchmarks of lip-sync scripting in Maya using speech recognition Gender bias and speech recognition(2255 kB)711 downloads
File information
File name FULLTEXT02.pdfFile size 2255 kBChecksum SHA-512
f47b17caebbb814f52eea305ed34fdbb65ffd1ab196b4b92aa49e6fed473e99a0df20f68399ad1a52d0d88e274618ba3c60743753ae5549d97fb9edb9fa45657
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Björkholm, Adrian
By organisation
Department of Computer Science
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 711 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 840 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf