Change search
ExportLink to record
Permanent link

Direct link
BETA

Project

Project type/Form of grant
Grant to research environment
Title [sv]
HINTS – Intelligenta verkligheter med människan i centrum
Title [en]
HINTS - Human-Centered Intelligent Realities
Abstract [sv]
HINTS syftar till att vara den främsta svenska noden med hög inverkan internationellt inom intelligenta verkligheter med människan i centrum för nästa generations digitala samhällen. HINTS-projektet är mitt i BTH:s strategi mot digitalisering och det ligger i linje med BTH:s strategi att bygga fokuserade och kompletta miljöer baserade på starka akademiska program, forskningsexpertis och samproduktion med externa partners.
Abstract [en]
The overall objective of the HINTS project is to develop concepts, principles, methods, algorithms, and tools for human-centered intelligent realities, in co-production with industrial partners and society, in order to lead the way for future immersive, user-aware, and smart interactive digital environments.
Publications (10 of 54) Show all publications
Sarwatt, D. S., Kulwa, F., Philipo, A. G., Runyoro, A.-A. K., Ning, H. & Ding, J. (2026). Aigc-driven human-machine intelligence in ITS: technologies, applications, evaluation framework, challenges, and future directions. Artificial Intelligence Review, 59(2), Article ID 75.
Open this publication in new window or tab >>Aigc-driven human-machine intelligence in ITS: technologies, applications, evaluation framework, challenges, and future directions
Show others...
2026 (English)In: Artificial Intelligence Review, ISSN 0269-2821, E-ISSN 1573-7462, Vol. 59, no 2, article id 75Article in journal (Refereed) Published
Abstract [en]

This paper explores the integration of Artificial Intelligence Generated Content (AIGC), a rapidly evolving branch of generative AI, with Human-Machine intelligence (HMI) to enhance the functionality of Intelligent Transportation Systems (ITS). As transportation systems grow increasingly complex, adaptive decision-making becomes essential for interpreting vast streams of real-time data from vehicles, infrastructure, and users. AIGC plays a transformative role in optimizing traffic flow through dynamic routing and real-time traffic management, while human intelligence ensures these systems remain responsive to evolving real-world conditions. For safety, AIGC is used to simulate complex driving scenarios for autonomous vehicle training and detect traffic anomalies, with human oversight providing contextual decisions in ambiguous situations. For sustainability, AIGC supports data-driven strategies to reduce emissions and energy use, while human expertise ensures alignment with ethical and environmental goals. This synergy enhances real-time decision-making, improving both accuracy and adaptability across ITS scenarios. The paper presents a comprehensive review of core and supporting AIGC technologies and their applications across key ITS domains. Case studies and initiatives from industry leaders demonstrate practical implementations of AIGC-driven HMI collaboration. To guide future deployments, we propose a conceptual five-layer evaluation framework for assessing AIGC-HMI systems, encompassing functional performance, human interaction, explainability, ethical compliance, and robustness. We also address challenges such as legacy system integration, data privacy, model bias, and scalability. The paper concludes by outlining future research directions, emphasizing the need for scalable, interpretable, and ethically aligned AIGC models. This work contributes to the development of intelligent, adaptive, and trustworthy transportation systems. 

Place, publisher, year, edition, pages
Springer Nature, 2026
Keywords
Artificial intelligence, Artificial intelligence generated content, Generative artificial intelligence, Human-machine intelligence, Intelligent transportation systems, Behavioral research, Data privacy, Decision making, Ethical aspects, Highway traffic control, Intelligent vehicle highway systems, Legacy systems, Man machine systems, Motor transportation, Real time systems, Evaluation framework, Human-machine, Machine intelligence, Technology application, Transportation system, Transportation system technology
National Category
Artificial Intelligence Transport Systems and Logistics Human Computer Interaction
Identifiers
urn:nbn:se:bth-29129 (URN)10.1007/s10462-025-11467-5 (DOI)001666005700002 ()2-s2.0-105027936552 (Scopus ID)
Funder
Knowledge Foundation, 20220068Vinnova, 2022-01768
Available from: 2026-01-30 Created: 2026-01-30 Last updated: 2026-02-25Bibliographically approved
Philipo, A. G., Sebastian Sarawatt, D., Ding, J., Daneshmand, M. & Ning, H. (2026). Cyberbullying Detection: Exploring Datasets, Technologies, and Approaches on Social Media Platforms. ACM Computing Surveys, 58(7), Article ID 186.
Open this publication in new window or tab >>Cyberbullying Detection: Exploring Datasets, Technologies, and Approaches on Social Media Platforms
Show others...
2026 (English)In: ACM Computing Surveys, ISSN 0360-0300, E-ISSN 1557-7341, Vol. 58, no 7, article id 186Article, review/survey (Refereed) Published
Abstract [en]

Cyberbullying has become a major challenge in the digital era, and many people, especially adolescents, use social media platforms to communicate and share information. Some exploit these platforms to embarrass others through messages, e-mails, speech, and public posts, causing severe psychological harm to victims. This study reviews existing research on technologies, approaches, datasets, and evaluation metrics for cyberbullying detection, while highlighting future directions and key challenges. The findings show that traditional models work reasonably well with small datasets but require constant updates; machine learning models face feature extraction and linguistic limitations; deep learning models perform better but lack multilingual and cross-lingual capabilities; and large language models (LLMs) achieve the highest performance, offering flexibility and rich linguistic features but face issues of high-energy use and real-time applicability. Addressing technological, methodological, dataset, and linguistic challenges will improve cyberbullying detection, helping to protect online communication and promote social responsibility. 

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2026
Keywords
datasets, detection approaches, Instances of cyberbullying, social media platforms, Computational linguistics, Human engineering, Large datasets, Learning systems, Social aspects, Social networking (online), Social sciences computing, Cyber bullying, Dataset, Detection approach, Digital era, Evaluation metrics, Instance of cyberbullying, Machine learning models, Small data set, Traditional models, Deep learning
National Category
Computer Sciences Natural Language Processing
Identifiers
urn:nbn:se:bth-29226 (URN)10.1145/3785654 (DOI)2-s2.0-105030926995 (Scopus ID)
Funder
Vinnova, 2022-01768Knowledge Foundation, 20220068
Available from: 2026-03-09 Created: 2026-03-09 Last updated: 2026-03-09Bibliographically approved
Angelova, M., Boeva, V., Abghari, S., Ickin, S. & Lan, X. (2026). FedCluLearn: Federated Continual Learning Using Stream Micro-cluster Indexing Scheme. In: Ribeiro R.P., Jorge A.M., Soares C., Gama J., Pfahringer B., Japkowicz N., Larrañaga P., Abreu P.H. (Ed.), Machine Learning and Knowledge Discovery in Databases. Research Track: . Paper presented at European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2025, Porto, Sept 15-19, 2025 (pp. 331-349). Springer Science+Business Media B.V.
Open this publication in new window or tab >>FedCluLearn: Federated Continual Learning Using Stream Micro-cluster Indexing Scheme
Show others...
2026 (English)In: Machine Learning and Knowledge Discovery in Databases. Research Track / [ed] Ribeiro R.P., Jorge A.M., Soares C., Gama J., Pfahringer B., Japkowicz N., Larrañaga P., Abreu P.H., Springer Science+Business Media B.V., 2026, p. 331-349Conference paper, Published paper (Refereed)
Abstract [en]

Artificial Neural Networks (NNs) are unable to learn tasks continually using a single model, which leads to forgetting old knowledge, known as catastrophic forgetting. This is one of the shortcomings that usually plague intelligent systems based on NN models. Federated Learning (FL) is a decentralized approach to training machine learning models on multiple local clients without exchanging raw data. A paradigm that handles model learning in both settings, federated and continual, is known as Federated Continual Learning (FCL). In this work, we propose a novel FCL algorithm, called FedCluLearn, which uses a stream micro-cluster indexing scheme to deal with catastrophic forgetting. FedCluLearn interprets the federated training process as a stream clustering scenario. It stores statistics, similar to micro-clusters in stream clustering algorithms, about the learned concepts at the server and updates them at each training round to reflect the current local updates of the clients. FedCluLearn uses only active concepts in each training round to build the global model, meaning it temporarily forgets the knowledge that is not relevant to the current situation. In addition, the proposed algorithm is flexible in that it can consider the age of local updates to reflect the greater importance of more recent data. The proposed FCL approach has been benchmarked against three baseline algorithms by evaluating its performance in several control and real-world data experiments. The implementation of FedCluLearn and the experimental results are available at https://github.com/milenaangelova1/FedCluLearn.

Place, publisher, year, edition, pages
Springer Science+Business Media B.V., 2026
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349
Keywords
Catastrophic forgetting, Concept drift, Data stream clustering, Federated continual learning, Time series data, Cluster analysis, Cluster computing, Intelligent systems, Learning systems, Neural networks, Concept drifts, Continual learning, Indexing scheme, Micro-clusters, Neural-networks, Stream clustering, Time-series data, Clustering algorithms
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-28834 (URN)10.1007/978-3-032-05981-9_20 (DOI)2-s2.0-105019303763 (Scopus ID)9783032059802 (ISBN)
Conference
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2025, Porto, Sept 15-19, 2025
Funder
Knowledge Foundation, 20220068
Available from: 2025-11-03 Created: 2025-11-03 Last updated: 2025-11-04Bibliographically approved
Wang, C., Garro, V., Sundstedt, V., Hu, Y. & Goswami, P. (2026). Immersive Analytics Meets Artificial Intelligence: A Systematic Review. Computational Visual Media, 12(1), 1-34
Open this publication in new window or tab >>Immersive Analytics Meets Artificial Intelligence: A Systematic Review
Show others...
2026 (English)In: Computational Visual Media, ISSN 2096-0433, E-ISSN 2096-0662, Vol. 12, no 1, p. 1-34Article, review/survey (Refereed) Published
Abstract [en]

Integrating artificial intelligence (AI) with immersive analytics (IA) represents a promising means of leveraging advanced computational techniques to enhance data visualization and analysis. This study examines the state-of-the-art of AI-IA integration by addressing three key research issues: the significant application domains, the AI techniques used and their combinations, and current challenges and future directions. Results of reviewing 43 relevant studies reveal that AI-IA integration is still in its early stages, as existing research has mainly focused on a limited range of data types and application scenarios. By analyzing the application domains, this systematic literature review supports previous findings of important applications in the fields of education, manufacturing, and healthcare. At the same time, it identifies emerging applications that have progressed from XR and AI domains to AI-IA integration, such as sports events, assistive systems, urban planning, and disaster management. We contribute to extending established visual analytics (VA) pipelines into XR environments with integrated AI techniques. AI techniques are identified as contributing in five ways to this IA pipeline. Our contribution also includes identifying four key challenges and seven opportunities for future exploration. The review concludes that combining AI and IA holds the potential to create innovative applications using advanced AI and immersive visualization techniques. We present an overview of these applications and address key issues for future development. 

Place, publisher, year, edition, pages
Tsinghua University Press, 2026
Keywords
artificial intelligence (AI), augmented reality (AR), immersive analytics (IA), virtual reality (VR), visual analytics (VA), visualization, Advanced Analytics, Artificial intelligence, Augmented reality, Data integration, Data visualization, Disaster prevention, Disasters, Engineering education, Integration, Urban planning, Virtual reality, Visual analytics, Analytic integration, Applications domains, Artificial intelligence techniques, Immersive, Immersive analytic, Visual analytic, Flow visualization
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:bth-29183 (URN)10.26599/CVM.2025.9450478 (DOI)001684103200008 ()2-s2.0-105029964479 (Scopus ID)
Funder
Knowledge Foundation, 20220068
Available from: 2026-02-25 Created: 2026-02-25 Last updated: 2026-02-25Bibliographically approved
Silonosov, A., Casalicchio, E. & Henesey, L. (2026). SoK: Evolution of the Key Encapsulation Mechanism's Role in Cryptographic Migrations for IoT Systems. IEEE Access, 14, 10595-10614
Open this publication in new window or tab >>SoK: Evolution of the Key Encapsulation Mechanism's Role in Cryptographic Migrations for IoT Systems
2026 (English)In: IEEE Access, E-ISSN 2169-3536, Vol. 14, p. 10595-10614Article, review/survey (Refereed) Published
Abstract [en]

Key Encapsulation Mechanisms (KEM) is a special case of Public Key Encryption (PKE) that was recently standardized by National Institute of Standards and Technology in USA. The broader adoption of the term in industry practice was necessitated by the discovery of the malleability property of ciphertext, which led to new approaches PKE. New standard initiated refactoring of all cryptographic software libraries and this process relates to the problems of cryptographic agility. This Systematization of Knowledge (SoK) addresses the developments of public key encryption methods and the main challenges that drive the specialization of KEM in cryptographic software. Based upon our findings from a systematic literature review, we present a formal analysis that provides cryptographic users a means of better understanding of KEM and its roles in cryptographic migrations in IoT systems. We have identified the main milestones of KEM evolution and structured it into four development areas. We found that the evolution of KEM is defined by a variety of mathematical foundations that always reflect various aspects of the cryptosystem. Our findings indicate that academia, industry practitioners and standardization bodies propagate such approaches into practice by additional abstraction layers in cryptographic software libraries. However, the libraries is still not in consensus, which is confirmed after the discovery of a new class of libraries, cryptographic bindings. To structure the mentioned phenomena, we introduced a novel, three-facet, consumer-centered mapping of the data security domain.We believe our contribution can help researchers and practitioners to have a broader and deeper understanding of data encryption tooling in context of cryptographic migrations.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2026
Keywords
algorithms, cryptographic agility, end-to-end encryption (E2EE), KEM, Key Encapsulation Mechanisms, migrations, post-quantum, PQC, pre-quantum, software libraries, standards, transition, Ciphertext, Computer software, Data encapsulation, Data privacy, Libraries, Quantum cryptography, CryptoGraphics, End-to-end encryption, Migration, Post quantum, Public key cryptography
National Category
Security, Privacy and Cryptography Computer Sciences
Identifiers
urn:nbn:se:bth-29131 (URN)10.1109/ACCESS.2026.3654143 (DOI)001669224300047 ()2-s2.0-105027663477 (Scopus ID)
Projects
SERICS
Funder
Knowledge Foundation, 20220068European Commission, PE00000014
Available from: 2026-01-30 Created: 2026-01-30 Last updated: 2026-02-25Bibliographically approved
Devagiri, V. M., Boeva, V. & Abghari, S. (2025). A Domain Adaptation Technique through Cluster Boundary Integration. Evolving Systems, 16(1), Article ID 14.
Open this publication in new window or tab >>A Domain Adaptation Technique through Cluster Boundary Integration
2025 (English)In: Evolving Systems, ISSN 1868-6478, E-ISSN 1868-6486, Vol. 16, no 1, article id 14Article in journal (Refereed) Published
Abstract [en]

Many machine learning models deployed on smart or edge devices experience a phase where there is a drop in their performance due to the arrival of data from new domains. This paper proposes a novel unsupervised domain adaptation algorithm called DIBCA++ to deal with such situations. The algorithm uses only the clusters’ mean, standard deviation, and size, which makes the proposed algorithm modest in terms of the required storage and computation. The study also presents the explainability aspect of the algorithm. DIBCA++ is compared with its predecessor, DIBCA, and its applicability and performance are studied and evaluated in two real-world scenarios. One is coping with the Global Navigation Satellite System activation problem from the smart logistics domain, while the other identifies different activities a person performs and deals with a human activity recognition task. Both scenarios involve time series data phenomena, i.e., DIBCA++ also contributes towards addressing the current gap regarding domain adaptation solutions for time series data. Based on the experimental results, DIBCA++ has improved performance compared to DIBCA. The DIBCA++ has performed better in all human activity recognition task experiments and 82.5% of experimental scenarios on the smart logistics use case. The results also showcase the need and benefit of personalizing the models using DIBCA++, along with the ability to transfer new knowledge between domains, leading to improved performance. The adapted source and target models have performed better in 70% and 80% of cases in an experimental scenario conducted on smart logistics. 

Place, publisher, year, edition, pages
Springer Nature, 2025
Keywords
Cluster integration, Clustering techniques, Domain adaptation
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-26090 (URN)10.1007/s12530-024-09635-z (DOI)001363397000001 ()2-s2.0-85210317128 (Scopus ID)
Funder
Knowledge Foundation, 20220068
Available from: 2024-04-09 Created: 2024-04-09 Last updated: 2025-09-30Bibliographically approved
Daliparthi, V. S., Tutschku, K., Momen, N., De Prado, M., Divernois, M., Pazos Escudero, N. & Bonnefous, J.-M. (2025). A License Management System for Collaborative AI Engineering. In: ACM International Conference Proceeding Series: . Paper presented at 2024 7th Artificial Intelligence and Cloud Computing Conference, AICCC 2024 and its workshop the 2024 6th Asia Digital Image Processing Conference, Tokyo, Dec 14-16, 2024 (pp. 77-86). Association for Computing Machinery (ACM)
Open this publication in new window or tab >>A License Management System for Collaborative AI Engineering
Show others...
2025 (English)In: ACM International Conference Proceeding Series, Association for Computing Machinery (ACM), 2025, p. 77-86Conference paper, Published paper (Refereed)
Abstract [en]

The AI marketplace ecosystem accelerates multiple modules of the AI engineering pipeline by fostering collaboration between stakeholders. However, marketplace collaborators often face a dilemma in striking a balance between sharing artifacts and protecting intellectual property (IP) rights. Thus, there is a need for a license management system within the AI marketplace to facilitate the exchange of artifacts in a trusted and secure manner. 

This work shares experiences while building such a license management system within the Bonseyes marketplace (BMP), a functional crowdsourcing AI marketplace that specializes in deploying real-time applications on edge devices. The BMP was developed, and its applicability is proven through the European H2020 project by a series of open calls and workshops, for gathering stakeholders and orchestrating the marketplace operations. 

The main contributions of this work are (i) implementation of an end-to-end license management system that deals with selecting license templates, license agreement interaction between seller and buyer, and the generation and enforcement of human- and machine-readable license files, and (ii) introduction of "Synchronization licenses'' concept from the music industry to the AI marketplace context where consumers acquire a license to integrate the artifact into another application, and a respective BMP use-case for collaborative AI engineering. 

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2025
Keywords
License Management, AI Marketplaces, Data Marketplaces, Collaborative AI Engineering
National Category
Information Systems
Research subject
Computer Science; Systems Engineering
Identifiers
urn:nbn:se:bth-27607 (URN)10.1145/3719384.3719395 (DOI)2-s2.0-105011739146 (Scopus ID)9798400717925 (ISBN)
Conference
2024 7th Artificial Intelligence and Cloud Computing Conference, AICCC 2024 and its workshop the 2024 6th Asia Digital Image Processing Conference, Tokyo, Dec 14-16, 2024
Projects
dAIEDGE: HORIZON-CL4-2022-HUMAN-02-02
Funder
Knowledge Foundation, 20220068EU, Horizon Europe, 101120726
Available from: 2025-03-17 Created: 2025-03-17 Last updated: 2025-09-30Bibliographically approved
Qin, B., Tutschku, K. & Hu, Y. (2025). A survey on Digital Twins for Multi-User Synchronization in Human-centered IRs. In: Proceedings - 2025 International Conference on Metaverse Computing, Networking and Applications, MetaCom 2025: . Paper presented at 3rd IEEE International Conference on Metaverse Computing, Networking and Applications, MetaCom 2025, Seoul, Aug 27-29, 2025 (pp. 163-164). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>A survey on Digital Twins for Multi-User Synchronization in Human-centered IRs
2025 (English)In: Proceedings - 2025 International Conference on Metaverse Computing, Networking and Applications, MetaCom 2025, Institute of Electrical and Electronics Engineers (IEEE), 2025, p. 163-164Conference paper, Published paper (Refereed)
Abstract [en]

Multi-user extended reality (XR) is an advanced technology that immerses users simultaneously into the meta-verse, aiming to provide a real-time interaction experience in different scenarios. However, different users' network conditions cause data transmission to be asynchronous. It forces users' behavior in the metaverse to go out of sync, seriously impacting the user experience. This paper presents a framework for human-centred synchronization of networks and metaverse applications, often denoted as Intelligent Realities, using Digital Twin (DT) technologies for the networks (network DTs) as well as for the individual users' XR parts (XR DTs). Our framework uses users' attention as a key input to dynamically allocate network resources and align content, thereby keeping widely distributed users in sync. To motivate this framework, we surveyed recent DT-based XR and network systems and found a significant gap: few studies work on synchronizing systems between network DTs and XR DTs. This highlights the need to explore integrated solutions for networked real-time multi-user interaction in the metaverse. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2025
Keywords
digital twin, HCI, multi-user, network resource allocation, XR, Behavioral research, Human computer interaction, Synchronization, User experience, User interfaces, Advanced technology, Data-transmission, Interaction experiences, Metaverses, Multiusers, Network condition, Network resource allocations, Real time interactions, User networks
National Category
Computer Sciences Computer Systems
Identifiers
urn:nbn:se:bth-28919 (URN)10.1109/MetaCom65502.2025.00036 (DOI)2-s2.0-105020807442 (Scopus ID)9798331522551 (ISBN)
Conference
3rd IEEE International Conference on Metaverse Computing, Networking and Applications, MetaCom 2025, Seoul, Aug 27-29, 2025
Funder
Knowledge Foundation, 20220068
Available from: 2025-11-21 Created: 2025-11-21 Last updated: 2025-11-24Bibliographically approved
Al-Saedi, A. A. & Boeva, V. (2025). ADF-SL: An Adaptive and Fair Scheme for Smart Learning Task Distribution. IEEE Access, 13, 122928-122942
Open this publication in new window or tab >>ADF-SL: An Adaptive and Fair Scheme for Smart Learning Task Distribution
2025 (English)In: IEEE Access, E-ISSN 2169-3536, Vol. 13, p. 122928-122942Article in journal (Refereed) Published
Abstract [en]

Split Learning (SL) is an emerging decentralized paradigm that enables numerous participants, to train a deep neural network without disclosing sensitive information, such as patient data, in fields such as healthcare. In healthcare, SL enables distributed training across a variety of medical devices, hospitals, and organizations, improving model robustness while maintaining patient confidentiality. However, training models within SL is affected by data heterogeneity and sensitivity, and often requires more computational resources than an individual data provider can afford. This can result in significant model divergence and decreased performance due to differences in data distributions between various clients. To address this issue, we propose a framework that integrates fairness and adaptivity considerations, called ADF-SL. In particular, ADF-SL dynamically adjusts the total number of clients involved in model training and the number of iteration required to achieve convergence without compromising participant privacy. To evaluate performance, we compare the effectiveness of ADF-SL with that of the naive (Vanilla) SL approach, SplitFed and FairFed. Extensive experiments performed on time series electrocardiogram (ECG) databases (MITDB, SVDB, and INCARTDB) indicate that ADF-SL significantly outperforms the three existing algorithms that served as baselines. Compared to these baseline methods, ADF-SL accelerates model training on clients by up to 22.7%, 10.4%, and 5.8% compared to Vanilla SL, SplitFed, and FairFed, respectively, while maintaining model convergence and accuracy. Furthermore, the conducted ablation study has confirmed the importance of ADF-SL decay enrichment, which has outperformed non-decay ADF-SL for each used dataset by up to 15.8%, 43.9%, and 7.6%, respectively. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2025
Keywords
Artificial intelligence, efficiency, fairness, layer distribution, split learning, Deep learning, Electrocardiograms, Learning systems, Medical computing, Decentralised, Layer distributions, Learning tasks, Model training, Neural-networks, Performance, Sensitive informations, Task distribution, Health care
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-28486 (URN)10.1109/ACCESS.2025.3586544 (DOI)001531849600040 ()2-s2.0-105010185357 (Scopus ID)
Funder
Knowledge Foundation, 20220068
Available from: 2025-08-12 Created: 2025-08-12 Last updated: 2025-09-30Bibliographically approved
Murtas, G., Boeva, V. & Tsiporkova, E. (2025). An evidence-based neuro-symbolic framework for ambiguous image scene classification. In: Gilpin L.H., Giunchiglia E., Hitzler P., van Krieken E. (Ed.), Proceedings of Machine Learning Research: . Paper presented at 19th Conference on Neurosymbolic Learning and Reasoning, NeSy 2025, Santa Cruz, Sept 8-10, 2025 (pp. 992-1003). ML Research Press, 284
Open this publication in new window or tab >>An evidence-based neuro-symbolic framework for ambiguous image scene classification
2025 (English)In: Proceedings of Machine Learning Research / [ed] Gilpin L.H., Giunchiglia E., Hitzler P., van Krieken E., ML Research Press , 2025, Vol. 284, p. 992-1003Conference paper, Published paper (Refereed)
Abstract [en]

In this study, we propose a novel neuro-symbolic approach to deal with the inherent ambiguity in image scene classification, combining the usage of pre-trained deep learning (DL) models with concepts from modal logic and evidence theory. The DL models are used to detect objects and estimate their depth in a set of labeled images. The obtained outputs are employed to form a dataset of instances characterizing the possible classes. Subsequently, a multi-valued mapping is defined between the data instances and the considered images resulting into each image being represented by the set of instances associated with it. The obtained mapping is utilized to infer necessity and possibility conditions of each class, or equivalently its upper (plausibility) and lower (belief) probabilities. Based on these interval evaluations, a rule-based and a score-based classifiers are built. The overall method is explainable and directly interpretable, robust to data scarcity and data imbalance. The presented framework is studied and evaluated on an abandoned bag detection use case. 

Place, publisher, year, edition, pages
ML Research Press, 2025
Series
Proceedings of Machine Learning Research (PMLR), E-ISSN 2640-3498
Keywords
Computer Vision, Evidence Theory, Image Scene Classification, Modal Logic, Multi-valued Mapping, Classification (of information), Computer circuits, Deep learning, Image classification, Mapping, Condition, Evidence theories, Evidence-based, Labeled images, Learning models, Logic theory, Multivalued mappings, Rule based
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:bth-28865 (URN)001669525800050 ()2-s2.0-105020237323 (Scopus ID)9781713845065 (ISBN)
Conference
19th Conference on Neurosymbolic Learning and Reasoning, NeSy 2025, Santa Cruz, Sept 8-10, 2025
Funder
Knowledge Foundation, 20220068
Available from: 2025-11-07 Created: 2025-11-07 Last updated: 2026-02-27Bibliographically approved
Principal InvestigatorSundstedt, Veronica
Coordinating organisation
Blekinge Institute of Technology
Period
2022-09-01 - 2028-08-31
National Category
Computer Sciences
Identifiers
DiVA, id: project:3003Project, id: 20220068

Search in DiVA

Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

Link to external project page

HINTS