CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Resource-Aware and Personalized Federated Learning via Clustering Analysis
Blekinge Institute of Technology, Faculty of Computing, Department of Computer Science. (AIDA)ORCID iD: 0000-0001-6061-0861
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Today’s advancement in Artificial Intelligence (AI) enables training Machine Learning (ML) models on the daily-produced data by connected edge devices. To make the most of the data stored on the device, conventional ML approaches require gathering all individual data sets and transferring them to a central location to train a common model. However, centralizing data incurs significant costs related to communication, network resource utilization, high volume of traffic, and privacy issues. To address the aforementioned challenges, Federated Learning (FL) is employed as a novel approach to train a shared model on decentralized edge devices while preserving privacy. Despite the significant potential of FL, it still requires considerable resources such as time, computational power, energy, and bandwidth availability. More importantly, the computational capabilities of the training devices may vary over time. Furthermore, the devices involved in the training process of FL may have distinct training datasets that differ in terms of their size and distribution. As a result of this, the convergence of the FL models may become unstable and slow. These differences can influence the FL process and ultimately lead to suboptimal model performance within a heterogeneous federated network.

In this thesis, we have tackled several of the aforementioned challenges. Initially, a FL algorithm is proposed that utilizes cluster analysis to address the problem of communication overhead. This issue poses a major bottleneck in FL, particularly for complex models, large-scale applications, and frequent updates. The next research conducted in this thesis involved extending the previous study to include wireless networks (WNs). In WSNs, achieving energy-efficient transmission is a significant challenge due to their limited resources. This has motivated us to continue with a comprehensive overview and classification of the latest advancements in context-aware edge-based AI models, with a specific emphasis on sensor networks. The review has also investigated the associated challenges and motivations for adopting AI techniques, along with an evaluation of current areas of research that need further investigation. To optimize the aggregation of the FL model and alleviate communication expenses, the initial study addressing communication overhead is extended to include a FL-based cluster optimization approach. Furthermore, to reduce the detrimental effect caused by data heterogeneity among edge devices on FL, a new study of group-personalized FL models has been conducted. Finally, taking inspiration from the previously mentioned FL models, techniques for assessing clients' contribution by monitoring and evaluating their behavior during training are proposed. In comparison with the most existing contribution evaluation solutions, the proposed techniques do not require significant computational resources.

The FL algorithms presented in this thesis are assessed on a range of real-world datasets. The extensive experiments demonstrated that the proposed FL techniques are effective and robust. These techniques improve communication efficiency, resource utilization, model convergence speed, and aggregation efficiency, and also reduce data heterogeneity when compared to other state-of-the-art methods.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2024. , p. 260
Series
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 2024:04
Keywords [en]
Federated Learning, Clustering Analysis, Eccentricity Analysis, Non- IID Data, Model Personalization
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:bth-26081ISBN: 978-91-7295-478-6 (print)OAI: oai:DiVA.org:bth-26081DiVA, id: diva2:1849033
Public defence
2024-05-17, C413A, Karlskrona, 10:00 (English)
Opponent
Supervisors
Available from: 2024-04-05 Created: 2024-04-05 Last updated: 2024-04-22Bibliographically approved
List of papers
1. An Energy-aware Multi-Criteria Federated Learning Model for Edge Computing
Open this publication in new window or tab >>An Energy-aware Multi-Criteria Federated Learning Model for Edge Computing
2021 (English)In: Proceedings - 2021 International Conference on Future Internet of Things and Cloud, FiCloud 2021 / [ed] Younas M., Awan I., Unal P., IEEE, 2021, p. 134-143Conference paper, Published paper (Refereed)
Abstract [en]

The successful convergence of Internet of Things (IoT) technology and distributed machine learning have leveraged to realise the concept of Federated Learning (FL) with the collaborative efforts of a large number of low-powered and small-sized edge nodes. In Wireless Networks (WN), an energy-efficient transmission is a fundamental challenge since the energy resource of edge nodes is restricted.In this paper, we propose an Energy-aware Multi-Criteria Federated Learning (EaMC-FL) model for edge computing. The proposed model enables to collaboratively train a shared global model by aggregating locally trained models in selected representative edge nodes (workers). The involved workers are initially partitioned into a number of clusters with respect to the similarity of their local model parameters. At each training round a small set of representative workers is selected on the based of multi-criteria evaluation that scores each node representativeness (importance) by taking into account the trade-off among the node local model performance, consumed energy and battery lifetime. We have demonstrated through experimental results the proposed EaMC-FL model is capable of reducing the energy consumed by the edge nodes by lowering the transmitted data.

Place, publisher, year, edition, pages
IEEE, 2021
Keywords
Federated Learning, Clustering Analysis, Energy consumption, battery lifetime, Human Activity Recognition
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:bth-22236 (URN)10.1109/FiCloud49777.2021.00027 (DOI)2-s2.0-85119667934 (Scopus ID)9781665425742 (ISBN)
Conference
8th International Conference on Future Internet of Things and Cloud, FiCloud 2021, Virtual, Online, 23 August 2021 through 25 August 2021
Available from: 2021-10-26 Created: 2021-10-26 Last updated: 2024-04-05Bibliographically approved
2. Context-Aware Edge-Based AI Models for Wireless Sensor Networks-An Overview
Open this publication in new window or tab >>Context-Aware Edge-Based AI Models for Wireless Sensor Networks-An Overview
2022 (English)In: Sensors, E-ISSN 1424-8220, Vol. 22, no 15, article id 5544Article, review/survey (Refereed) Published
Abstract [en]

Recent advances in sensor technology are expected to lead to a greater use of wireless sensor networks (WSNs) in industry, logistics, healthcare, etc. On the other hand, advances in artificial intelligence (AI), machine learning (ML), and deep learning (DL) are becoming dominant solutions for processing large amounts of data from edge-synthesized heterogeneous sensors and drawing accurate conclusions with better understanding of the situation. Integration of the two areas WSN and AI has resulted in more accurate measurements, context-aware analysis and prediction useful for smart sensing applications. In this paper, a comprehensive overview of the latest developments in context-aware intelligent systems using sensor technology is provided. In addition, it also discusses the areas in which they are used, related challenges, motivations for adopting AI solutions, focusing on edge computing, i.e., sensor and AI techniques, along with analysis of existing research gaps. Another contribution of this study is the use of a semantic-aware approach to extract survey-relevant subjects. The latter specifically identifies eleven main research topics supported by the articles included in the work. These are analyzed from various angles to answer five main research questions. Finally, potential future research directions are also discussed.

Place, publisher, year, edition, pages
MDPI, 2022
Keywords
artificial intelligence, context-awareness, edge computing, wireless sensor network, computer network, human, wireless communication, Computer Communication Networks, Humans, Wireless Technology
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-23537 (URN)10.3390/s22155544 (DOI)000839768900001 ()2-s2.0-85135202158 (Scopus ID)
Note

open access

Available from: 2022-08-12 Created: 2022-08-12 Last updated: 2024-04-05Bibliographically approved
3. Contribution Prediction in Federated Learning via Client Behavior Evaluation
Open this publication in new window or tab >>Contribution Prediction in Federated Learning via Client Behavior Evaluation
(English)Manuscript (preprint) (Other academic)
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-26080 (URN)
Available from: 2024-04-05 Created: 2024-04-05 Last updated: 2024-04-05Bibliographically approved
4. Reducing Communication Overhead of Federated Learning through Clustering Analysis
Open this publication in new window or tab >>Reducing Communication Overhead of Federated Learning through Clustering Analysis
2021 (English)In: 26th IEEE Symposium on Computers and Communications (ISCC 2021), Institute of Electrical and Electronics Engineers (IEEE), 2021Conference paper, Published paper (Refereed)
Abstract [en]

Training of machine learning models in a Datacenter, with data originated from edge nodes, incurs high communication overheads and violates a user's privacy. These challenges may be tackled by employing Federated Learning (FL) machine learning technique to train a model across multiple decentralized edge devices (workers) using local data. In this paper, we explore an approach that identifies the most representative updates made by workers and those are only uploaded to the central server for reducing network communication costs. Based on this idea, we propose a FL model that can mitigate communication overheads via clustering analysis of the worker local updates. The Cluster Analysis-based Federated Learning (CA-FL) model is studied and evaluated in human activity recognition (HAR) datasets. Our evaluation results show the robustness of CA-FL in comparison with traditional FL in terms of accuracy and communication costs on both IID and non-IID  cases.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2021
Series
Proceedings of the IEEE Symposium on Computers and Communications, ISSN 15301346
Keywords
Federated Learning, Communication Costs, Clustering Analysis, Human Activity Recognition
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:bth-22237 (URN)10.1109/ISCC53001.2021.9631391 (DOI)000936276000023 ()2-s2.0-85123206373 (Scopus ID)9781665427449 (ISBN)
Conference
26th IEEE Symposium on Computers and Communications, ISCC 2021, Athens, 5 September 2021 through 8 September 2021
Available from: 2021-10-26 Created: 2021-10-26 Last updated: 2024-04-05Bibliographically approved
5. FedCO: Communication-Efficient Federated Learning via Clustering Optimization †
Open this publication in new window or tab >>FedCO: Communication-Efficient Federated Learning via Clustering Optimization †
2022 (English)In: Future Internet, E-ISSN 1999-5903, Vol. 14, no 12, article id 377Article in journal (Refereed) Published
Abstract [en]

Federated Learning (FL) provides a promising solution for preserving privacy in learning shared models on distributed devices without sharing local data on a central server. However, most existing work shows that FL incurs high communication costs. To address this challenge, we propose a clustering-based federated solution, entitled Federated Learning via Clustering Optimization (FedCO), which optimizes model aggregation and reduces communication costs. In order to reduce the communication costs, we first divide the participating workers into groups based on the similarity of their model parameters and then select only one representative, the best performing worker, from each group to communicate with the central server. Then, in each successive round, we apply the Silhouette validation technique to check whether each representative is still made tight with its current cluster. If not, the representative is either moved into a more appropriate cluster or forms a cluster singleton. Finally, we use split optimization to update and improve the whole clustering solution. The updated clustering is used to select new cluster representatives. In that way, the proposed FedCO approach updates clusters by repeatedly evaluating and splitting clusters if doing so is necessary to improve the workers’ partitioning. The potential of the proposed method is demonstrated on publicly available datasets and LEAF datasets under the IID and Non-IID data distribution settings. The experimental results indicate that our proposed FedCO approach is superior to the state-of-the-art FL approaches, i.e., FedAvg, FedProx, and CMFL, in reducing communication costs and achieving a better accuracy in both the IID and Non-IID cases. © 2022 by the authors.

Place, publisher, year, edition, pages
MDPI, 2022
Keywords
clustering, communication efficiency, convolutional neural network, federated learning, Internet of Things, Convolutional neural networks, Cost reduction, Learning systems, Privacy-preserving techniques, Central servers, Clustering optimizations, Clusterings, Communication cost, Optimization approach, Shared model, Workers'
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-24176 (URN)10.3390/fi14120377 (DOI)000901037100001 ()2-s2.0-85144590253 (Scopus ID)
Note

open access

Available from: 2023-01-10 Created: 2023-01-10 Last updated: 2024-04-05Bibliographically approved
6. Group-Personalized Federated Learning for Human Activity Recognition Through Cluster Eccentricity Analysis
Open this publication in new window or tab >>Group-Personalized Federated Learning for Human Activity Recognition Through Cluster Eccentricity Analysis
2023 (English)In: Engineering Applications of Neural Networks: 24th International Conference, EAAAI/EANN 2023, León, Spain, June 14–17, 2023, Proceedings / [ed] Iliadis L., Maglogiannis I., Alonso S., Jayne C., Pimenidis E., Springer Science+Business Media B.V., 2023, p. 505-519Conference paper, Published paper (Refereed)
Abstract [en]

Human Activity Recognition (HAR) plays a significant role in recent years due to its applications in various fields including health care and well-being. Traditional centralized methods reach very high recognition rates, but they incur privacy and scalability issues. Federated learning (FL) is a leading distributed machine learning (ML) paradigm, to train a global model collaboratively on distributed data in a privacy-preserving manner. However, for HAR scenarios, the existing action recognition system mainly focuses on a unified model, i.e. it does not provide users with personalized recognition of activities. Furthermore, the heterogeneity of data across user devices can lead to degraded performance of traditional FL models in the smart applications such as personalized health care. To this end, we propose a novel federated learning model that tries to cope with a statistically heterogeneous federated learning environment by introducing a group-personalized FL (GP-FL) solution. The proposed GP-FL algorithm builds several global ML models, each one trained iteratively on a dynamic group of clients with homogeneous class probability estimations. The performance of the proposed FL scheme is studied and evaluated on real-world HAR data. The evaluation results demonstrate that our approach has advantages in terms of model performance and convergence speed with respect to two baseline FL algorithms used for comparison. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Place, publisher, year, edition, pages
Springer Science+Business Media B.V., 2023
Series
Communications in Computer and Information Science (CCIS), ISSN 1865-0929, E-ISSN 1865-0937 ; 1826
Keywords
Clustering, Eccentricity Analysis, Federated Learning, HAR, Non-IID data, Computer aided instruction, Iterative methods, Learning systems, Pattern recognition, Privacy-preserving techniques, Centralised, Clusterings, Eccentricity analyse, Human activity recognition, IID data, ITS applications, Learning models, Well being, Health care
National Category
Computer Sciences
Identifiers
urn:nbn:se:bth-25227 (URN)10.1007/978-3-031-34204-2_41 (DOI)2-s2.0-85164039066 (Scopus ID)9783031342035 (ISBN)
Conference
24th International Conference on Engineering Applications of Neural Networks, EANN 2023, León, 14 June through 17 June 2023
Funder
Knowledge Foundation, 20220068
Available from: 2023-08-07 Created: 2023-08-07 Last updated: 2024-04-05Bibliographically approved

Open Access in DiVA

fulltext(17764 kB)116 downloads
File information
File name FULLTEXT02.pdfFile size 17764 kBChecksum SHA-512
18741155b8bab4de1d0a1ac442f3adcf1503bba8026eb6b818a5b225b41919ecb6daa54caa8ae0c66744fa79b10c07680ad4463787f63587a202efa56fb6329b
Type fulltextMimetype application/pdf

Authority records

Al-Saedi, Ahmed Abbas Mohsin

Search in DiVA

By author/editor
Al-Saedi, Ahmed Abbas Mohsin
By organisation
Department of Computer Science
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 116 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1183 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf