The complex dynamics in telecommunication networks that vary over time is due to network reconfiguration, hardware reinstallation, and other factors related to user behavior. This requires machine learning (ML) models that are robust to dynamic and time-varying data distributions in order to assist reliable network operations. This motivates us to focus on building a generalized ML model that is robust to temporal changes in data distributions by continuously training on various data distributions, that is, concepts, at different times. In a federated learning (FL) setting, when there is a concept drift at one client, iterative aggregation of client model weights causes model entanglement. This may slow down both the forgetting of the data concept learned at previous rounds and also the learning the new data concept in the current round. Traditionally, models are trained using both old and new datasets to prevent performance degradation due to catastrophic forgetting and to simultaneously adapt to new data. This requires the storage of old and new datasets. Continuous increase in the training set size with changing concepts leads to increased arithmetic operations on the data, which subsequently causes an increase in computation requirements. In this article, we propose and evaluate various sample selection methods to sustain the overall performance of ML models while reducing the number of training samples. The proposed methods reduced the training set size, addressed high computation requirement, reduced the impact of catastrophic forgetting, and helped to adapt to new data concepts better.