Search
Filters
total: 4
filtered: 3
Chosen catalog filters
Search results for: continual learning
-
Divide and not forget: Ensemble of selectively trained experts in Continual Learning
PublicationClass-incremental learning is becoming more popular as it helps models widen their applicability while not forgetting what they already know. A trend in this area is to use a mixture-of-expert technique, where different models work together to solve the task. However, the experts are usually trained all at once using whole task data, which makes them all prone to forgetting and increasing computational burden. To address this limitation,...
-
Looking through the past: better knowledge retention for generative replay in continual learning
PublicationIn this work, we improve the generative replay in a continual learning setting to perform well on challenging scenarios. Because of the growing complexity of continual learning tasks, it is becoming more popular, to apply the generative replay technique in the feature space instead of image space. Nevertheless, such an approach does not come without limitations. In particular, we notice the degradation of the continually trained...
-
Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learning
PublicationIn this work, we investigate exemplar-free class incremental learning (CIL) with knowledge distillation (KD) as a regularization strategy, aiming to prevent forgetting. KDbased methods are successfully used in CIL, but they often struggle to regularize the model without access to exemplars of the training data from previous tasks. Our analysis reveals that this issue originates from substantial representation shifts in the teacher...