Search
Abstract
Exemplar-Free Class Incremental Learning (EFCIL) tackles the problem of training a model on a sequence of tasks without access to past data. Existing state-of-the-art methods represent classes as Gaussian distributions in the feature extractor's latent space, enabling Bayes classification or training the classifier by replaying pseudo features. However, we identify two critical issues that compromise their efficacy when the feature extractor is updated on incremental tasks. First, they do not consider that classes' covariance matrices change and must be adapted after each task. Second, they are susceptible to a task-recency bias caused by dimensionality collapse occurring during training. In this work, we propose AdaGauss - a novel method that adapts covariance matrices from task to task and mitigates the task-recency bias owing to the additional anti-collapse loss function. AdaGauss yields state-of-the-art results on popular EFCIL benchmarks and datasets when training from scratch or starting from a pre-trained backbone.
Authors (4)
-
Grzegorz Rypeść
-
Tomasz Trzciński dr hab. inż.
-
Bartłomiej Twardowski dr inż.
Cite as
Full text
- Publication version
- Accepted or Published Version
- License
-
open in new tab
Keywords
Details
- Category:
- Conference activity
- Type:
- publikacja w wydawnictwie zbiorowym recenzowanym (także w materiałach konferencyjnych)
- Language:
- English
- Publication year:
- 2024
- Bibliographic description:
- Rypeść G., Cygert S., Trzciński T., Twardowski B.: Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning// / : , 2024,
- Sources of funding:
-
- Ze środków IDEAS NCBR
- Verified by:
- Gdańsk University of Technology
seen 2 times
Recommended for you
MagMax: Leveraging Model Merging for Seamless Continual Learning
- D. Marczak,
- B. Twardowski,
- T. Trzciński
- + 1 authors
MP3vec: A Reusable Machine-Constructed Feature Representation for Protein Sequences
- S. R. Gupte,
- D. S. Jain,
- A. Srinivasan
- + 1 authors
Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learning
- F. Szatkowski,
- M. Pyła,
- M. Przewięźlikowski
- + 3 authors