Argmax
Argmax
Vahe Hagopian, Taka Hasegawa, Farrukh Rahman
3: VICReg
44 minutes Posted Mar 21, 2022 at 11:00 pm.
0:00
44:46
Download MP3
Show notes

Todays paper: VICReg (https://arxiv.org/abs/2105.04906)

Summary of the paper
VICReg prevents representation collapse using a mixture of variance, invariance and covariance when calculating the loss. It does not require negative samples and achieves great performance on downstream tasks.

Highlights of discussion

  • The VICReg architecture (Figure 1)
  • Sensitivity to hyperparameters (Table 7)
  • Top 5 metric usefulness