Note_simsiam_mi

January 23, 2022

Put a short note on SimSiam self-distillation objective.

---

Enjoy Reading This Article?

Here are some more articles you might like to read next:

Sequence-level vs. Token-level Importance Sampling in RL for LLMs

Intro: REINFORCE as Importance Sampling

Note on the SimSiam objective