Contrastive Learning in PyTorch - Part 1: Introduction

▬▬ Notes ▬▬▬▬▬▬▬▬▬▬▬
Two small things I realized when editing this video
- SimCLR uses two separate augmented views as positive samples
- Many frameworks have separate projection heads on the learned representations
which transforms them additionally for the contrastive loss

▬▬ Papers/Sources ▬▬▬▬▬▬▬
- Intro: https://sthalles.github.io/a-few-word...
- Survey: https://arxiv.org/ftp/arxiv/papers/20...
- Supervised Contrastive Learning: https://arxiv.org/abs/2004.11362
- Contrastive Loss:  / losses-explained-contrastive-loss  
- Triplet Loss: https://towardsdatascience.com/triple...
- NT-Xent Loss: https://medium.datadriveninvestor.com...
- SimCLR

▬▬ Used Icons ▬▬▬▬▬▬▬▬▬▬
All Icons are from flaticon: https://www.flaticon.com/authors/freepik

▬▬ Used Music ▬▬▬▬▬▬▬▬▬▬▬
Music from Uppbeat (free for Creators!):
https://uppbeat.io/t/t-check/lemon-limes
License code: KJ7PFP0HB9BWHJOF

▬▬ Used Images ▬▬▬▬▬▬▬▬▬▬▬
All Images are from pixabay.com (Cats, Dogs, ...)
and royalty-free.

▬▬ Timestamps ▬▬▬▬▬▬▬▬▬▬▬
00:00 Introduction
00:22 Overview
01:35 Supervised vs. Self-Supervised CL
03:25 Applications
04:24 Popular Papers
06:17 Metric Learning
07:21 Loss 1
09:39 Loss 2
10:54 Loss 3
13:22 Variations between Losses
13:42 Part 2 Outlook

▬▬ Support me if you like 🌟
►Website: https://deepfindr.github.io/
►Support me on Patreon: https://bit.ly/2Wed242
►Buy me a coffee on Ko-Fi: https://bit.ly/3kJYEdl
►Coursera: https://imp.i384100.net/b31QyP
►Link to this channel: https://bit.ly/3zEqL1W
►E-Mail: [email protected]

▬▬ My equipment 💻
- Microphone: https://amzn.to/3DVqB8H
- Microphone mount: https://amzn.to/3BWUcOJ
- Monitors: https://amzn.to/3G2Jjgr
- Monitor mount: https://amzn.to/3AWGIAY
- Height-adjustable table: https://amzn.to/3aUysXC
- Ergonomic chair: https://amzn.to/3phQg7r
- PC case: https://amzn.to/3jdlI2Y
- GPU: https://amzn.to/3AWyzwy
- Keyboard: https://amzn.to/2XskWHP
- Bluelight filter glasses: https://amzn.to/3pj0fK2