- 32
- 120 355
Soroush Mehraban
Canada
Registrace 19. 08. 2020
Diffusion Models (DDPM & DDIM) - Easily explained!
In this video I review how diffusion models work for the task of image generation.
DDPM paper: arxiv.org/abs/2006.11239
DDIM paper: arxiv.org/abs/2010.02502
P.S.: I used Invincible for visualization of my explanations, as I find the series pretty cool. :D
Table of Content:
00:00 Intro
00:23 DDPM
11:26 DDIM
18:14 Outro
Icon made by Freepik from flaticon.com
DDPM paper: arxiv.org/abs/2006.11239
DDIM paper: arxiv.org/abs/2010.02502
P.S.: I used Invincible for visualization of my explanations, as I find the series pretty cool. :D
Table of Content:
00:00 Intro
00:23 DDPM
11:26 DDIM
18:14 Outro
Icon made by Freepik from flaticon.com
zhlédnutí: 947
Video
GLIGEN (CVPR2023): Open-Set Grounded Text-to-Image Generation
zhlédnutí 225Před 3 měsíci
In this video, I review the GLIGEN paper from CVPR2023 that proposes a new type of diffusion model that can receive grounding condition (bounding box, depth map, etc.) in addition to text to generate images with more constraints. Project page: gligen.github.io/ Table of Content: 00:00 Intro 01:00 Grounding Tokenization 03:55 Architecture 07:21 Scheduled Sampling 09:21 Results Icon made by Freep...
The Entropy Enigma: Success and Failure of Entropy Minimization
zhlédnutí 655Před 3 měsíci
In this video I review the entropy enigma paper that proposes a new method to estimate the accuracy of a model without having any labels. Paper link: arxiv.org/abs/2405.05012 Table of Content: 00:00 Intro 00:27 Tent 01:15 Excluding samples 03:31 The two-phase clustering 06:00 Label flips 07:06 Weighted Flips 08:24 Results Icon made by Freepik from flaticon.com
Tent: Fully Test-time Adaptation by Entropy Minimization
zhlédnutí 206Před 3 měsíci
In this video I review the tent approach to increase the test-time accuracy by minimizing the entropy at test time. Paper link: arxiv.org/abs/2006.10726 Table of Content: 00:00 Intro 05:40 Entropy analysis 07:05 TENT Icon made by Freepik from flaticon.com
VPD (ICCV2023): Unleashing Text-to-Image Diffusion Models for Visual Perception
zhlédnutí 199Před 3 měsíci
In this video I review the VPD paper from ICCV2023 that proposes a method that uses the diffusion model as a backbone for a visual perception task. paper link: arxiv.org/abs/2303.02153 Table of Content: 00:00 Intro 02:49 VPD Icon made by Freepik from flaticon.com
TokenHMR (CVPR2024): Advancing Human Mesh Recovery witha Tokenized Pose Representation
zhlédnutí 337Před 3 měsíci
In this video, I review the TokenHMR paper that reconstructs 3D human mesh from an RGB image. Project page: tokenhmr.is.tue.mpg.de/ Table of Content: 00:00 Intro 00:26 SMPL 02:04 Current issue 03:08 Foreshortened legs 05:31 Analysis 11:45 TALS loss function 15:16 Tokenization 18:47 TokenHMR 22:16 Results Icon made by Freepik from flaticon.com
SHViT (CVPR2024): Single-Head Vision Transformer with Memory Efficient Macro Design
zhlédnutí 603Před 3 měsíci
In this video, we review the SHViT (Single-Head Vision Transformer) paper, which introduces a memory-efficient Vision Transformer with competitive performance. SHViT reduces computational redundancy with larger-stride patch embeddings and a single-head attention module. paper link: arxiv.org/abs/2401.16456 Table of Content: 00:00 Intro 00:54 FastViT 01:33 EfficientFormerV2 02:48 Macro Design An...
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation
zhlédnutí 457Před 4 měsíci
Introducing InstaFlow: A game-changer in text-to-image generation! This one-step diffusion model, leveraging Rectified Flow's 'reflow' technique, achieves SD-level image quality in milliseconds. With an FID of 23.3 on MS COCO 2017-5k and training taking just 199 GPU days, InstaFlow sets new standards in speed and quality. Paper link: arxiv.org/abs/2309.06380 You can also read: arxiv.org/abs/220...
FastV: An Image is Worth 1/2 Tokens After Layer 2
zhlédnutí 384Před 5 měsíci
Large Vision-Language Models (LVLMs) tackle question answering in images and videos, but they struggle to efficiently utilize attention, resulting in high computational costs during inference. The FastV paper introduces a method to prune up to 50% of image tokens while maintaining comparable performance to the original model. Paper link: arxiv.org/pdf/2403.06764.pdf Table of Content: 00:00 Intr...
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
zhlédnutí 1,4KPřed 5 měsíci
Large language models (LLMs) typically demand substantial GPU memory, rendering training impractical on a single consumer GPU, especially for a 7-billion-parameter model that necessitates 58GB of memory. In response, the GaLore paper introduces an innovative strategy that projects gradients into a low-rank space, enabling the model to fit within the constraints of a single GPU. Remarkably, this...
PoseGPT (ChatPose): Chatting about 3D Human Pose
zhlédnutí 681Před 7 měsíci
Human pose estimation models struggle to grasp contextual information in images or video frames. Meanwhile, text-to-pose generation models, having limited training data, cannot effectively generate accurate poses for novel prompts. PoseGPT, a novel multimodal language model, not only comprehends 3D human pose but also processes image and text data. This innovative model excels in speculative po...
MotionAGFormer (WACV2024): Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network
zhlédnutí 900Před 8 měsíci
In this video, I review the MotionAGFormer paper for the task monocular 3D human pose estimation. Paper link: arxiv.org/abs/2310.16288 GitHub link: github.com/TaatiTeam/MotionAGFormer Table of content: 00:00 Intro 00:10 MetaFormer and GCFormer 01:25 MotionAGFormer 03:52 GCNFormer's Adjacency Matrix 06:34 MotionAGFormer Variants 06:56 Results and Comparison
HD-GCN (ICCV2023): Skeleton-Based Action Recognition
zhlédnutí 1,3KPřed 9 měsíci
In this video I review the paper "Hierarchically Decomposed Graph Convolutional Networks for Skeleton-Based Action Recognition" which is published in ICCV2023. Paper link: arxiv.org/abs/2208.10741 Table of content: 00:00 Intro 00:24 ST-GCN overview 01:28 MS-G3D overview 02:25 CTR-GCN overview 03:31 Hierarchically Decomposed Graph 20:55 A-HA module 27:12 Six-way Ensemble 29:41 Network Architectu...
ST-GCN: Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
zhlédnutí 3,1KPřed 10 měsíci
ST-GCN is the first GCN-based method for the task of skeleton-based action recognition. In this video, I explain how it works. Paper link: arxiv.org/abs/1801.07455 Icon made by Freepik from flaticon.com
Graph Convolutional Networks (GCN): From CNN point of view
zhlédnutí 4,6KPřed 10 měsíci
Table of Content: 00:00 CNN Summary 00:58 Analogy of CNN with Graph 03:00 Self-loop connection 04:22 GCN paper link: arxiv.org/abs/1609.02907 Icon made by Freepik from flaticon.com
DINO: Self-Supervised Vision Transformers
zhlédnutí 2,6KPřed 11 měsíci
DINO: Self-Supervised Vision Transformers
MoCo (+ v2): Unsupervised learning in computer vision
zhlédnutí 2,6KPřed rokem
MoCo ( v2): Unsupervised learning in computer vision
TrackFormer: Multi-Object Tracking with Transformers
zhlédnutí 4,5KPřed rokem
TrackFormer: Multi-Object Tracking with Transformers
MetaFormer is Actually What You Need for Vision
zhlédnutí 939Před rokem
MetaFormer is Actually What You Need for Vision
ConvNet beats Vision Transformers (ConvNeXt) Paper explained
zhlédnutí 1,6KPřed rokem
ConvNet beats Vision Transformers (ConvNeXt) Paper explained
Masked Autoencoders (MAE) Paper Explained
zhlédnutí 3KPřed rokem
Masked Autoencoders (MAE) Paper Explained
Relative Position Bias (+ PyTorch Implementation)
zhlédnutí 3,6KPřed rokem
Relative Position Bias ( PyTorch Implementation)
Vision Transformer (ViT) Paper Explained
zhlédnutí 2,6KPřed rokem
Vision Transformer (ViT) Paper Explained
Convolutional Block Attention Module (CBAM) Paper Explained
zhlédnutí 6KPřed rokem
Convolutional Block Attention Module (CBAM) Paper Explained
Squeeze-and-Excitation Networks (SENet) paper explained
zhlédnutí 4,8KPřed rokem
Squeeze-and-Excitation Networks (SENet) paper explained
Receptive Fields: Why 3x3 conv layer is the best?
zhlédnutí 7KPřed rokem
Receptive Fields: Why 3x3 conv layer is the best?
can i implement efficientNet with squeeze and excitation
can i implement efficientnet + SE...???
i still cannot understand how self.selative_postion_bias is changed during training. Could please elaborate it ?
how do they detect poses from heatmaps for say 'k' people?
nevermind it doesn't detect multiple poses
Great explanation
Thanks for the Explanation!!! Loved it
Excellent Explanation. Thanks for that!!
Great Explanation!!! one of the best video I've ever seen in GCN thank you
It was very comprehensive, thanks a lot Soroush
Nice short explanation!
Thanks!
Great video, keep it up!
@@vidaadelimosabeb6689 Thanks😃
Wow! great video. thanks a lot.
@@HassanHamidi-v8s Thanks for watching it 🙂
Best explanation of GCNs! Thank you.
I really appreciate it, very good explanation. Thanks!
awesome soroush. nice and clear explain
@@armanhatami5706 Thanks 😃
you are great , please keep going
Thanks Ahmed! Appreciate it.
Such a wonderful and clear video! Thank you
@@user-qp9so1by1j thanks for the feedback 🙂
sick channel, you're a nice orator! Working on a TTA project right now in Uni 💪
@@lucacazzola5327 Thank you so much Luca! I’ve put more recent TTA papers on my agenda to create videos about. Stay tuned🙂
@@soroushmehraban I'll definitelly check It out! Do you have by chance any paper suggestions which specifically targets improving over TPT? I'm running out of ideas (and time 😢)
@@lucacazzola5327 What is TPT?
@@soroushmehraban paper - "Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models" Basically a TTA solution which uses CLIP as backbone
@@lucacazzola5327 Oh sorry I didn't know about that. Thanks for introducing it though.
This part of the SwinTransformer paper is the least understood and took a long time, finally I understood it clearly thanks to this lecture. I would really appreciate it if you could find these points of many papers in the future and explain them easily! Super Thanks!
@@laspinetta2954 I have to understand them first lol. I like to focus on them and make cool videos but recently I got so busy unfortunately
Yeah I’m with this guy
thank you for great explanations!
nice one thx
very nicely explained thank you! likes are at 314 so didnt hit like it😁subbed instead
The best video on this topic hands down
Fudge, you copy other's work
2:43 C would be equal to the number of filters not the number of kernels. In the torch.nn.conv2d operation being performed we have 3 kernels for each input channel and then C number of filters. Each filter having 3 kernels not C number of kernels.
awesome
Thanks for the intuitions, faqs and clearly explained topics!
Glad you liked it Mario🙂
@@soroushmehraban Just one question. Why is centering only applied to the teacher and sharpening to both the student and the teacher? Could we not apply centering to both? Maybe if we add both operations to both sides we play a sum 0 game and we have the collapse problem again, I dont know 😅 Maybe we need then artificially create an unbalance
@@marioparreno24 From my understanding, sharpening makes the model more confident that this sample belongs to a certain sudo-class (the output label of model that we don't have ground truth). And we want the student to be kept certain about it and we sharpen it. The less certain the student is, the less certain it is to differentiate samples from different images. But for images we do both to prevent the mode collapse. But this is just based on my intuition. Don't quote me on that lol.
Very interesting! need more videos.
Hi, I'm still a bit confused so could you please tell me how do you solve the 3D pose judder. The 2D pose contains the judder problem, but I can not find it after lifting to 3D pose in the demo video of your code. Thank you.
Hi Jialiang, Throughout training the model also sees 2D poses with jitters but as the ground truth output, it sees motion capture 3D and we have a velocity loss (we multiply by 20 to make it 20 times more important than MPJPE), that make the model estimation to have the same velocity as the ground truth and penalizes it if it has jitters. So the model in addition to lifting the input from 2D to 3D and inferring the underlying 3D structure, it also has to denoise the input.
veryy veryyy well explained..... i also loved your video on fast rcnn:) amazing workk
WOw so well explained....thankyou very much:)
Is there a way to use grad cam on a Siamese cnn network. I'm getting graph disconnect error whenever i try and i have read that it's because grad cam was originally designed to accept a single input instead of multiple inputs.
By far one of the best + complete, SWIN transformer explanations on the entire Internet.
Thanks!
@@soroushmehraban Hi sir, could you also explain the FasterViT and GCViT paper...
Great video. keep up the good work
This made it so much clear!! Great job :)
Excellent video! Would you recommend any resources that explains the theorems they propose for low-rank gradients and their convergence in-depth? Also, what tools do you use to create such cool animations?
I may not be fully caught up but this gives some context around why cross entropy loss is minimized as a criterion during training. Thanks for this overview.
Prefect !!
Thanks for watching! 😃
Great video Soroush. Thanks.
Thanks for the feedback 😃
You are a great teacher 🙏
Thanks😃
Hello Soroush, how can I contact you please? I am working on my thesis and wanted to need your intake on something. Thanks
Hello, Just search my name on google and you find me on Twitter or Linkedin. My email is also shared here on CZcams
<3
Liked (❤)
Perfect, Thank you.
😃❤️
Fantastic👌
Thanks!
You're amazing, create more!
Thanks for the kind words 🙂
Great Work! You explained 1000 times better than my uni lecturer :D
The loss functions were definitely a bit tricky to get around. But that was a really cool video tho! One thing you could've also touched upon is the usage of deformable detr in place of detr. I can see the trackformer code does incorporate it but wanted to know what changes in trackformer when you switch from detr to deformable one?
you explain complicated stuff very clearly.. thx
usefull