Wavelets: a mathematical microscope
Vložit
- čas přidán 5. 05. 2024
- Wavelet transform is an invaluable tool in signal processing, which has applications in a variety of fields - from hydrodynamics to neuroscience. This revolutionary method allows us to uncover structures, which are present in the signal but are hidden behind the noise. The key feature of wavelet transform is that it performs function decomposition in both time and frequency domains.
In this video we will see how to build a wavelet toolkit step by step and discuss important implications and prerequisites along the way.
This is my entry for Summer of Math Exposition 2022 ( #SoME2 ).
My name is Artem, I'm a computational neuroscience student and researcher at Moscow State University.
Twitter: @artemkrsv
OUTLINE:
00:00 Introduction
01:55 Time and frequency domains
03:27 Fourier Transform
05:08 Limitations of Fourier
08:45 Wavelets - localized functions
10:34 Mathematical requirements for wavelets
12:17 Real Morlet wavelet
13:02 Wavelet transform overview
14:08 Mother wavelet modifications
15:46 Computing local similarity
18:08 Dot product of functions?
21:07 Convolution
24:55 Complex numbers
27:56 Wavelet scalogram
30:46 Uncertainty & Heisenberg boxes
33:16 Recap and conclusion
Credits:
Vector assets: freepik.com
- Microscope vector created by freepik -www.freepik.com/vectors/micro...
- Lab room vector created by upklyak: www.freepik.com/vectors/lab-room
- Semaphore vector created by macrovector: www.freepik.com/vectors/semap...
Mathematical animations were done using manim (docs.manim.community/en/stable/) and matplotlib python libraries.
3D animations were done in Blender
This is the best discussion of wavelets I Have seen. Your graphics are in the best tradition of 3B1B.
More please.
I agree. My masters-level classes covering Fourier and wavelet transforms were some of the only classes I ever really struggled with and resorted to rote in order to pass.
I wish I had these videos to watch in concurrence with those classes. I remember almost nothing from them because I had no intuition about the subjects I was learning. This explanation is so simple and intuitive I actually want to revisit the subject and see what I missed by using a purely mathematical approach without a deeper understanding.
@@Fred-mv8fx 2
so true!
Now imagine him and 3b1b and vsauce work together on a topic
As an engineer, I can only regret I was born a bit too soon... how lucky of those who are learning thest things with amazing videos like this!
I feel the same
i feel like I was born too late, thats so much to learn even if i learn so much id still be behind 😭
I regret I started to appreciate maths too late
😂 even worse for that Fourier guy
bullshit. as an engineer you have had a lot of money to spare in order to buy cheap bitcoin. meanwhile those of those of us born later had shit and were not able to profit
Man please don’t ever stop making these videos, they are extremely well done and edited and very entertaining while magnificently informative for such complex topics!!!
god, someone watches videos from terror*ssians in late 2022 and likes it
@@romanscerbak5167 what are you even trying to say?
@@romanscerbak5167 I don't see any terrorist here, only a scientist. Just go cry anywhere else.
This is like a third of a semester of intro to signals processing in computer science curriculum, packed into one half-hour video, and I actually understood more from it now than I did from lectures. Huge thanks for doing this!
For those who wonder whether to watch: notable things include good mental models for complex numbers, Fourier transform, convolution and its relationship with vector dot product and functions as infinite-dimensional vectors, with an unexpected cameo from Heisenberg's uncertainty principle. This video is gold.
I can't imagine the amount of work that has gone into this masterpiece of a science yt video❤️🔥
Thank you very much, more content like this is needed😍!
Thank you! ❤️
@@ArtemKirsanov Thanks for sharing Artem. I really hope you can respond to my other comment when you can. Thanks very much.
@@ArtemKirsanov Hey Artem I hope you can respond when when question about the frequency values when you get a chance. I would appreciate it.. Thanks very much.
it's copied.
everyone talks about how amazing are the animations, and forget how amazing is the explanation,
Artem Kirsanov is truly a genius
Hands down the best video on Wavelets. This video packs so much information but in such a succinct & intuitive way, that makes watching it a delight.
This video literally blown my mind about wavelets. There're been several weeks of works studying wavelets (in the discrete domain) for the work of my thesis. So far, more or less I have all the concepts explained in the video clear, but the amazing graphic representation of the signals and wavelets in the video, and also of the entire process of wavelet analysis almost filled all my remaining gaps! This video is incredible to understand wavelets!
What is your major?
@@DannyOvox3 I got master's degree in Computer Science at Roma Tre University; we're using wevelets to analyse BGP anomalous traffic
@@samuelequinzi3153 Oh wow, I am going for a CS degree. I know is a masters level where you are at but these topics seem alien to me, I thought this was related more to electrical engineering.
@@DannyOvox3 this goes far deeper. As you drill down through the sciences in search of core truths, You will find that it all leads You to this, the Key to understanding this existence.
@@DannyOvox3 You'll be surprised how much math is there in CS. CS is not the exact same as software engineering.
This is absolutely mind blowing - especially when you bring in the complex wavelet.
The gradual addition of concepts is extremely well done, and everything is well explained.
27' 33" is just gorgeous. It's wonderful to see visualisation tools that were undreamed of when I was studying Maths in the 1970s, and how expertly people like you can use them.
The subject is highly interesting.
On top of that your video is amazing with all details. The music is very quiet but "opens" the space, the subtle effects on "static" graphs that make them dynamic, the not-so-subtle but entertaining and functional use of sound effects and the use of special effects in manim make this very nice to watch.
I've played around with manim a bit and can only imagine how much work this must've been, holy heck.
Wow, thank you so much!! I really appreciate it
I have to say, the subtle effects were a major negative for me - good video though!
@@laurenpinschannels I definitely love those aesthetic subtle effects
Holy moly, with my startup, I am working on an image analysis project collaborating with hospitals in Spain and the next steps on the project are similar to what you just showed to us. You just gave me more ideas to test and your visualizations are the best! (it reminds me of 3b1b videos)
I will send you some results as soon as we finish it :)
Wow, that's fascinating! Good luck ;)
I can't tell you how much I learnt from this one video. Thanks a lot ! Please keep making these videos.
So many details touched upon, such clear imagination of the underlying geometric intuition. So many little programs written to produce those graphs, diagrams, and visualizations. So refreshing to not rely on the Haar wavelet for an introduction to the topic. This video is going to leave many lasting memories in many minds. I am in awe.
Me too. Cookie cutters you can exit with a cube can leave many questions.
Finally a video that uses manim without being a 3b1b clone. There's clearly a distinct personality here through the sound effects, fonts, and animations. Thinking about the "personality" of your math explainer is important, but unfortunately is neglected often.
I am a math guy, I worked with wavelets on impact noise and vibration analysis... a long time ago. in the mid-90s. 😁 The introduction of wavelets was pretty recent at the time. And we were not far from Brussels and the VUB.
I have to say that this is one of the best presentation I have seen on the topic. Certainly very useful for engineers here in North America.
You can tell how much thoughtfulness went into every visualization here. For example, during the dot product explanation the value of the dot product was mapped onto the distance of the angle marker from the origin, and scaled such that the right angle location made a perfect square. Little touches like that were abound in the video and really help drive home intuition. Every bit of information was there for a reason!
I literally just burst out with a loud "whoa" at 21:14, about the insight of similarity as captured by the inner-product interpretation of the integral. This video is too well done!!!!
This is fcking amazing. The connection of various topics, generalized dot product for functions, Fourier transforms, convolution as a sliding dot product, uncertainty principle... and the visuals amazing as well. Nice job ♡
About 30 years ago I was priviliged to work on an ASIC that performed the Fast Wavelet Transform as the main architect of the ASIC. Before starting the project I was only familiar with the common FFT and how to impliment that in ASIC form, but with help from the math team behind the project at Aware, I was able to learn all the new ideas to get the project into the FFT form. It was almost certainly the most challenging and rewarding ASIC design of my career and as a souvenir I still have the classic text book on the Fast Wavelet Transform. There were 2 dozen other people involved and a half dozen companies in the project. Good times except the money.
It's out! I can't wait to finish it-- the first few minutes is already fantastic!
You've done an amazing job. By far the best short exposition on wavelets on CZcams. Please keep sharing your work with us!
I paused the video to say thank you. You unfolded everything, turned it into a comprehensible lesson.
One of the best videos I have ever watched about a complex subject, congrats!
One of the best videos I have ever seen, and the best explanation of wavelet transform on the internet. I can't imagine how many hours of work you invested here , but it tells a lot about your passion on knowledge sharing. Kudos ! 👏
The legend is back
The traffic light simile is awesome. Never got such a clear intuition of what the Fourier transformation is.
The dot product reveal blew my mind. Subscribed immediately!
You are my favorite channel I’ve found all year. The production and information value of your videos is absolutely unheard of. Please keep doing this, it’s an incredible contribution to the informational commons!
This is a MASTERPIECE, thanks for you for the huge effort to come up with such video.
It takes a visionary genius to be able to transform complex and abstract mathematical concepts into stunningly beautiful animated works of art - Wow!
Your video gave me more insight about wavelet transform than the signal processing course in my graduate school. Thank you.
Excellent. You have an intuitive sense of pace and information that keeps the viewer fascinated and intrigued. This video alone should be mandatory viewing in any university's physics or electronics courses, and I hope you follow it up with others in the same vein.
This is just amazing. The level of detail in this is just baffling. Keep it coming. Your videos are scintillating. I have read wavelet transforms back when i was in Undergrad but this level of detail, wish I had known these intuitive interpretations behind this. All the best to you. This made my day.
lol this is such a hilariously ambitious video, summarizing so many semesters of grad school stuff into one video.
good job man, I hope you get whatever you're aiming for
I've been using and teaching the mathematics of the Fourier Transform, but had never seen a description of wavelets. This is a beautifully done presentation. And, I can tell that once I get to correlation and convolution with my students, they will really enjoy this description of wavelets. To be human and alive these days is just amazing. We are truly in an explosive evolutionary phase of information. And, I am astounded by how many people are so incredibly intelligent, able to describe complex topics so well, and willing to take the time to make these beautiful animated videos.... I wish I could give 1000 thumbs up.
This video is an absolute masterpiece! Not only do you clearly have a gift when it comes to explaining things, but you clearly have an amazing work ethic as well. I can't even imagine how much effort must've gone into making all these gorgeous animations! Definitely gained a subscriber!
This video is so absolutely incredible, I'm in awe. Your script, your animations, your understanding and explanation of the mathematics... This is a masterclass in education videos.
This is one of the best mathematical video ever. The link with heisemberg principle is just astonishing
Finding the mathematics in natural phenomenon and converting it to our use is the entire science and technology. We can realize why Mathematics is called as queen of science.
You are unparalleled. I have never seen such a master piece on youtube. Please continue the noble efforts. Hope you will make more videos sooner than later . stay no blessed
This is probably one of the best educational videos on youtube. Absolutely superb!
Brilliant explanation, thank you!
This video is excellent, showing many concepts in functional analysis in a very simple and clear way with great visualizations. Please add more!
This has to be the most well edited video I've ever seen. Can't imagine watching this all for free. Thank you so much for your contribution towards Science.
I have been trying to get an intuitive understanding of wavelets for a lot time. This video explained it perfectly!
This video is insanely good! Best one I've seen to explain the intuition of Fourier and wavelet transform! Thank you very much!
Best explanation of wavelets I've seen
You’ve truly done the World a great service by putting this together in such beautiful fashion.
Masterfully done. Mindblowing animation, interesting and engaging topic, clear and well-structured script: you've got it all!
I’m blown away by how clear and informative this video was. Nicely done - it’s an inspiration to communicate this clearly.
This is brilliant. Thank you, Sensai. Your ability to explain a complex mathematical principle in a way that makes intuitive sense demonstrates, in addition to intellectual mastery of the material, a rare and humane understanding of the way people embrace and incorporate new information.
Excellent video! Explained in a really clear and logical way, with impeccable sound design and animations.
This is incredible! Both, the wavelet transform itself and this amazing video explaining it! :D
So it's a localised fourier transform. Cool. Spent 20 years wondering what they were. It was that easy. Thank you!
This video is simply amazing. I'm saving it to rewatch later
This video was amazing, thank you! The ideas seem very helpful in quantum mechanics as well
This video is incredible! High production value and amazingly clear explanations!
Not enjoying this kind of math is almost impossible after watching your beautiful video!
Such an amazing quality of explanations! The small details and animations to clearly show your arguments make this just so much better and superior to anything else!
This video is the best summary of the Fourier Transform I've ever seen, it's given me greater insight into what it even means, and what it's transform cousins are really about.
Просто блестящая работа! Спасибо!
As someone with a background in signal processing: amazing video, explanation wise as well as animations. I wish that would have been the introduction at university. beautiful work!
Totally amazed by the illustrations and the explanation. Excellent job.👏
Single handedly one of the most awesome videos I saw in youtube. Perfect Job. Phenomenal.
Masterpiece, as usual. Спасибо!
Outstanding explanation ever!!!! I have never come across something this clear. Please don't ever stop making such videos please you are helping mankind to grow at multiple dimensions. I support your work from my heart. ❤❤
Wow. This is a full lecture with a very effective set of graphics. Well done! I think I understood most of it and was never bored or overwhelmed.
Great video! Really well explained. Even the parts that is easy to miss or misunderstand is connected in a great way in this video.
Great video, really appreciated the explanations and cool animations! I've been wanting to understand this topic for a while but couldn't quite get my hands on as I'd like, so this served as a great push. I'm getting close to using this technique in my work (not neuroscience though), so this was a nice way of getting a bit of contact with the topic before having to go deeper in the subject. It's funny that I found you a while ago by your videos about Obsidian and Zotero and didn't know you did videos like this one, now I'm definitely subscribed. Keep up the great videos!
The bit at the end where you talk about the wavelet transform's adaptive uncertainty is neat, and explains something I was wondering about the entire time -- how is the wavelet transform different from a time-windowed Fourier transform? This seems to be the answer! Because the support of a wavelet varies over frequency, unlike the static window size of a windowed FFT, you can get more information where it matters.
I was wondering the same thing 🙂
There's two major differences between wavelet transforms (WT) and windowed FTs (say STFT/DFT) that I would highlight, along with their practical implications.
1) First is the multiresolution, stemming from the non-static frequency-time windows (as you've mentioned). Of course, the obvious benefit is that we can collect more time information at frequencies too high to care about discerning accurately instead of simply dropping all that info, which is great for something like any audio processing with a human auditory factor in it, or anything produced by an animal. But the biggest application is that do all sorts of multiresolution analysis like analysing rapidly changing frequencies without having to run FFT several times per frequency or narrowing your frequency as to lose time information. As it turns out, there's a huge amount of nonstationary signals out there in the real world that this perfectly solves.
For example, you need to detect gravitational wave which produce a characteristic chirp. Windowed FTs really struggle with these since the output spectrogram ranges from "some ringing artefacting" to "it's literally smaller than my window size". Maybe it shows up somewhat alright, but you may lose some complex features along the way. But if you look at your WT's scalogram, you get a really nice curve, a distinct and empirically detectable feature.
This actually works really well for all sorts of transients like discontinuities which may go undetected with windowed FTs. This is great for fault detectors. And detecting and characterising heart irregularities or complex brain wave features.
(Technically, there are multiresolution windowed FTs. One of these was a STFT variant called the Constrant-Q transform, developed before wavelet transforms kicked off in full power around the 2000s. In actuality, this is really close to a modern WT, but had certain downsides that come with a less developed understanding of wavelets, like the difficulty in inverting your signal back and some of the jankery that comes with STFTs.)
2) The second is the ability to use different wavelets. This is a much more powerful tool than you would expect. Certain mother wavelets are well suited for certain applications, such as Ricker wavelets for superior seismic processing, or Daubechies for closely spaced features and DWT. A lot of work has been done here, so you have a pretty big toolbox for hotswapping wavelets for your needs. The coolest thing is that you can design your own wavelet tailored for pattern matching your known signal or picking out the set of features you want.
Side note (DWT):
It's worth noting that there are currently two major categories of WTs, being continuous wavelet transforms (CWT) and discrete wavelet transforms (DWT). Most discussions are implied to be around CWT, since it simply works for both continuous and discrete signals, but DWT offers a whole set of other applications.
As you can guess, convolution can be an expensive operation. You are comparing every point of some decently long wavelet to an equal number of points, which is done across every point of the input signal. Sure, you can do some optimisations using FFT itself or adjust your wavelet parameters, but CWT is still generally slow enough that you just can't do certain things with it. Not to mention that its extra redundancy (which windowed FTs also have to some extent) leaves some to be desired for speed and memory performance.
The DWT family of algorithms uses a different approach from raw convolution, instead using a fixed set of child wavelets like a filterbank. It loses its redundancy, limits it to certain mother wavelets, and locks it to frequency-time windows to powers of 2. In exchange, it gains better speed and memory performance in a purely discrete environment, allowing it reach its full practical potential. It turns out that this is often enough (or even ideal) for many digital computing applications. The perfect reconstruction with no redundant information makes it an excellent choice for audio/image compression or performant denoising of images. You'll also find it used in real-time applications where CWT just isn't built for, but require multiresolution that FFT can't provide.
Damn, that was a long comment.
@@MrSonny6155 This was very informative, thank you!
This is the most well-made video I've ever seen! Absolutely great animation and explanation, really high quality. Thanks for your time and effort
This channel definitely deserves way more subscribers. The production of the video is simply marvelous, the way of presenting complex concepts is great too.
Amazing video!
Just a remark about 5:25 : it's not that we lose sense of time, rather that the decomposition gives us pure sine waves, meaning they stretch from -inf to inf.
The relative timing of the different sine waves is represented in the phase of the Fourier transform.
This is indescribably well explained, I can't thank you enough for this feat!
I have been looking into this subject for some time every once in a while, but could never accomplish something that could be honestly called a grasp on this matter. My work is related to non-destructive testing and the analysis of acquired signals, so, Wavelet Transform can obviously very much enhance the way in which we handle the signals, store them and derive useful information from them.
I know that medical ultrasonics is relying heavily on such signal processing, like IQ-demodulation for the sake of Doppler-measurements of blood stream velocity differences. Applied to non-biological targets, we are dealing with different challenges, but Wavelet Transform is bound to improve the way we handle ultrasonic echoes, once we get to harness initial successes on this path.
This is my favorite video on wavelets! 😊Thank you for making such great content and for the CZcams algorithm for recommending it.
What a beautiful video. I learned about Fourier transform when I was at collage, heard about the wavelets, but never really explored it further. Now I feel a bit ashamed for not doing so. This video is a great introduction for the topic. I'm definitely going to learn more about it. Thanks for the video
Thanks for nice content dude.I would like to know how to learn this complex topics in neuroscience, math, programming and have one of the best video compositions (about the visual effects and aesthetics as whole)I ever seen on youtube
Отличная работа! Все понятно и довольно просто, как для введения в вейвлеты. Спасибо за работу!
Thanks Artem for this great video. I got side ttracked with other research work until your latest vidoe popped up. Will be allocating some time in Nov to delve deeper into these topics. Keep them coming.
The quality of your videos is really incredible. The way you explain it is really insightful and the visuals are so detailed. Thank you so much
Wow man what a video! Can't imagine how much work must have gone into producing such a great explanation of such an interesting and useful technique, really really good job. I'm currently doing a PhD in systems neuroscience and your videos like this really make me feel like I need to up my game when it comes to learning complex topics like this. Convinced I'll find the technique or insight that makes my work next level from this channel, I can't wait to go look into how this has been used. Is this all self researched or do you have a seriously top notch neuroscience professor somewhere?
Thank you! I really appreciate it!
Well, I’m doing research in the Laboratory of Extrasynaptic signaling, led by Dr. Alexey Semyanov in Moscow, so I’d say I have really great supervisors ;)
I’m using Wavelet transform in my work to write code for extraction and analysis of theta rhythms, recorded from hippocampus in freely moving mice. (We are currently preparing a publication on this topic, and I really hope it will be out in a few months)
But surely writing a video script requires a lot of additional research.
I feel like only after making the animations and going through the process myself, I can finally understand wavelet transform much better, even though I’ve been routinely using it for almost 2 years now 😅
@@ArtemKirsanov Best of luck to you, looking forward to seeing what comes next
@@ArtemKirsanov same for me in my thesis using wavelets. Your animations are amazing!
Oh nice topic!
Oh, best topic!
It is a beautiful and deep explanation with enlightening animations. Loved it.
Hands down the best overview of wavelets I've ever seen. Good luck with SoME2!
Oh. My. God. If this video doesn't win SoME2, I'll lose my mind.
What did you use to make the video?
Thank you!!
The basis for animations was done in manim and matplotlib python libraries and Blender for 3D surfaces. Then everything was synced and composed in Adobe After Effects
8:00 I don't think that uncertainty in the time domain would mean that you're not sure what a value is at a given moment. Rather I see it as when you take a fourier transform of a signal that is defined over a long period of time, it will have a more specific fourier transform. Think of a cosine in the time domain which translates to a delta function in the frequency domain. This function is defined at exactly one value for the frequency. Therefore we observe that the longer and less determined a signal is in the time domain (cosine's domain extends from minus infinity to infinity) , the more determined it gets in the frequency domain and visa versa. The problem therefore is that when you take a fourier transform of a too short signal, that the frequent domain will start to show less specifically which frequencies are contained. That's the trade off you need to make.
@pyropulse It have to do with uncertainty if we are taking about uncertainty as given in Information theory tho. Its not exactly the same thing as we consider uncertainty in real life, but what it really says is about information entropy.
It's very similar to the waterfall spectrograms in audio software (spectrum analyzers). It just shows you the Fourier transform of the X recent milliseconds of the audio signal, so the frequency definition of a pure sine wave will be limited to the inverse of the chunk length in time. Wavelet transform does basically the same in a mathematically smarter way (convolution instead of Fourier transform, though they are very related) using the optimum window shape, which allows for the "dynamic" trade-off between time and frequency resolution. In a simple waterfall-plot spectrum analyzer this trade-off is fixed and defined by the chunk length.
Wow, this video is fantastic. Beautiful colors, visualizations and sounds. Never stop making these videos 🙂
Артем, офигенная лекция. Большое спасибо.
Good introduction to wavelets! But you give the Fourier transform too little credit :).. It DOES contain information about the time sequence/"order" of the frequency components, after all it's a "dual" representation of the time-domain signal, right? That temporal order is contained in the spectral phase - and that's what most people miss about the Fourier transform, since they only plot the magnitude (or power) spectrum but forget about the phase and lose half of the contained information (which happens to be about the timing order).
No phase in frequency space?!
You're absolutely right - there is definitely a very important notion of phase both in the case Fourier transform and Wavelet transform (computed as the angle of the resultant complex number). I didn't really have the time to mention this in the video, not to make it too overwhelming.
But the Morlet wavelet, being a complex function, has amazing capabilities of dealing with phase of the oscillations. One example of such is the Cross Wavelet Analysis, which allows us to compare two signals and study the relative phase shifts.
Thank you for pointing this out!
Finally a video that explains Wavelts in an approachable manner. Thanks a lot!
Awesome video! Not only the explanations are very clear, but the animations are of a stand-alone quality. I had already scratched the surface of wavelet analysis when I wrote a paper about signal processing and the differences between the Fourier and wavelet transforms as part of a school project, but the visuals of this video gave me insight on what was previously blurred (especially the complex representation). Thanks a lot for your dedicated work Mr. Kirsanov!
How about wavelet isit orthogonal matric, like DCT
If you are using discrete wavelet transform (DWT), then the wavelets of different scales indeed form an orthogonal basis.
The key difference of DWT, compared to the continuous wavelet transform (which I showed in the video), is that the scale parameter (a) can be varied only discretely, to make sure that wavelets of different scales are orthogonal.
It depends on the particular application and what type of wavelet you are using. For example, the Morlet is a continuous one, while many other wavelets (such as Haar, Daubechies) are used only in the discrete case
Thank you so much Artem!
I thoroughly enjoyed that this talked about convolutions. I always try to connect useful new concepts to things I've previously learned, and this video really helped with that process.
Truly outstanding video! The best explanation of wavelets I have ever had...by far!
wow, this video is the probably the best video out there that talks about wavelet transform. The visuals, the explanations, all are top tier. It not only explains wavelets but explains the need for wavelet transform. Great Job.
This is the best video to explain wavelets. Thanks!
Awesome video! Every part of this was just perfectly explained and visualized
This is absolute class, thank you so much for the work you put into this. Every part of it was totally clear.