My Problem With (Most) Spectrograms
Vložit
- čas přidán 16. 04. 2024
- Get Minimeters: minimeters.app/
Spectrum Reassignment articles:
en.wikipedia.org/wiki/Reassig...
people.ece.cornell.edu/land/P...
arxiv.org/pdf/0903.3080.pdf
hal.science/hal-00414583/docu...
Follow Au5:
TW: / au5music
FB: / theau5
SC: / au5
IG: / au5_official
Store: au5.bigcartel.com/ - Hudba
I love that wave candy is in the bottom left corner just dunking on these inferior ones the whole video 🤣😅
Hey, audio plug-in dev here! I'm currently working on a plug-in UI component library and I'll definitely check out how spectrum reassignment is implemented. I've always wondered how the really clean spectrum analyzers and spectrograms that retain so much detail in the bass frequencies work. Thanks for this awesome video!
Heh, I just started playing with the vst3 dev kit, coming from audio DSP, mostly in research. I guess this is something I might want to try as my next project. Possibly interesting to learn Vulkan.
all i can say is check our airwindows, he stream his coding
Let's goooo
@@Mefistofyprobably better off learning CUDA rather than Vulkan if you just want to parallelize stuff. Numpy (jax) are close to matlab if you know that and just want something more convenient.
The thousands of lines of boilerplate to create a single triangle in Vulkan is not really worth the time unless you are a big team creating a game engine or something. For OpenGL its only two hundred but still rarely worth it. Use libraries!
Ooooh?
"today i'm not gonna do a tutorial"
*does a tutorial*
lol “Today’s tutorial: how to get frustrated over sub-par spectrograms”
@@au5music czcams.com/channels/u5jfQcpRLm9xhmlSd5S8xw.html
@@au5music www.youtube.com/@LofiGirl
@@au5music czcams.com/video/9UMxZofMNbA/video.html
@@au5music www.youtube.com/@ChillhopMusic
5:25 this is actually quite literally the uncertainty principle. In quantum mechanics, the position of a particle is related to the momentum of that particle by a fourier transform, just like how an audio file's waveform is related to its spectogram. The tradeoff between frequency resolution and time resolution in these VSTs exists for exacty the same reason that there is a tradeoff between knowing where a particle is and how fast it's going in our universe
i know, its crazy!
That is so cool
Its more than that in the quantum world. Wee don't affect the observed object here, its just a tradeoff. In quantum systems your measurement actually collapses the particle into a state of one variable measured which causes another variable to become uncertain probability. It occurs with spin measurements too, due to the effect of things at scale being affected by measurement and observation. Spin doesn't use fft yet its still operating under uncertainty principle because what we are measuring is collapsed into a condition of that variable due to being fundamentally so small, plus particles can not have continuous values of spin so measuring in new direction orients the particle in a new way
. Here its continuous and doesn't change the state by observing its just our knowledge of the continuous state. It definitely just a classical uncertainty.
@@jorgepeterbarton it’s still about wave properties, just with the added issue of only getting one measurement. if the wave isn’t affected by the measurement, each measurement has this tradeoff, but you can just do it more than once
@@jorgepeterbartonthat's only one interpretation of QM though. Not everyone agrees on that interpretation. What we see does appear that way, but there are other possible explanations for that behavior beyond the wave function collapse theory.
I'm a developer and I've done a lot of signal processing work. Reassignment is not a secret, and neither are the tradeoffs. It's doing multiple DFTs and using the phase derivative to estimate the frequency. It takes a lot more CPU, and all those estimations turn into error when you add harmonics or multiple sinusoids. It totally falls apart vs. white noise. But if you want a sharp spectrogram of a monosynth, definitely use reassignment. Spectrograms are kinda like compressors. Everybody just wants them to work, but dialing them in is very situational.
This needs to be higher up. In case another devs want a reason as to why it hasn't gotten more research (and widespread implementation in VSTs) already.
Shoutout to Direct for creating MiniMeters! Amazing producer
For Ableton and other daw users it’s basically all we’ve ever wanted for an alternative to wave candy. Also i have a Mac on 10.14 still, and he still supports the OS, what a king.
He didnt create it . The external vst builders dis…
Minimeters was perfect if the osciloscope wasnt so small
@@SjarMenace what do you mean by small
@@LarsWilms you cant change the osciloscope like oszilloscope from ben schulz
Just a note, Image-Line didn't discontinue the VSTs to restrict people to FL Studio, but because we couldn't justify the maintenance effort required for them. All of the VSTs had significant bugs which took ages to get fixed, because the 2 devs who could work on them were Miro, who does all of the UI stuff, and Frederic, who is the lead dev for FL Studio.
The plugins made barely any money compared to Fl Studio and we didn't want to continue selling something broken at the time.
IL has stated on multiple occasions that they were planning to re-introduce the VST plugins, but they just never have.
I hope they might someday. Especially if it enables compatibility with the DAW project format or importing into Bitwig in future or something.
I fully agree with the sentiment though.
Your first plugin SpectrAu5
⚡✌🏻
I rarely comment on videos, but as a musician, mathematician, DSP lover, and general fan of your videos, this information has been too absolutely mind-blowing to NOT leave a comment. I'm definitely going to check out minimeters ASAP! Love your music and can't thank you enough for the awesome videos!
Honestly the person who coded Wave Candy (Didier Dambrin aka "Gol") is just a Savant, so I'm not surprised to see it here. I mean all the major FL Plugins were coded by him; Sytrus, Maximus, Harmor, etc. It's a shame he's no longer really involved with the company, but they are still delivering solid content.
He's happily playing with Lego nowadays :)
I highly respect the dude, but I think it's a good thing he moved on eventually, since it allowed Image-Line to work a lot more on user requested features. He laid a really really solid foundation with a really unique singular vision.
I use Wave Candy on my default template. Glad to see someone showing the love it deserves
Interesting to hear this. You don't see many videos covering this topic!
For those interested, the FFT groups ranges of frequencies inside "bins" (basicaly ranges) and those bins are linearly spaced in frequency. Since when dealing with frequencies we tend to use log scales, you're basically zooming in at low frequencies and what you see there is "pixelated". You'll have the same amount of bins between 0-1kHz and 10-11kHz. Increasing the FFT size increases the amount of bins and thus the "resolution" improves, but the FFT size is correlated with the amount of samples the FFT uses.
This is an EXCELLENT video. this has always driven me nuts and I have never known there was a better alternative until JUST now. THANK YOU
Really weird how long I've been sleeping on Wave Candy even though I've known about it for ages. Never knew how good it actually was, just took it for granted.
I have the same, lol
The frustration in finding a good spectrogram is so relatable. Like you realise how valuable simple and powerful plugins like Wave candy and Harmor are once you switch to a different DAW.
As always, thanks for the valuable insight Austin.🙌
Simple? Harmor?
@@TollsterMensch Yes. Unlike other plugins, Harmor is very good in coming up with new and unique sounds.
Dang. Honestly shows how much IL plugins are ahead of their time in some regards but also can agree with that they should be more open to where you can use it. :/
I've been waiting for a VST version of Vocodex for nigh on a decade 😅 Someday my prince will come
@@rynabuns I think I’ve actually run vocodex on ableton before
@@rynabuns what's so special about Vocodex
There's a recent reassignment method called Synchrosqueezing that would be dope to see implemented in spectrum analysis. Thanks for bringing awareness to the issue!
Oh I believe I’ve seen this in Steinberg Spectralayers
This has been driving me crazy for years! I just picked up minimeters a few weeks ago, didn't know there was a sharp setting. Absolutely clutch 👌 Thank you, stay lit 🔥
@Au5 if you have any interest, i threw together a really simple M4L device which will hide the IL Wave Candy vst when the Hint/Info box isn't showing. This way it automatically goes away and gets out of the way when you are scrolling through the library and such on the left side of the screen. It only appears when the bottom "Devices/Midi/Audio" area is visible, which is when the Hint/Info box is visible.
JEEEEZZZZ,,,, you are just exactly when and where i need you to be. I sat down with the intention of buying minimeters last night and just when i did i saw this video thumbnail. Soooo,, thank you Mr, Perfect timing.
No wonder why I couldn’t find that spectrogram. This was very informative thanks for sharing!
Thanks for making us aware, I always figured that mind of low end resolution must simply be imposible, didnt even bother looking
Finally someone is talking about this
One more thing! Edison is even better than Wave Candy!
Along with enhanced frequency, it has 'enhanced time' (shortcut: T) which totally sharpens up the transients, at no expense
pushing the industry forward once again - inspiring being and great sensei
Music producing turns back to be science. Finally. Thanks for this video, can't wait for more!
Haha there’s so much science in signal processing used to make good-behaving filters and oscillators for synths! Well, basics of it are quite simple but when you try to, say, reduce aliasing, or ringing, or unwanted tails, or something else, it amps a lot.
12:32 you designed that sounds to look like a bifurcation plot in a spectrogram? crazy
also... watching this video just made me realize a link between the content of THREE courses I'm taking at school. Fourier transforms, how they are a more fundamental cause of the uncertainty principle in QM due to position and momentum being conjugate, and how this sense of being conjugate is like the sense of conjugate variables in the Hamiltonian formalism of classical mechanics. And I have no business learning about audio production tools, I just watched this video because I like your music. Insane.
that's actually not hard! there are plugins that can generate sounds using pictures, and the picture is shown in a spectrogram when analyzing the sound
e.g harmor's image synthesis function
12:31 isn't that a famous math fractal with a chaotic behavior ?
Definitely, that's the Mandelbrot set as a bifurcation diagram, Veritasium has that one video on it
@@jinto-music oh thanks for info, i thought it was a different set, no related to the mandelbrot set. ngl it sounds pretty good btw
@@kinka16 it can actually arise in many other ways (though they could always be related to fractals in some deeper mathematical sense)!
Example: Free-swinging pendulums generally behave periodically - if you keep track of the pendulum's position over time, as well as its speed over time, you'll find that its position and speed return to the same values after every natural period passes. Now if you drive/push the pendulum in an oscillatory fashion at a different frequency than its natural frequency (think pushing someone on a swing but NOT necessarily in the way that maximizes their speed), you can introduce chaotic behaviour; by increasing the driving amplitude (how hard you push it), you will find that the behaviour tends to become 'more chaotic,' corresponding to the pendulum taking a larger number of its natural periods to return to the same position and speed values. Increase the amplitude enough, and eventually the pendulum will NEVER revisit a single position and speed configuration value, i.e. its motion is completely unpredictable. The bifurcation plot arises when you plot the driving amplitude on the horizontal axis and the position value(s) it takes on at every period on the vertical axis. At small amplitudes, it is periodic in the normal sense, revisiting the same position after every natural period, so the plot is a horizontal line here. Increase the amplitude, and eventually it takes two periods to revisit the same point, meaning every even-numbered period it will visit one point, and every odd-numbered period it will visit another point - the plot here is two parallel horizontal lines. Increase some more, now it will visit four distinct points, and so on. The plot splits and splits (bifurcates), generating that distinct shape.
@@gursach4435 yeah i did a lot of mechanical engineering and i have a lot of interest on science, physics, but im not too familiar with in some way, i really need to go deeper and somewhat experimental. it is absolutely facinating how simple things can be this chaotic !
@@kinka16 Check out the logistic map, it’s very simple x ↦ a x (1 − x) where the upper bound for a is 4 and I forgot the lower but it can go a bit below zero. The magic is that this is a discrete thing where you change x at discrete times; it you try to write an analogous continuous system, the chaotic behavior for large values of a doesn’t happen anymore (there’s a theorem somewhere that for a continuous system, you need no less than three dimensions (and so, real variables), like in the system that gives the Lorentz attractor; but discrete systems are weirder and so even this simple quadratic thing goes).
Really hope Image-Line opens up their walled garden. Currently an Ableton user but I used to use FL Studio and I really miss some of these plugins (Wave Candy, Harmor, Maximus)
Harmor is just incredible.. mind blowing. Zebralette3 seems promising
Agree
Harmor is a stand alone plugin
@@stereokuuji Harmor only available for windows
@@stereokuuji unfortunately Image Line discontinued updates for the standalone VST a few years ago, so you can currently only use it within FL Studio unless you happen to already have an older version and you're not running on Apple Silicon
Dang I always wondered why Wave Candy looked so good lol. It’s awesome that you got the minimeters dev to add it though! Love that tool
legit working on my first DSP project and this seems very applicable for what I'm doing. Thanks for spreading the word
This is so important, thanks for sharing. So glad minimeters added an enhanced frequency option
I love your videos so much man. I literally just bought Vision 4x. 30 minutes ago and came across this video...... wish I viewed it before. Also I am one of those who have also wondered what spectrogram you use in your videos.
Loved this video! I've always been unsatisfied with the low frequency precision of Izotope Insight's spectrogram, so thanks for recommending the alternatives!
its about time more people start to give a shit bout the spectrogram standards, Wavecandy is ahead of the curve, seems like a pattern with Image Lines plugins, they always been ahead of the curve, insane stuff!
So happy that you pointed out how this is similar to the uncertainty principle. The mathematical reason is exactly the same for both. Unfortunately, better plugins won't be able to get rid of this tradeoff :)
Very coherent message. Thank you for bringing awareness.
Most of them not being able to display the side channel is my main problem. It works if you set up a utility before it though.
Came here to say this
there's a reason though: the side channel can phase cancel the mid channel depending on the information it has
the same thing can happen when you sum the left and right channels to mono. the most ideal solution is to have separate spectrograms for each channel
i think you can just split the mid side into two tracks
There was a very old iTunes visualizer plugin called SpectroGraph that displayed stereo information in a spectrogram by coloring the L and R channels red and green (yellow for combined/mono). I never saw such a thing implemented since and would also love to see that as L/R and M/S analysis spectrograms along with enhanced frequency. Would be the end-all for me.
Absolutely love this video!
Its incredible how he got the best intro i've ever seen with just 5sec
thank you so much I hope this helps devs achieve a better standard for this. aguante image line!
The reassignment method is cool, but it's not the only solution worth considering here. Two common DSP methods:
1) You can estimate the true frequencies of peaks using splines that are only calculated near maxima in the spectrum above a certain threshold. This is pretty computationally efficient and is used in a lot of algorithms that extract partial data from audio (like pvanal in Csound).
2) You can take a wavelet transform. You're right about the uncertainty principle between time and frequency. The best time resolution and worst frequency resolution is the untransformed signal (just the list of sample values). The worst time resolution and best frequency resolution is an unwindowed DFT across the whole signal. Spectrograms are a series of shorter snippets of the signal Fourier transformed; the longer the snippets, the better the frequency resolution and the worse the time resolution. There's also a whole different class of transforms called wavelet transforms that gets the same excellent frequency resolution at the lowest end that a windowed Fourier transform would have and good time resolution at the high end (with a slight loss in frequency resolution).
It's honestly frustrating how few spectrograms give you the option of different wavelet transforms instead of the short time Fourier transform.
Couldn't the FFT window be slided a few amount of samples instead of the length of the whole window? I guess you'll be doing a higher amount of transforms and using more processing power, but it could improve the time resolution. In DSP courses we've also studied zero padding/stuffing. I'm not sure if this could be applied in some way here.
@@filipenicoli_ Sliding the FFT window is regularly done, but it doesn't improve frequency resolution.
@@adamluhring2482 but wouldn't sliding improve time resolution for bigger fft sizes?
@@filipenicoli_ Oh sorry, read that as frequency resolution, not time. Still, the answer is no. For a window of a given length, the FFT will always be the worst time resolution. The fact that those extra time steps can provide practical value is because human hearing works more like a wavelet transform; we'll never actually hear a sinusoid of infinite duration. But in the sense meant by time-frequency uncertainty, you can't beat that limit
as an electrical engineer with a focus on signal processing, I’m honestly blown away by that wavecandy “enhanced frequency”spectrogram. how tf are they doing that? this is really hard for math reasons…
*edit wow! thank you so much for mentioning the papers! you’ve given me something to geek out about for the next few weeks 😂
great video as always!
Wave candy in Live myself too. I didn't know IL stopped the non-FL version. I shall cherish this old version. Good analysis
Much respect as usual❤
As someone who works as a sound analyzer under 3khz we have to change our sample size many times throughout an analysis. If I am working with board-band frequencies we have to change our spectrum output to get more detail as opposed to a narrow-band frequency source. The short of it is don't rely on a single processer setting and move through the options provided to get the detail you want.
ive been annoyed about this for years! thank you for this video and spreading awareness
i was going to sleep but i had this call from Austin !
Thanks, just implemented this into a tool of mine👍
Thanks for the links. I'll check them out
Vision 4x devs right now 👁️👄👁️
great explanations!
Incredible. I have 4x and thought it was the best thing out there. Seems IL Wave Candy already nailed it, although sadly obsolete. It's amazing that mathematicians and scientists have found a way to analyse the phase to get super accurate frequency measurement using FFT, with magnitude as a trade-off. I'm going to get Minimeters while it's still in beta. Looks great.
This is very interesting. I'm curious about your wavecandy preset to get it to look like that, because the default spectrum view does not look quite that sharp. Thanks!
The minimeters one gets a higher resolution depending on the windows size, i like to keep it vertical, with a really wide window size, its on my second moniter so space is never an issue
Fantastic.
Electrical engineers: Why would you need that much performance?
Sound designers: I need it.
Thanks for sharing, peace
thanks for sharing
Extremely helpful. I haven't been using spectrograms at all and I really should because I lose sense of my mixes so often and so quickly
Now I gotta figure out how to get the wave candy version for Windows since I'm on Ableton too lol
Hey Au5 you should also check out the Wave Candy vectorscope so you can fine-tune your stereo shaping! I have been using an FL Studios' patcher patch that I made to listen to melodic dubstep for the last year and it sounds incredible. I also combine it with the convolver to make it sound like I'm listening in any venue I want.
I have, but its not nearly as wild as Signalizer's vectorscope. It's got a time dimension so its actually a 3d oscilloscope.
...And loudness is represented by the brightness of the image
Great video Au5. Love the insights you have.
And yeah FL is definitely infringing on our freedom by limiting their software to their daw
that alien weapon sandwich at the end was yummy tasty
So happy rn, being a Windows FL user. Love being the unsung hero of the music game lol
It seems like vision 4x makes up for the low FFT resolution by having the horizontal bar graph that shows the volumes of frequencies
FL Out here flexin
Wavecandy enjoyer for years. I was grateful how good this is, but I didn't realise how crap the competition was lol
Preach it brother
thank you!
This is a cool video - heads up that the reassignment method spectrogram is available as an option in iZotope RX, but not yet implemented into Insight.
Also custom color gradients would be the cherry on top
Omg, I've started doing sound design and using spectrum analyzers in a huge part thanks to you, especially the spectrum analyzers, and I've been trying Vision X4 and I am literally so disappointed by it. I didn't even understand why I was not like what you do in your videos until I saw this
12:34 omg it plots the logistic map!
I thought I was the only one that cared about how Wave Candy was the only spectrogram I knew that did such a thing, and when moving from FL to Bitwig, I was seriously missing having it.
As an FL user, thanks for saving me money by sticking to Wave Candy
I think alot of it was that it just wasnt heard about. It may also be really difficult to implement (fft analysis alone isnt easy. some very spooky binary manipulation and instruction set utilization going on).
I've implemented fft spectrograms into projects of mine, and yet have never heard of spectrum reassignment, so thanks for bringing this to my attention. I was planning on implementing wavelet analysis, but theres not alot of info on that algorithm so maybe this is a better alternative instead to get better resolution.
btw, if you can find a wavelet transform visualizer, give it a go. It may be similar to spectrum reassignment in terms of quality, as it essentially breaks past the fft uncertainty principal by cheating lmao. You get extremely good low frequency information and really good time accuracy because it treats each frequency as its own independent calculation
🤩 1024 FFT FTW 🩵
@ 12:34 bhahahh that's the chaos map, but rotated 90 degrees. cool math easter egg!
Id love harmor on mac so much. Maybe there is something we can do? If enough musicians sign a petition to bring it and other vsts to mac too.
Nice video! I just found a new purpose for Minimeters (i only used it for making twitter clips lmao)
Thx for being a voice we need more vst be adressed such things. Inhope they will get canceld
perfect
nice bifurcation diagram at 12:32 :D
It looks like it's performing a peak picking on the fft following up with phase correction on the peaks, and then is just graphing those results. (Hence why it's not showing side lobes around the peak for the improved spectorgam.) I wonder if it's bases off of quadratic peak interpolation, or if it's more along the lines of a phase vocoder.
I'm curious if this can also be applied to all fft based processing plugins to improve their sound?
I knew what this would be about when I clicked on the video, but it was still surprising to see what spectral reassignment looks like in real time!
Still I doubt it's if very CPU intensive. It's basically just using instantaneous frequency to map to another frequency space with better resolution?
I love that Ableton skin, is it from Livethemes?
Hey Austin,
Would it be possible for you to do a tutorial on how you use Morph? Only a 2-3 min video would be great!
Mathematically it's impossible to increase resolution of low frequencies without sacrificing time resolution.
But what you described seem more like a UI issue, where the detected frequency isn't communicated to the user well.
Using morlet wavelet transform, it's possible to get a better balance of time and frequency resolution
Some kind of compositing of FFT sizes would probably work decently.
How do you get wave candy? I can’t find it, besides being part of FL studio, but I want to use it with Live.
Is there a way to get Wave Candy in Ableton on Mac?
Can you do a video on how to read wave cand/ mini meters? I get that it shows a graph, but how can one benefit from using it?
Image Line needs to bring the Juice Pack back! Even as an FL owner who uses it 50-75% of the time, I'd drop more money on the Juice Pack in a heartbeat.
I've been building my own M4L one the past 3 weeks. And my goal has and still is, to create a IL Wave Candy style one. With a few extra Ableton specific features. Random to see this video come out today
anywhere we can follow your work? I'd be super interested in buying a M4L device that checks all the boxes
@@ridethewavz I will update this video when its ready. I'm hoping to figure out this spectrum soon.
Any idea why the sound at 12:33-12:34 produces a spectrogram like a Bifurcation diagram of the Ricker model or logistic map?
That's my battleground!
I got my minimeters copy a month ago and I think I will never switch to anything else ever again.
Wonder when the skin option will be available tho
I've been using spectrogram's a lot back in the days. What I realised is that I've been focusing more on image than sound so I stopped using these.