Spectral Un-rotation : Effects and Watermarking
Vložit
- čas přidán 13. 05. 2024
- adding effects before un-rotating the spectral image produce some interesting results
this is the third video of the spectral rotation tutorials
previous video on how to achieve 90 degree spectral rotations : • 90 degree rotations : ...
in the first spectral rotation video i mentioned it was "impractical" or "only for looks". now i am enlightening myself with other experiments, that are explored in this video.
applying an effect before un-rotating the signal can yield very interesting results.
additionally near the end of the video, i explore how one could watermark a song with a vocal phrase (albeit i did in not the most optimal way ahaa~)
0:00 intro
1:10 distortion
3:11 frequency shifter vibrato
4:04 pitch vibrato
6:22 delay
8:40 reverb
10:26 using an eq curve as a volume envelope
13:21 watermark experiment
Has anyone tried this w cubases stock convolution plugin? Its not working for me.
Its clearly doing something but it neither sounds nor looks anything like whats happening here. Great video btw!
You discovered a whole new paradigm of weird sound design, it's insane. There's so much you can do in between a sandwich of spectral rotation. Ahhh this is exciting
I wouldn’t say he discovered it, but I agree it’s not popularized and a very cool concept
@@orthodynamicstereonails I would say he discovered it, maybe just not first
@@dominobuilder100 that’s what discovering is to me, doing it first lol
if two different people, unknown to each other, discover the same thing at different times, is the person who found it first the only one who "discovered" it?
WrangleLine!! I'm glad you found this vid. I'll be spectrally-rotating all your future music looking for secret watermarks. lol
I love absolutely everything about this
For some reason, I thought you'd end up here 😭
love seeing u on all these niche vids
The man himself!
10:17 flashback to the time i said “reverb is kinda like blur for audio” and every musician in the room acted like that was a crazy comparison
this would be pretty clear if people were more educated on reverb algorithms. blurring our perception of micro-time is literally the purpose of the basic components like modulated delay lines and all-pass filters. then those in combination as the whole algorithm blurs over macro time. its almost self-evidently true.
i mean you're not too far off! more accurately a LPF is like blur for audio though. i mean nothing is stopping you from applying image filters on 1d data, you should try it
reverb is like a spectral blur
additional things to consider too is that a lowpass is a blur in the time domain
cus you can lowpass images in photoshop, just simply by using gaussian blur
a high shelf is sharpening (photoshop has a highpass effect too btw)
you can actually treat an image as a "signal" with fourier transform to do that
@@diegoswaddipong6196 i actually do a lot of technical shader art stuff for fun and have been wanting to code audio stuff and ive had the idea for a while to literally just transform audio visually because i already know how
@@idiotinium yeah ive had the idea to apply some video knowledge to audio and mess around and i feel like i could design some nice reverb by blurring different pitches/hues at different rates
this is the most invested i've been on a youtube video series in a good while, that's some CRAZY sound design
YEAH same lol
Ok, hear me out. that eq-as-volume section got me thinking, technically sound doesn't just have the 2 dimensions of time and frequency, but also the 3rd dimension of intensity/volume. so clearly, the next step is 3-dimensional rotations
oh shit
no, cuz we measure amplitude against time or frequency. Ie, the Y variable is amplitude, and the X is time or freq.
You can measure amplitude against time AND freq using wavelets. But the math is too intense for me to teach myself.
Well, it's a 3d graph, but unlike the 2d rotation, you would lose the quite important quality that there is a unique amplitude associated with each (frequency, time) pair
What about phase?
LMAO okay hear me out... time, frequency, phase, and amplitude for 4 dimensions!
1 temporal dimension, 2 spatial dimensions and 1 modular dimension
broooo no way. the frequency shifter automation was unexpectedly pleasant when rotated back
Interesting results! I am tempted to try coding a little tool that can do the rotating quickly offline, so you don't have to record it every time.... I'll see if I have some time this weekend
👁 folowing
PLEASE DO
YES PLEASE
report back when you do o7
yoooo nice that would be awesome
This needs to be a plug-in
Working on it!
@@Kaixo hows it going? thank you!
I understood so much more when you pulled out the khan academy explanation for this one
16:15 this is halfway between watermarking and steganography. Quite cool.
this is an absolute gamechanger for experimental artists
seriously yaaa
That technique will surely reveal another ton of "secret satanic/subliminal messages" hidden in songs :)
6:55 reminds me of FEZ, and nice that you explained this effect.
This series is no longer sound engineering. It's sound science.
virtual riot racist samples
you were quick to notice
I don't think I'm able to grasp just how important these videos are. Both the processes and the outcomes are insane and can be applied in so many creative ways... Thanks for posting this
3:29 YO
OMG, this gets me so much closer to my life's goal of: Projecting an audio spectrum onto the top of the the surface of a 3D torus, rotating the torus in the poloidal direction 90 degrees and then reconstructing the spectrum from that (inverse projection), and selling that as an Ableton plugin.
I love how you where like "don't know for what cool stuff this could be used but it's funny that it works" in the last video and now uploaded this. I also don't know if anyone ever did that but I believe you just discovered and displayed a whole new dimension of new effects and sound design. And that you are sharing this technique with us all makes you the pioneer of it imo. Keep it up King 💪
0:30 Okay, where do I need to sign up to fund your linear algebra journey? If you’re making these connections without even studying, I tremble at the idea of what you could do with all that additional knowledge
that is true, i wanna be learning these things cus i feel like i could get a ton of new intuition if i knew some more linear algebra. like wtf is an eigenvalue?? oh it does that to a matrix but like what does that mean? y'know what i mean
like if i truly learnt about it deeper i could definitely imagine it better for sure
When you showed the EQ acting as a volume envelope I was thinking the inverse would be interesting - by muting sections of the clip and then unrotating you get an infinity dB/octave filter.
yeah pretty much
if you do it on immediate though, it might click and so create a resonant frequency *after* re-rotating!
that is, you should try not not to make it immediate but rather a small time change in volume, so that really won't solve that infinitry dB filter
however, it's one way to do it
@@victorfunnyman True. Can't beat Fourier I suppose.
@@jhonbus I mean, if you have exactly what you ened there
really excited to play around with some more phasey stuff with this method - it's like having the frequency information of every instrument interact at once. super interested to hear the effects of dispersion and vocoding etc on the reoriented signal
This is the most interesting discovery of digital sound processing I have ever seen. This could be revolutionary.
the delay might be less annoying if you turn the feedback down and modulate the delay time a bit
its like a softened bitcrusher yeye
what a nice time to be alive
I love your visualisations in paint. I appreciate it!
This absolutely changes the fucking game, i thank you for these discoveries
I have no idea what is happening but I found it hella interesting
very high feeling this tech gonna blow up this is so alien and sick
when you invert time and frequency, you're entering in the cepstrum domain (instead of spectrum), in wich frequency become the so called quefrency, the phase turns in to the saphe, and so on. it's interesting how you found a way to do it with common tools, nice!
i've heard about a quefrency cepstrum before but i never really knew what it was about aha
@@idiotinium now you discovered!
Whoa. Okay, cool. I wasnt aware of this different noccept
That watermarking is incredibly cool, and sounds cool to boot!
Keep this up and you’ll be destined for greatness
ive seen a few producers watermark their track in a diff way, where the waveform is their name. ive seen Soltan do this
oh neat ive tried that too but with a xy oscilloscope
Ok watching the other vids i didnt think there were too many use cases, but i was so wrong. You just changed like the next month of my life lol.
this is actually groundbreaking information for music!
this is so unbelievably fucking cool
somehow the way you presented it was super intuitive, and I was able to predict how the majority of them would sound before even hearing them(the reverb sounds so cool btw) what level of autism does that make me
high level autism bro
thats an absolute win right there
i’m completely in awe
That FSV at 3 minutes in is beautiful. Temporal madness 😅
This is so fun and interesting, subbed!!
this whole series is on par with nasko figuring out how to do pitchmap with a bunch of EQ's in phaseplant. just pure next level DSP wizardry, love it.
some of the effects remind me of classic FFT fx like NI spektral delay & dtblkfx, but a lot wilder. especially the freqshifting.
yoo i love dtblkfx
New to sound processing but this approach is making things so much clearer for me.
I'd be curious to hear what mp3 compression does on a rotated sample, in between rotations. Also, applying effects on non 90 degree rotations
the problem with non-90 degree rotations is you lose information for everything rotated below 0hz and above nyquist. but should still work, just extra lossy
it would just be a more lossy result, but could have a cool quality who knows
yeah non 90 degree rotations get cut off but you could shift them up and down to get extra coverage or process them above 48khz while shifting them above that range for more space that would be cool
@@idiotinium yep I’m currently trying to explore freq shifting content up in a 96khz sample rate, rotating, then shifting back down. trying to see if I can make an all-in-one max msp device for it
My current issue is how lossy this is - I’ve found that the longer the convolution sweep is, and thus the freq shift time, the higher resolution I’m getting. Maybe the convolution reverb I’m using sucks, or there’s some discretized jumping in the frequency shifter automation
Using a convolution of 16 seconds gives a clean rotated signal, while faster times sounds ring modded, highpassed and lossy. The only issue is it takes well over a minute to fully rotate, process, and unrotate something
"would be more lossy" Guys, this is just because you are thinking in squares. What you need to be doing is thinking in circles. Just what the hell I mean by that, I'll leave as an exercise for the reader.
Very cool stuff! I've done some similar effects by running an FFT on an entire audio file, applying processing, and then running the FFT again at the end. But it's awesome that this way works completely inside your DAW. You can get really nice ambient results if you use the rotated signal as the modulator of a vocoder with white noise as the carrier. It destroys most of the rhythmic information and just smears the pitches out across time.
oh nice, that kinda sounds like paulstretching right?
THE FUCKING VIBRATO ONES HOLY FUCK ‼️‼️‼️
This is just cool the entire way through
Everything about this is cool to my brain
WAIT THAT'S SO SICK IMMA SEND THIS TO MY FRIENDS RIGHT NOW
Actually bonkers. Going to do research on this myself!
new stuff discovered, very cool
thank you this is awesome
This got me thinking, Harmor can do image synthesis, as well as converting audio to an image, which you can then edit and re-import. I wonder how effective this can be to achieve similar results.
not that effective since harmor works radically differently to a simple fourier transform. yes it uses it but theres a lot more going on in there, theres only 516 partials
I just started playing Noita (the video game) and the CZcams algorithm sent me here because of your folders there
this is so cool
adding effects and then unrotating is such a cool idea, i feel like (if possible) doing this to a 45 degree would give interesting results!!
lol you can hide a message in one of your tracks like this. Imagine someone discovering that in 20 years and losing their shit
If you limit/compress the 90deg signal, you basically get spectral compression when you rotate it back
oh yeah true, that's a really good way to flatten the frequency response of a piece of audio
6:21 👀
👁️
14:45 "i did some mining off camera", that shit was sick
lmao yeah i paused the recording cus i thought it would have been a lot to edit but it wasn't that much so i should have recorded it
Marvelous
that's awesome
the rotation saga continues
I'd try cranking up the bit depth and sample rate to see if you can get greater fidelity from the rotation scheme. I'm going to have to try this...
Would the nyquist frequency issue be fixed with a higher project samplerate? Or is that an issue with the convolution reverb and its resolution?
Other than that loving these videos so much ❤
it could be a factor of both and other issues too
yeah i do think higher samplerate would fix some issues? potentially, but i think the frequency shifter i'm using has a built in 24k lowpass anyway so i doubt it'd help in my case specifically.
other issues i've spotted are :
the convolver isn't perfect cus it cuts off abruptply at the end making a click, and it starts at 1hz instead of 0hz
the frequency shifter only has 5khz of range so i'm stacking multiple of them in a row to reach 24khz, which add some weird phase inconsistencies with each one in the stack (i also caught them randomly ringing out at particular frequencies)
Wow
This is so cool
This is like Au5 stuff but cooler 😮
this is crazy!
will be stealing. thank you very much
I wonder what would happen if you time stretched in the rotated form before rotating back. It would appear maybe similar to the reverb one when thinking about it, but I imagin it would sound quite diferant. OFC if you timestretched x2 you would only have upto 10Khz (now being doubled to 20Khz), but might do something cool sounding... with the rest reflecting back down from nyquist maybe.
Also if you time squashed something, it might produce better results, making everything fit in the freqency band. I guess it's basically a pitch shifter at that point, except the freqency spectrum would be shifted by either compacting or exapnding the freqency range, rather than shifting the formants or however other more traditional freqency shifters can work.
On that same veign, if you pitch shifted in rotated form I imgine it would just delay something or make it happen quicker, adding what ever artifacts it would, spread out acorss a transient rather than time.
With the first example; I wonder what rotating it, adding the effect, then unrotating it, but then running it through a denoiser would yeild audiably? Maybe even one that samples the noise, and just feeding it the delta signal!
this is WILD! Hoooleeeey
huge technique
ooh insane
seems like rotating sounds really could have a porpuse and not just only for visual spectrum xd, nice effects here
MINDBLOWING?????????????
pls share your FL theme, its so sleek
Yooo that frequency shift vibrato sounds just like an effect in the intro to Radiohead's Codex
Someday you will turn the whole sound design industry..
Insanely cool stuff
Really love the frequency shifted transformed signal. Absolutely sick. You mentioned it's possible to do the same with convolution, amd though I could probably figure it out: how would you do it lol
The delay example is only very superficially sinilar to bitcrushing, I'd say. When you add delay to the frequency axis you're duplicating all of the frequencies that happen at the same _time_ at higher frequencies. Bitcrushing creates a hard limit to the output frequency in the original signal at Nyquist and everything above that is aliasing. In the delay example, all of the original frequencies remain, plus a bunch of other shit on top of them as they're duplicated upwards. It's like adding a bunch of inharmonic false overtones to everything from the original signal, and although bitcrushing also adds inharmonic frequencies to the signal, instead of being correlated to the original they are relative to the downsampling rate of the effect
yeah exactly
the sort of interval series they both create sound similar lol
No idea what is going on but I know I’m learning something lol❤
i love this lololol so cool
Next: vocoder
omg soooo coool
you can reverse sounds instantly on edison with a shortcut, i dont remember if it was alt or shift + left key, that will save you time from dropping audio into the playlist
oh true actually yea i forget about it
that eq bit was so hard to wrap my head around lol
"The tones here are what used to be the transients"
you should be making sfx for movies n games n stuff. truly impressive
i should but i have no idea where to look for jobs like this, it would be my dream job
A bit late to the party, but if you put the watermark on the side (diff.) channel, it'll be easier to separate from a mix.
oh yeah true
damn you're blowing up a little bit
anyway, that's, th- yea- cool
you are insane
Is there a way to make a vst that does all of this automatically, I buy it!
very cool stuff bro keep exploring this
ahaa i might, but i have no more ideas for this topic atm, so i beseech thee, to explore this in all your own worlds wherever they are~
because of your todo list: obs has a feature that automatically converts mkv to mp4 (remuxing). so you don't have to use ffmpeg. but using ffmpeg is way cooler!
I love the signature thing!!!
oh right yeah i forgot about that lololol
@@idiotinium btw. I just remembered you. I just read about solfeggio frequencies like 528, 369Hz etc. Do you happen to know how one would create a sound with frequency from scratch in FL? And measure it? Or do you have tips where I should look? I would like to experiment with that.
? what are those
12:56 "uhhhhhhhhhh yea this is cool" - lol yup
new sound design technique just dropped
i gotta make a m4l or pure data patch for efficiency
So i wonder if the inaccuracies in the final output after all the rotations have more to do with floating point limitations more so than methodology error? Maybe you could try rendering it out at a higher bit depth and see if that helps with some of the output errors?
Oh yeah and i wonder of using a higher sample rate would help too since you could get moee time resolution that way (i imagine the results would get worse the longer the clip youre trying to rotate is due to limited frequency resolution and the fact that frequency becomes time when rotating
nah it wouldn't be floating point errors, those sound really noisy, and this is a smooth cutoff of the low frequencies
time and frequency go hand in hand, its just the more information you have the more you get out of it sorta, you can rotate an entire song and it be completely retrievable
w vid
Next project for you. Make the sound to be negative. Queit will turn loud and loud queit.
pray tell, what happens to silence on a logarithmic scale? infinite loudness?
@@idiotinium I don't know any of that stuff. All i have is the idea in my head. Like when they make copies of vinyl there's the one disc where sound is mirrored so that the groove starts from the top and goes down. Instead of hearing sound, transients appear you would hear them disappearing from the white noise or how ever the quiet things will sound when taken up. This is not the case with the disk in analog but in digital audio i think this could be possible somehow :D
There are some better spectral manipulations out there, but interesting video dude
Hey could u upload ur patcher presets? I never really use patcher.
epic
my audacity can support up to 22050hz as opposed to 24khz on chirp, why?
also, the audio kinda isnt even sometimes
i flip it, its been over-rotated
i leave it like that or not, i un-rotate, either the time is uneven or the frequency is uneven, maybe both
your sample rate is set to 44100 instead of 48000 in audacity
@@idiotinium k i found where to change khz and it isnt over-rotated but i got a new problem,
theres a weird artifact above 20khz at the spectrometer, is that normal and does it matter much?
@@SomeRandomGerman probably not normal, i can't see what you're doing
@@idiotinium czcams.com/video/jlJmchEelbI/video.html
@@idiotinium idk if my last reply went through due to youtube sending my comments to the shadow realm for no reason so this is why im sending the link again
czcams.com/video/jlJmchEelbI/video.html
13:35 went full aussie for a moment there😂