Spatial audio in Godot: Dynamic acoustics update
Vložit
- čas přidán 17. 01. 2024
- FAQ: www.landervanregenmortel.com/...
A quick showcase of the current WIP state of my spatial audio system in Godot. These past weeks I've been working on a system in which spatial acoustics are procedurally calculated based on the geometry of the level. My previous video showcased audio propagation, which is not included in this showcase.
The range of different acoustics you hear is completely dynamic: The differences between being outdoors and indoors, different sizes of spaces, being outdoors but having large walls around, the sound source being in a different acoustic space than the listener, the resulting acoustics are all completely procedural. This means it is not required to set up individual audio volumes anymore to define the acoustic characteristics of a specific space, which can be a huge development cost depending on the fidelity the developer is aiming to achieve. Besides this solution working well for a static environment, this solution is great for games/experiences that are built around procedural environments, games with dynamically changing level geometry, and for developers or teams who cannot afford the cost, time investment, or complexity that comes with setting up believable acoustics in their projects.
Even though it is completely procedural, I am currently working on a custom audio volume node that will give users control over the sound of a space, while retaining the advantages of the dynamic nature of this system. An initial version of these audio volume nodes is used at the end of the video, in the pit and the large underground chamber.
To put my spatialisation system to the test, I've created a simple movement/footsteps audio system.
My website: www.landervanregenmortel.com/
--------------------------------------------------------------------------------
Car model - skfb.ly/6sUFy
Wow. Seems like the sound as much important as the graphics. The stairway room, the clock ticking, so immersive.
A great point. I think good sound design can complement or is even more important than good graphics.
Ofc it's important. Don't you ever dare to overlook it
This video kinda shows us, on practice, how audio can enhance the experience, very helpful for game developers!!
Darkwood is a good example of how a simpler looking top down horror game has better atmosphere than most first person perspective games. Peak sound design there. The sound occlusion there is insanely good.
I've heard that saying way too much lately. It's not true. You absolutely wouldn't buy, or play a game that has no visuals, and was only sound - but you would totally play a really good game that you could see, but the volume was stuck at "0". Shoot, I play Halo with the sound off all the time, and it's still exciting. The opposite would not be true whatsoever.
Sound is *really nice to have*, and good sound can be transformative.
But it's not "as important as the graphics". Outside of a game made for blind people, it will never be the case - I don't care how many thousands of years later we're talking.
my god this has been my dream for YEARS. how in the WORLD did you manage to do this. absolutely mad.
My guess? Line of sight and filtering the sound from an average of the surrounding materials.
You will buy crappy games and play them and be happy. No seriously. Klaus said so at the World Economics Forum. It is like feeding the pigs. They just eat it.
@@techpriest4787 what
@@techpriest4787 ?
@@techpriest4787 ?
outsideness is a beautiful name for a variable
I would LOVE to see a video or a paper on how this is done!
it's like raytracing of light/pixel hitted to geometry of the mesh, but audio can be processed faster than the each light/pixel, that's y we can hear this in realtime
@@animestation7225 is that really how it works? that sounds awesome. i mean it definitely makes sense. it would have to trace all around the player. i would have guessed its more like it senses how far away an obstacle is from the player and calculates it from there. maybe with voxels that travel with the player
@@ThatsMySkill I think you are talking about HRTF (Head-related transfer function), which for example is implemented in the game PUBG PC (Steam or Epic Games)
@@animestation7225 just reading up on it seems like its not quite what i meant. as far as i understand it HRTF only calculates the direction sound is coming from but doesnt calculate sound going around corners or reverb in big rooms. im not into the matter at all i just thought it would make sense to draw a cloud of invisible voxels around the player that recognize a wall when a voxel collides with a wall around you and then calculates on how the sound waves should bounce/ distort. at least for reverb that sounds like a good approach if you dont wanna use that raytracing technique you mentioned. this is just me straight talkin out of my ass tho so no idea what the best approach is or if that voxel idea is even viable in the first place.
@@animestation7225 that's simply not true at all. Reason why raytracing in any form is slow is because you would have to check for ray collision against all meshes in the world. Doesn't matter what information is being is collected, sound or light, raytracing itself is slow.
This is so dope. A must buy once available to purchase on the asset store! Will this also work for 2D games?
Thanks Miguel! I haven't really considered 2D yet to be honest since game-dev wise it's not an area I'm very familiar with. That said, on the top of my head I can't think of any obvious blockers, it seems like it could potentially be ported to work with 2D. Currently my focus is just on 3D though, so besides looking into it, I can't make any promises on that end.
Isn't this a bit useless for 2D?
@@stickguy9109 realistically yes. You would be surprised though how dynamic acoustics in a 2D game drive immersion.
@@bitbraindev Yeah would probably help a bit but you don't need that much accuracy as in 3D games. Simple reverb zones would do the trick I believe
@@landervanregenmortelis it paid or free?
EAX. 90s gamers will remember. Great work, reminds me of Thief with EAX effects.
Forgotten technologies of the past. The time when if someone down the corner would say something you will hear it sound coming from the corner and not from through the wall
Man my relationship with godot is like watching a pie inside an oven you just gotta wait a lil more and it's gonna be so good
You could say that you're Waiting for Godot
this is so awesome, please tell me you plan to release this to the public in some form. stunning work!
You could probably achieve this affect by casting raycasts around the player, and for every raycast that is touching an object, increase reverb volume, and if the raycast is shorter, decrease reverb wet value. But idk for sure, the system shown in the video sounds cool.
This is very cool! Is there a specific research paper or technique that this is based off?
Also very interested to know
also ALSO very interested to know
That's really good! Great work!
Will there be any way to modify diffusion, tone, material and such parameters? Or to add some filters and such to modify manually based on the scape the player's in? I'm still a newbie in Godot but a music technologist by trade so I'll be very excited to learn more about the audio side of Godot.
This is awesome. I always forget how critical good audio is to making a space feel real.
this is just insanely good work, can't wait to see what you come out with next!!
that falling sound was amazing , great work!
This sounds absolutely stunning.
This sounds amazing! Really brings the world to life. Keep up the awesome work!
This is extremely cool, it's amazing how much immersion this adds
this is absolutely amazing, you did it so well. i would one million percent support this when its available. Will be watching closely ! thanks
Running back with the flashlight to show the reverb on the click was very much needed to showcase thank you so much my ears are grateful
I've been searching for stuff like this in Godot! Keep this work up! ⭐⭐⭐⭐⭐💯
So cool! Really goes to show you how important audio is for immersion.
Fantastic work! Watching this project closely
Outsideness!! Bravo, truly excellent
Such an awesome demonstration!
I can't say anything other than a big fat WOW!
Simply amazing!
😘👍
Fantastic, I am thrilled. This is definitely a must-have plugin.
This is great! Keep it up, we need you
Wow. Just... WOW! Thank you for making this!!!
As an audio engineer and game dev this has been on my to do list for my own project once I get around to audio. I'm using unreal, but can't wait to give this a go because it sounds just as good as I thought it would.
this is incredible work 👏
This looks/sounds fantastic!
It's insanly well done big respect !
Excellent. I wanted to build something like this for a shooter I was working on, but I have a lot more research to do about sound in general first... This is so satisfying!
Truly phenomenal!!!
Incredible. Just incredible.
I would never have thought that this would make such a difference!
this is wonderful work yay ♥
That's some top tier stuff. I'd love to buy this as an asset and try to use it in a scenario I have in mind. I'm also highly fascinated on the way you achieved it, it would be awesome to watch an explanatory video from you
Totally agreed, keeping my eyes peeled for this 👀
fantastic work!
WTF Dude. This is crazy. Congratulations. Wonderful work.
This is incredibly awesome, captivating, and immersive! I can't wait to see what you'll do with it!
The Vincent Price record is a hilarious but awesome touch. Great work, man!
I really need that grid texture.
It looks amazing
I'D PAY to implement it, sounds amazing. Great work, holy!
Wow this is incredible!
This is sooo amazing !
Absolutely incredible.
this sounds amazing!
This is the best kind of ASMR
Quite impressive!
Wow! Really dope
so cool! awesome stuff :)
that was freaking awesome!!!
Really cool system, I'm interested in this too
This is sick
Soooo cooooool!
Just awesome!
This is amazing.
Amazing!
That's impressive!!! keep it up
Broo is AMAZING!
jesus, does need some tweaking with relativity to the walls in between, but this sounds (and looks) incredible! great job man and keep pushing for further development!
ha yes the outsideness value, very important information right there lol
great work btw sounds amazing
This is awesome, Godot needs a really good system like this for audio as well as VOIP. It's what makes games like Tarkov so immersive.
Tarkov does not have immersive audio lmao
Lethal company is a good example of VoIP with dynamic acoustics
BRUUUUH If only had Tarkov audio like that.
I've gotten VoIP to kinda work in the past, though it's really rough around the edges.
Speechless
This is absolutely killer. How did you achieve this? Is it essentially a simulation of spatial audio or something like that? I know nothing about audio but this makes me want to study it!
Very interesting!
This is crazy good.
Wow, just wow!
sounds awesome
Very cool
Thanks man!
This is just... wow
Very nice!
I don't know weather it was intentional or not but this gives me ASMR
Amazing.
awesome!
very cool, I love this tech. I used it in Minecraft with a VR mod and it was insane.
I am speechless.
This is Soooooo good
this is sick
Wild.
Spectacular! I wish every game had audio design done as well as in this demo. Take notes triple A studios…
You're going to revolutionize FPS games with this.
Yuss!!! I remember playing Half-Life with an Aureal A3D soundcard and being able to enable it and have such awesome audio like this - then Creative bought them out and ditched the tech, never to bring it to market again. I've been waiting for someone to replicate what they did 25 years ago so we can have rad immersive audio again. Mad props!
Lovin it
holy shit this is so good
Impressive!
This rocks.
Dang. THIS IS COOL!!!
Can we please get a video on how you did some of this stuff? The audio is incredibly impressive. Your movement looks great. The lights are insane too.
Wow this sounds so good!!! Will you make a video explaining how this works? It would be so interesting!
Thank you! I would like to do that when it's more developed and more set in stone :)
Awesome
Amazing work!
Thanks, Beau!
this is so fucking sick
The muffling around corners seems just a tad exaggerated, but yeah, very exceptional work. Games have been missing this.
Godot Battlefield here we come
off the chain ! Do you do something also in Unity? it's very interesting
Wow that big room sounds so realistic
WHY DO I SEE YOU EVERYWHERE
You are incredible! I'm audio engineer for music and I've been dabbling in Godot. This was so so good to my ears, and that jump at the end was sold so well with the sound. I honestly can't believe it. Is there any way we can support your work so it 100% makes it to us to use in our own projects? I'd also love to help playtest it and give audio-focused based feedback if that helps at all.
My one comment on what could be improved (if possible) is room reflections bleeding into each other, 00:46 was when I noticed it, the sound got very muffled and I feel a bit more direct reflections would hit the ear from the wall to the right and behind them. My diagnosis could be totally wrong but that's what I noticed. Regardless, that is a tiny blip in what feels like a masterpiece of achievement.
Thanks for wanting to support, the best way to help me currently would be to share my work!
If you're interested in testing this down the line, please feel free to drop me an email through my website, it would be great if you could outline any projects you would try to implement this in. :)
this looks amazing, how do you calculate outsideness?
This is quite cool! Will this become available sometime?
Also a question: does this have the ability to take room material into account? i.e. we all know when you move into a new house, the living room is always quite echo-y. However, when we add more furniture such as a sofa, chairs, curtains and stuff, thats going to absorb the sound, reducing or even negating the echoes. Would it be possible to implement something like this? Could be either you, or a user of this system (if you intend to release this).
Whether fully automatic, or if your objects need a certain material/property, i dont care. Would be cool and add a lot to immersion
Thank you! No ETA yet, there's still quite a lot of work and testing left in order for this to be production ready. This is still in a pretty rough/primitive stage.
Currently I haven't implemented surface materials yet, but this is on my to-do list. I'm not entirely sure yet how exactly I'll implement those yet, but I'm trying to make things as easy to use/accesible as I can.
That said, I want to clarify that my system is not aiming to be a physically accurate acoustic solution, even though material interaction will probably be implemented and it should deliver the expected behavior (A same sized room with carpet will sound less reverberant than a same sized room with tiles), the way that is computed will not be physically accurate. Adding furniture into a space has a significant influence over the acoustics of a room in real life, but my system might not be accurate enough to properly represent or detect that change reliably. Graphics have historically relied on smoke and mirrors to achieve convincing results (e.g. screen-space reflections), I'm following a similar mindset with acoustics currently. I'm hoping to achieve something that is easy for indies/small teams to implement, that still offers convincing and immersive results.
There will still be audio volumes that you can place in case my solution doesn't achieve the desired acoustics in a certain space, those volumes will allow you to make adjustments for a particular space, and can encapsulate multiple spaces while still keeping relative differences in acoustics (e.g. for a furnished house you could add an adjustment volume that reduces the reverb length, but still keep the difference in acoustics between rooms). In this video I already use an initial implementation of those for the pit and underground room at the end, since I wasn't achieving the desired long-tailed reverb that I wanted there.
i remember something similar wich the creator of the sonic ethers shaders made a while back. it wasnt as crazy as this one is but still. he coded it so that if you are in a big cave it would add a lot of reverb and if you are in a tight spot that reverb was completely gone wich made it feel very claustrophobic and added a lot of atmosphere.i wish big studios invested more into actual dynamic audio like this instead of faking these effects. i bet this could be greatly used in horror games too. great job man!
AAA studios typically have horrible sound. Reverb, localization, occlusion, velocity... Shoot, they can't even get wind right.
@@Go_Coup exactly and thats why it doesnt sound outlandish to suggest valve would innovate on that field