Lessons Learned from a Decade of Audio Programming
Vložit
- čas přidán 6. 08. 2016
- In this 2014 GDC talk, Telltale Games' Guy Somberg offers a breakdown of his experience in 10 years of audio programming, including tricks for working with sound designers and lessons about building audio engines.
GDC talks cover a range of developmental topics including game design, programming, audio, visual arts, business management, production, online games, and much more. We post a fresh GDC video every weekday. Subscribe to the channel to stay on top of regular updates, and check out GDC Vault for thousands of more in-depth talks from our archives.
Follow us on Twitter
/ official_gdc
Checkout our Facebook page for GDC exclusives
/ gamedevelopersconference
www.gdconf.com/
1:23 *#1:* Playing audio is easy
3:45 *#2:* The audio mix is hard
8:12 *#3:* Learn to work with sound designers
[9:36 Conversion] volume = 10^(dB/20)
11:35 *#4:* Middleware makes bootstrapping quick
16:31 *#5:* Sound is always the first to get the shaft
18:43 *#6:* Listen!
19:23 Post-mortem
21:56 Audio programming is fun! + Q&A
I love how he looked around in the audience to make sure everyone finished taking a photo hahaha.
Very clever man, seems like a true master of the craft.
Oh
I like the pacing of this presentation. Very efficient delivery.
Thats funny about his offhand comment abount stream limits on the xbox 360. I totally remember that instead of the gameplay lagging it would be the audio that would go first, especially remember this when it would have to load new areas, or chunks near player. Happened a lot in Fable 2.
Careful with the 6dB = double.
That is true for a voltage or current , but things change for power signals. 3dB is double the power. The ear perceives power, not voltage.
haha yeah he seemed like he knew he was slipping up slightly there
I was sold on watching this since he showed he worked on Dora the Explorer.
Is that the one with a Virt soundtrack with songs in XM format? :3
@@boptillyouflop now I want to hear this
@@nintendude794 Dora the Explorer: Dora's World Adventure (not the other GBA Dora Game)... seen one XM file floating around, not sure how to get the 5 others...
10:22 - *NO* -. This answer solves almost every problems.
As a guy who was interested and practicing in music producing for several years now but started programming in the web field ~2-3 years ago, only now I decided that it's time for me to merge my two favourite things which is audio and programming and thus focus on audio programming more.
Where are you in the transition now?
floating in the vibe
How is it going, I'm thinking of doing that in college.
I just started doing this, been the happiest I’ve ever been.
@@earvingallardo1391 been thinking of doing this too. going to be a long road for me (trying to self teach) but this seems like something i really want to do haha
What a god tier talk. Well done. Lesson #6 is meta af. And I feel like a game such as league of legends has utterly mastered sound mixing within radius
It's funny thinking back to the Sound card wars in the mid to late 90's, all the ads in PC Gamer etc. for Aureal, Creative etc. sound cards/technologies, Vortex etc. some interesting things with 3D Spatial sound and now we've pretty much stagnated.
incase somebody missed the code or coming back to get it that he posted on the screen at 9:36 :
float dBToVolume (float dB)
{
return powf(10.0f, 0.05f * dB);
}
float VolumeTodB (float volume)
{
return 20.0f * log10f(volume);
}
Oldsiren I would add a check for volume==0 in the volume to db function, because this will be -Inf
@@charlesparker6167 But it should be -Inf in that case, shouldn't it? I don't understand.
@@sheboltaev Well, you're right, it should be. The problem is that the computer can't really handle infinities. (Well, technically, floating points have the NaN (Not a Number) for anything undefined, but I don't think it woud be really helpful in this case).
The solution Charles Parker gives seems good to me (I'm far from being an expert tho). You could also take care of that beforehand (You never need to stream a sound at -inf dB, so maybe you could cut it if it's under a threshold or something, Idk...)
@@ohhnyx9229 Maybe I misunderstood what Charles said, but I was under impression that volumeTodB would already return -Inf for 0 even without any check. Haven't tested though...
@@sheboltaev What I think he meant is that it mathematically would, obviously. The thing is, you *don't* want your function to be like "Yea This goes to negative infinity. I'll return a Not-a-Number and everyone will be happy!", Because it could break things in a lot of situations (if it doesn't throw an error at you in the first place).
The simplest example is this algorithm:
You have a float A corresponding to a volume.
You set A to VolumeTodB(A)
You set A to dBToVolume(A)
Now, try to find what happens when A starts as 0 :
VolumeTodB(A) returns NaN, since it goes to -inf. (or it may throw an error directly.I don't know exactly how C and family handle that, so let's assume it doesn't for the sake of the example)
So now, A equals to NaN
DbToVolume(A) then returns NaN too, because the log to something undefined is obviously undefined.
You end up with A = NaN
In other words, you start with a volume at 0, then convert it, deconvert it, and get an undefined volume. Not ideal, right? x)
That was for the long explanation. Since you specified that you haven't tested the code, I'm guessing you program at least a little, so you may have understood my second explanation without the example. Was still worth putting it there, just in case someone with no programming knowledge reads this. If you still have questions (or you figure out that I'm totally wrong, could always happen), don't hesitate !
Thanks man for this vid! :) Very helpful
Shout out to the one guy in the audience that learned audio programming lmao
great talk!
Actually the +6db = Double is kinda tricky subject, depends on what youre talkin about, sound pressure level is indeed +6db, but sound Power level is +3db and loudness level is +10db... Sound designer here Sorry ahhahahahha
>Sound designer here sorry haha
It's time for you to stop.
@@baronvonbeandip nah let him laugh like supervillain, it's enjoyable
1:25 "Playing audio is easy"
* me who was introduced into audio programming back in 1994 *
Rrrrrrrrigh
i love how the talk about sound has the worst audio problem
that's your crappy phone speakers ;P
Its the unspoken law of youtube
the classic "hairdresser with bad haircut" type problem, guess the audio fellas are too busy giving the talks rather than setting them up hahaha
usually the magnet is stationary and the coil is attached to the cone
So like I have experience in sound design and mixing but I want to get a job in this field. What course should I take to merge code and music/sound design?
What a legend.
I haven't worked on a game in my life but was happy with a lot of things he pointed out. I found them a bit obvious every now and then, but I know most people are very poor at listening.
I find this both as a musician and as a language learner.
Doing sound well is hard and a lot of people will not consciously notice. It's also the cause of the loudness wars in pop music.
Playing sounds badly is nearly free. Playing sounds well is incredibly resource intensive to the point that the nobody even tries anymore.
There is only one sound source and only two ears, so how does that generate complexity? The sound can reflect, refract around and transmit through objects and it's all frequency dependent. You've got doppler shifts, you've got the propagation delay before the sound reaches the listener which is different in different media, you have head related transfer functions that simulate the way the shape of the head and ear filters the sound. It gets very complex very quickly as soon as you leave the comfort zone of modern games with bog-standard stereo, or perhaps an inferior quality HRTF and some simple echo filters.
Even just a very simple and incomplete implementation of partial wave tracing and with HRTFs for directional implementation, which was done 20 years ago with the aureal vortex 2 chipset, is better than anything we have today.
This is a very sad fact, and I hope VR will revive the quest for accurate physical simulation of audio.
Honestly, if you want to hear good sound, Nintendo games on the WiiU are the shit.
Well... they're not *the* shit, but they are shit, as is everyone else since the early 2000's when creative patent trolled Aureal to death and developers stopped trying to do correct sound rendering.
Do you even know what I'm talking out?
Oh boy, I just searched Aureal Vortex 2 on YT and you're serious. That stuff sounds pretty awesome compared to stuff we got today (not even considering how old the Vortex 2 is). Obviously these kind of virtual 3d simulations have their flaws too (everything sounds "too close") or things like the 3d projection on your screen not matching the location where you actually hear it. Not sure how feasible it would be to get around that, but if those peolple in CG can do awesome sh!t, the audio people should be able to do that too.
@@theIpatix I suspect that how well 3d audio matches depends on geometry of head+ears and the HRTF used in audio engine.
I was pulling my hair out until 13.10 (initialization of 3D sound causes POP sound). THANK YOU!
So now a question: If I want to get into engine development, and am tired of practicing graphics code, what should I learn for audio programming?
Start with understanding basic audio engineering concepts, like sample rate, bit-depth, channel formats, audio codecs, etc, and also get an understanding of audio mixing concepts like signal buss, effects sends/chains, and various common effects like reverb, stereo delay, dynamic compression, and maybe some type of 'distortion' if your feeling gutsy.
As for programming concepts, there will be lots of buffers and queues, so understanding various ways to implement these will help.
You will probably need a very reliable method of thread synchronization. Because audio will almost always require streaming from disk, and because you don't want to risk file IO blocking on the audio thread, you will want some sort of lock-free (or at least non-blocking) queue to separate file IO form the audio thread. If you let the audio thread stall, then you will probably get horrible sounding stutters, which are generally considered more annoying than frame-spikes.
You will also need a way of interfacing with the sound hardware... this could be done through your operating systems API (SDL, ALSA, etc), or from various other libraries like OpenAL, FMOD, etc. (Though, the higher level library you rely on, the less control you have, and the more of it's issues you are stuck with.)
Once that works, then you could move on to some of the more advanced math subjects, like Nyquist Theorem and Fast-Fourier-Fransforms.
These concepts will help in implementing effects like pitch shift, doppler, EQ, etc...
Good luck!
For me, what did it was writing some VST plugins (and Buzz plugins back where Buzz was a thing).
@@stephenborntrager6542 It's obligatory having a strong calculus background to learn about Fourier transforms and trying to implement those myself?
I'm actually in first year of computer engineering but that stuff is being taught in like 4th year or 5th year. Is it realistic for me to try going for those subjects?
09:57 is what you are looking for
Lesson learned: Use FMOD
Wwise is better ;)
@@DukeJon1969 Why is Wwise better in your opinion? I am trying to prioritize learning one of them, so if you'd explain, I'd be happy
@@Caglarcomposes
If you're a sound designer/composer and learning on your own, I'd recommend Wwise - their tutorials are better. Once you learn one, you'll be able to pick up the other without much trouble.
It's an oversimplification, but Wwise has kept up with more advanced features for the sound designer - stuff like an integrated audio occlusion/spatialization system that scans your 3D meshes, better 1st party plugins, more flexible parameterization and 3D options - you can tweak and randomize things like detailed 3D positioning without messing with your game engine, for example. Basically you can do more with less code.
The pricing options are different as well. FMOD is more generous in this regard if your budget is less than $500k US.
awesome
very thorough overview of audio programming, might be true not only for a game audio engine...but what seriously happened with that hand at 18:08
feels good to know both the languages 8:45
Hey Guy
why does it sound like your voice is being parallel processed by an aliased spectral resynthesis engine
1000% TRUE
Teach us how to make a online mastering robot
This dude cracking jokes all the time and nobody laughs smh...
In another video I saw a similar comment and the speaker replied that people were laughing, the mic is just set up to only pick up the speaker’s voice.
Audience isn’t mic’ed up
i.e. use FMOD
13:00
Programmer time and priorities => Sound < AI < Gameplay < Graphics :(
aloluk in terms of immediate noticeability, this is also the order.
Sound < AI < Gameplay < Graphics < Lootbox Roulette Screen
is this C
Same goes for theme parks and experiences.
Big budgets for the rendering machines and projectors. Scraps for the audio system.
the coil is fixed to the cone, suspending inside the magnet which is fixed to the speaker housing.
it's the coil that moves the cone, not the magnet.
I'm just bad at audio stuff to where I'm fairly certain this just isn't a talk for me. Like the other day a DJ said that spotify had bad audio quality, and I still don't hear it. A shaky camera and I'm done though because I'm more visually artistic.
HAHAHA this is a job? HAHAHA I'm lost.
Paputsza it’s a hard task and most people don’t even realise it’s important until it’s too late and we save their asse(t)s near the finish line of their project.
Audio coder is a job and I've done it. And yeah, I can do the thing he talks about where you can figure out the type of sound bug by ear.
trust me i am a sounddesigner and 90% of sound designers, that imply they hear the minutiae of sound are full of shit. There is plenty of blind test that have shown, that anything above 192kbit/s audio quality is almost inaudible to an uncompromised audio - now translated: spotify usually plays at 192 or 320 kbit/s, while uncompressed audio plays at 1,4 million kbit/s. So if an sound designer tells you spotify sounds like "shit", well... they are full of shit. They just insecure about their hearing capabilites and must flex on others, but i guarantee you if you do blind testing on spotify premium vs cd at same volume and lufs, the difference will be inaudible. I challenge every audio engineer to this test, they will fail (cuz it has been done before and even the best couldnt diff. 320kbit vs 1,4mil kbit)
You dont hear anything. Then great because they're spending too much money to really care whats really matter like buy flac 24 bit 192 kHz for little improvement is just ridiculus
Damn this guy lost his job
why doghead
@@SuperTrisset Telltale shut down like a month ago
He left Telltale in 2015.
Haven't we all?
Thought I'd learn about audio programming, how hard can it be right? Now I'm sitting here looking at differently colored letters, symbols and numbers and I don't know what the hell is going on. Guess this isn't the day I learn code.
Learning the grammar of a programming language is easy, doing something useful with it takes research and experience as well as a logical mind. It's the difference between being able to sketch some doodles with a pencil and being able to sketch a landscape scene. This is a guy who has spent over a decade learning/building experience in audio programming, you're not going to get to his level (or any other professional) without effort.
So 12 years of working on 300 lines of code.
Lol, no. That was just the minimum he ever shipped. He talked about a bunch of other things that could be done
I'm sure he's knowledgeable, but what's the takeaway from this talk?
Just saying... don't ever say the words "NO" to a creative... especially an audio engineer. They know what they want when they ask for it. Nos will always create tension between the two of you. Word choices are very important. Us audio people are very stubborn, "NOs" get under our skin almost immediately. Maybe instead, ask them "Would there be something else we could create for you to use to make this task better/easier than *insert current solution here*?"
What are you, four years old? You’re a grown fucking adult, you can be told no especially if you’re asking for something.
@@chonchjohnch You missed the point. By a long shot. Like, I'm confident you didn't read between the lines in anything he said.
Tell them no and give them trash?
Leaving.
what
what
underwhelming