YouTube's Existence is Insane: How Video Compression, Encode, & Decode Work (Basics)
Vložit
- čas přidán 4. 05. 2024
- Sponsor: Get 10% off Squarespace purchases (geni.us/BqEpf)
This video goes through the very top-level basics of how videos work. Most of the discussion is hardware-agnostic, talking about video encode, decode, and compression. At GN, none of us are experts in these topics (and they exit our usual coverage spectrum), so graphics engineer Tom Petersen will be joining to help provide the foundational knowledge as a part of our educational series of engineering discussions. Towards the end, he talks about the hardware-level choices that affect media processing. This is the last of our series of 3 videos with Tom Petersen. Check the others below, and check back for videos with other engineers later!
Watch our educational video on graphics/video drivers and game optimization: • Fixing Intel's Arc Dri...
Watch the video on Simulation Time Error & Presentmon: • FPS Benchmarks Are Fla...
The best way to support our work is through our store: store.gamersnexus.net/
Like our content? Please consider becoming our Patron to support us: / gamersnexus
TIMESTAMPS
00:00 - Video is Complicated
02:07 - Basics of a Block Diagram & Media Hardware
04:51 - Basics of Colors, Pixels, & Media Format Formula
08:08 - Encoding, Compression, & The Human Eye
12:32 - Spatial & Temporal Redundancy
15:00 - Frequency Quantization
18:07 - Symbol Coding & Bit Reduction
20:18 - Intel's Specific Hardware for Media
** Please like, comment, and subscribe for more! **
Links to Amazon and Newegg are typically monetized on our channel (affiliate links) and may return a commission of sales to us from the retailer. This is unrelated to the product manufacturer. Any advertisements or sponsorships are disclosed within the video ("this video is brought to you by") and above the fold in the description. We do not ever produce paid content or "sponsored content" (meaning that the content is our idea and is not funded externally aside from whatever ad placement is in the beginning) and we do not ever charge manufacturers for coverage.
Follow us in these locations for more gaming and hardware updates:
t: / gamersnexus
f: / gamersnexus
w: www.gamersnexus.net/
Steve Burke: Host
Vitalii Makhnovets: Video Editing
ft. Tom Petersen, Intel Engineer (Guest) - Hry
![[실시간] 전철에서 찍힌 기생생물 감염 장면 | 기생수: 더 그레이](http://i.ytimg.com/vi/4nkFvwxGAfA/mqdefault.jpg)
Hey everyone! This is our third and final installment of educational deep dives with Tom! You can watch our previous two below. We'll have some other industry engineer videos from NVIDIA and case manufacturers coming up. I'm working on booking something technical with AMD hopefully in the near future as well! Aside from the big 3 silicon companies, what other engineering professions within the industry would you like to see on this channel? Even if I'm not familiar with the subject matter, I can study enough of it to at least interview someone for the basics like this!
Watch our educational video on graphics/video drivers and game optimization: czcams.com/video/Qp3BGu3vixk/video.html
Watch the video on Simulation Time Error & Presentmon: czcams.com/video/C_RO8bJop8o/video.html
Very cool!
Cerebras wafer scale engine for ai and scientific computing
Since Nvidia is already planned and AMD too, I hope you bring in Microsoft and Direct Storage would be nice. Haven't got much updates past 1.2. A deep dive would be very cool. Want to see how they will maximize the performance and utilize modern NVME SSDs. Because currently we havent seen much advancement.
Please, head up to Accursed Farms and watch the last video from Ross, this is adjacent to GN, but it's still a topic you guys should cover.
Would be good to hear from id Software to know how their games are so scalable and amazing.
Switching to 144p for increased immersion.
It'll really help relate the topic back!
Make sure you enable 1440 Hz refresh rate as well
make sure you sit on a gaming chair
upto interpretation type shit
It's all "retina" if your eyesight is bad enough 🤔
TAP is the perfect example why vendors should let their engineers talk to the buyers. The way he makes you understand horribly complex topics is awesome. And it makes you appreciate their products more than any marketing BS.
It really does seem like the companies are allowing engineers in front of our cameras more and more! It's been great for deep dives on technicals we're not familiar with!
@@GamersNexusThe only people capable of making complicated topics seem simple, or easy to understand, are professionals. I love these kinds of videos, keep it up👍
I think part of it is its easier to teach an engineer on how to publicly speak than it is to teach a public speaker engineering.
@@DJFIRESTONE92This be the truth.
This is the right marketing for the technically inclined part of the crowd. Good job Intel for figuring this out
Please don't give up on those technical interviews. They are what we need.
We may need better affordable sw and hw, but in the meantime we can find interesting actual information being presented to us.
Agreed!
Small correction: YUV 4:4:4 / 4:2:2 / 4:2:0 doesn't describe bits, it describes how many chroma samples are stored. The first '4' says that we are talking about rows of 4 luma samples, the second number describes how many chroma samples are stored in the first of two lines and the second number how many chroma samples are stored in the second line.
That means a 4×2 block of luma samples contains
• 8 pairs of chroma samples in YUV 4:4:4
• 4 pairs of chroma samples in YUV 4:2:2
• 2 pairs of chroma samples in YUV 4:2:0
Another way to think about it is that in YUV 4:4:4, each luma sample has its own pair of chroma samples, in YUV 4:2:2, each 2×1 block of luma samples shares a pair of chroma samples (the chroma planes have full vertical but half horizontal resolution) and in YUV 4:2:0, each 2×2 block of luma samples shares a pair of chroma samples (the chroma planes are half horizontal and half vertical resolution).
4:2:0 is truly a nonsensical shorthand made by an insane person. Because 4:2:2 and 4:2:0 are the only ones that realistically exist, you could just as easily describe the stored chroma resolution of a 2x2 square of chroma pixels instead of the insane self-referential sample numbers, for example:
YUV 2x2 = YUV 4:4:4
YUV 1x2 = YUV 4:2:2
YUV 1x1 = YUV 4:2:0
Even if you argue that you must keep the 4x2 rectangle, describing the stored resolution still works and is way less insane than the "number of changes of chrominance samples between first and second row" like bruh
This comment needs more attention. It’s good knowledge
@@krakow10It is a truly bizarre naming convention, I've heard it explained multiple times and never fail to promptly forget what it means. It's easier to just remember one is full res, then half res, and quarter res .
@@tiarkrezar Indeed. It makes no sense because its roots are in analog TV and the numbers are not bits but factors of carrier frequencies intermingled with interlace logic.
Thanks for this! I asked Intel (since, like I said in the video, I know nothing about this field). Intel said this:
"Good catch! Although there is a correlation between how many bits and how many chroma samples you store, the numbers represent the actual amount of chroma samples."
I just realised they wrote L2 cache as L2$ and I'm rolling.
3:10
Edit: I wrote L2 cash at first and was noted it was a mistake, so I changed it to Cache.
Thanks to everyone.
that’s actually pretty common short hand. it is funny tho
Cache, but yeah.
Cache rules everything around me.
Its not cash rather cache and thats how its commonly referred to.
@@dojelnotmyrealname4018 Fixed it, thanks!
13:00 that's why it's so hard to compress confetti, snow, or other super small moving parts in a video. There's even a term called "compression nightmare" for these scenarios. Videos appear to be at a low bitrate, internet usage spikes, as well as cpu utilization.
slow mo guys have show this well with their glitter stuff, gavin is actually very knowledgable about this stuff.
I remember part of the reddit blackout protest was to upload videos of static that are essentially impossible to compress.
Yup... a lot of new information gets introduced and removed by the next frame, which makes it impossible for most algorithms to deal with this situation...
But the good news is H265/HEVC/VP9/AV1 encoder/decoders can deal with these situations A LOT better compared to the old ones such as Xvid/DivX/h264/AVC/etc...
@@6Saturn9 by blurring them out lmao, gotta love sao in x265
and also why common things like a gradual fade in a video is actually quite difficult for compression to deal with, there's very little per-pixel frame-to-frame stability
As others have pointed out, these videos with Tom have been fantastic. I think the information is presented in a way that is not only valuable for gamers, but also for many ComSci students as well.
Thanks to everyone involved, and hopefully we can see Tom back in the channel on another occasion!😄
Thank you! And looking forward to the next guests also!
Just sitting here watching this, eating some dinner, and half way through it just becomes immediately apparent to me that real, tangible people figured all of this stuff out and continue persevering and innovating on greater ideas and technologies. It just blows me away how intelligent the people were who designed and produced this stuff. I guess it's just very impressive. I mean, not even 100 years ago did we even have the first television.
early compression was simple stuff most the temporal stuff but yes its as he said magic, honestly people forgot how much tech is involved in just getting a single video delivered in real time, from the undersea cables the high speed fibre and relay to all the tech in chrome/youtube etc.
Something that amazes me even more is how all of this took many many different people contributing to just one great thing of millions created. There should be a class in schools dedicated to kids working together.
Or maybe, the school itself should be organized in a way to encourage and teach kids to work together
Me press button, moving photo come out.
Me no understand how works but me happy
Bro releases a top-class uni masterclass and uploads it to CZcams for free. As a Computer/Telecomunications uni student, this is really interesting and amazing.
WRT frequency domain on images.
Picture it like this: the corner (0,0) is 0 oscillations -- constant value. the pixel at (0,1) has zero horizontal frequency, but 1 oscillaiton on the vertical, meaning it starts at 1, goes to 0, then back to 1, sine wave style (well, cosine actually but you get the idea, it's smooth and connects cyclically end to end). the pixel at (0,2) is the same but has two oscillations vertically, and so on. This step is usually performed on small blocks, 8x8 or 16x16. So on a block of 8x8, the frequency pixel at (8,8) is a checkerboard, and (0,8) is a series of 8 horizontal lines black white black white etc. the bottom right pixel (N,N) on any resolution ALWAYS coincides with the pattern that gives you a checkerboard.
Edit: Welp, I tried to explain Frequency Quantization and Symbol coding when PAT did it better... When they talk doing it in the residual, that's another layer of optimization in newer video codes, as it works the same as with JPEG images.
Just to expand on it, as this was a moment of realisation for me when studying this JPEG compression:
For each block of 8x8 pixels of the original image, we will assign a concrete "checkerboard/frequency" coefficient value. Meaning, if the first 8x8 are mostly black, they will recieve the coefficient (0,0). As this is a standard, we know that if we recieve (0,0) this can be reconstructed directly to a set of 8x8 pixels. If you zoom in all the way, you can appreciate the DCT patterns.
We can apply a low-pass filter (reducing sharpness) and then can compress further by assigning compression algorithms to how many bits are needed for each coefficient. Meaning that coefficients that are more common (low frequencies), will use fewer bits that higher frequency ones.
I probably have some of this mixed up as this was some time ago, but it was cool understanding how videos are compressed.
Video compression + ffmpeg is a modern marvel that powers so much without users knowing.
These discussions and presentations have been fantastic. Thank you (everyone involved) for producing this.
We'll make sure Tom knows this sentiment! He's very understanding that we want to minimize marketing and maximize engineering. Looking forward to our next discussions with other engineers in the industry as well!
This was AWESOME!
Computerphile did an introduction on DCT a few years ago going into more details of the math and intuition of the algorithm, in their 3 part series of covering JPEG compression. For those interested, it is a series worth a watch, as video compression seems to be very similar to JPEG compression on differences between frames.
HEIF, the format that is likely going to replace JPEG everywhere, is in fact just a single frame of H265 video. It just makes sense to reuse the same format for many reasons.
@@Finder245 same case with AVIF, but HEIF will definitely not supersede jpegs
3 Blue 1 Brown did an EXCELLENT video on the Fourier Transform which explains how these sinusoid transforms work, and I think explained the tricks they use for the FFT.
@@simulping4371 why not? Apple already uses it in place of JPEG for pictures taken using iPhones.
Veritasium also did a pretty good video on the Fast Fourier Transform that underlies DCT.
I can't wait for AV1 to truly take off, so that 8K, 120 fps, HDR, 12 bit colors, rec2100, 444 chroma subsampling and all that jazz can become common.
Tom is such an awesome guy he deserves his success 100% really appreciate him doing things like this.
I could watch videos of you and Tom all day and not get bored, and learn many new things. These technical series are fantastic.
Yes, I'm definitely not bored, but my brain may overheat from trying to parse all the data
7:44 *”CZcams is suckin’ down the bits”*
Thanks, Tom!
This guy is great. Thanks for collaborating…
It is so great to see a technology channel that actually talks about tech (instead of making funny, reality show-esque videos with graphics cards).
The videos you guys made a while back about latency/input lag and GPU drivers were amazing as well.
Crazy informative video series
Thank you! Love hosting these because we learn a lot from them also. Now we just need to figure out what topics and companies to work on next!
Sony with their version of dlss
Capcom, micro transactions.
@@GamersNexus see if you can work with some game devs and break down each step of development (storyboarding, writing, coding, modelling, rigging, texturing, lighting, etc.)
should be pretty relevant and would give viewers a better understanding of what goes into the games they play
@@chillnspace777 pisser
It's almost impossible to get bored with GN.
Also starting with the bandwidth CZcams would need is crazy.
Imagine how much fewer bullcrap would come out of youtube if there were that fewer videos... maybe democracy would be thriving instead of being on the verge of collapse.
I always see LTT fanboys saying Gamers Nexus is "boring"... they're insane lol
I look forward to more in the TTAS (Tom Talks About Stuff) series of videos.
I'm throwing "TTT (Tom's Tech-Talk or Tom Talks Tech)" in there. And if snippets should get published on TikTok it would be a "Tom's Tech-Talk TikTok". 😄
Talking Tom
@@scrittle I apologize but I cannot help the 14 year old creeping around inside me thinking aloud: "I love TT's".... ***snicker***
I do love the Tom Talks and want more of them. ***sigh***
Sometimes I disappoint myself...
What makes GN interviews with specialists and engineers so engaging is that Steve can keep up. Brilliant communicator that can translate the info into ELI5 for us idiots.
This was incredible. I've always wanted to know more about compression and although I knew the basics, the step-by-step process overview was super helpful to get a greater understanding of how cool compression is. It's one of those mostly invisible technologies that most people don't know exist but are absolutely essential to keeping everything functioning.
Frequency domain analysis is an extremely fun branch of math that has applications in so much stuff
Circuits, sound, images, video
You can view any information as a combination of several waves and instead of analyzing the signal you study its frequency components
One of the steps there is the same as doing a low pass filter in audio, except high frequency in images correspond to sudden changes in values. Clipping it blurs the image (or whatever the equivalent is in YUV)
Doing a high pass filter meanwhile is an useful way to get any edges which is useful for image recognition algorithm/AI but makes the image look like a normal map (that's tangent space, not frequency space, but hey)
Steve saying "I'm coming to this with very little knowledge" is rare and really shows how humble and ready to learn a new thing he is. Love this channel and how the mindset goes. I hate people who act automatically as if they know exactly what someone is talking about when they actually barely have a superficial idea of what the subject could be.
Agreed! I have more respect for humility than bluffing bravado. Steve you know more that you realized or vocalized. Very impressed with your modesty.
It’s always a great day when we get to see Tom and Steve in the same video
Just last year I had a class in university about video encoding. 40 seconds into this video it was already worth it, the 2D discrete cosine transform was part of the exam. The class covered everything up to the Huffman encoding although of course in much more detail. The Huffman encoding was part of a different class.
Exciting to see my university classes have some real life uses.
This type of content, really makes me appreciate the existence of this channel!
Shout out to the video editor, incredible transitions from video footage to the slide deck!
I payed hundreds of dollars to have a weirdo that claims to be a university teacher, show me a slide that is 1% as informative as this whole video which is for free.. Can't stress enough how good it is of you guys to share this. thank you so much Gamers Nexus. Keep em coming
Very nice side-step to highlight other tech. I always had an idea of how it works but having it explained in a structured and clear manner is great!
Refreshing to see some more in-depth presentations about how it all works instead of the usual high-level marketing slides, really enjoyed this series.
Every video with you and Tom is an absolute delight. Thank you all for the hard work to make these topics approachable. The passion from everyone involved really comes through and means a lot!
Damn you sure kept the best for last. Thanks Steve. Hoping for more like this in the future.
Fun fact: one example where you can see the effects of removing high frequency information from the residual is when you have a scene with a lot of confetti. It gets destroyed by the compression.
What i find amazing in video compressing is that the it can compress so much without loosing much experience. An audio is much more sensitive to me when compressed.
As some one who does computer graphics programming for a living, Tom has defined things that I can't even TRY to explain to someone else. Steve, MORE videos with Tom! I would love for him to show how non-realtime applications like Maya and Blender interact with the hardware. Heck Tom start a series on Intel's youtube channel or something where you explain more CG stuff. There is a serious lack of good resources for learning CG.
I really enjoy these conversations with Tom. Most of it is way over my head but it does give me some insight into what is happening behind the scenes.
Thanks for these enlightening videos.
These Tech-Talks with Tom are incredible. Such a wealth of information delivered in a way that even a layman, such as myself can understand. Please keep these coming. 👍🏻
This is incredible stuff, thank you for putting this together these conversations!
You two did a great job going through that. He is a great guest, able to explain things pretty simply.
I love that GN does this content. It doesnt have to apeal to everyone. Just genuinely nerdy content that few fully understand (including me) is great.
I work in a place with a lot of nerds. Some love to talk about stuff they probably shouldnt, and i love listening to them. Even if i dont really get it. They are so pationate about what they do and its great.
Thank you James! (And Tom who is the Intel's papa introducing us their best engineers.)
Great and informative video. Thanks GN and Tom for taking the time to talk about it...
G'day Steve & Tom,
HUGE THANK YOU to Tom for his time working with the GN Team to make these Educational videos.
I can't even tell how much I enjoy these educational pieces with Tom Peterson and the GN team!
Thx a lot!
Just an outstanding series of videos, real best-of-youtube stuff. Talking to customers in a non-marketing way by showing the breadth, depth and enthusiasm for the subject and how they think about their products. It's advertising that's actually worth something to the consumer. Outstanding.
Loving your teaching videos. The info we've all been looking for.1. Brilliant work mate. Love from Aus.
Super informative - brilliant work Tom and Steve!
I loved this 'trilogy' with Tom Peterson, he's such a good presenter and explainer, even for such complex topics and ideas. I hope we see more of him in the future, these content heavy videos are really interesting to say the least
I for one LOVE this kind of content. I've always been a "but how does it work?" kind of guy, so these types of conversations are pure gold.
This video series that you did with Tom is so good! I enjoyed all of them.
This is greatly informative. I'm halfway through the video, and it's helped demystify concepts that were previously very fuzzy or esoteric. Thanks.
Thanks, great series of technical explanation, love it!
This is awesome. Very well explained basics of something really complicated and ever present for all of us. If Tom has more topics like this one I would love to see him from time to time with those deep dives. Really cool 💛
Tom's seems like a great dude, actual super educational stuff. Love this type of content
Fascinating video, the amount of information packed was impressive for this short format but not enough to be overwhelming plus Tom's presentation...chef's Kiss. As someone who did encoding of movies for trackers for about 2 years it made me understand more what I was actually doing and how it all works "under the hood", the amount of methods and "recipes" to optimize encoding/image quality is another BIG rabbit hole though. Thanks Steve, Thanks Tom. :)
Saw Tom talking the same thing about this exact topic on Intel Arc's channel a few weeks ago
This not just interesting but super helpful on learning how encoding/decoding works under the hood. Tom's explanation on colorspace is very easy to understand and perhaps miles better than any text articles do.
Thank you for making this!
Excellent content Steve, appreciate these types of videos!
Great presentation and really enjoyed the dive into compression process!
This was a wonder piece. They were able to take something I knew very very little of and was able to break it down into very relatable and easy-to-understand content.
This upload/discussion is priceless. I had wondered about YT's compression. Thank you for sharing this discussion. 🌸
Fantastic explanation, you guys did an awesome job of making this accessible to layman enthusiasts 👏
Thanks for this wonderful video!
love hearing about the technical side. Keep it up!
Video & audio encoding (lossy) is absolutely wild with modern formats. Hats off to the people that came up with it & those that somehow still squeeze more out of it.
We take it for granted now that we can just throw massive data amounts on the cloud and expect it to just be there. Thanks for exploring what’s behind the curtain
Excellent, in depth on a very interesting topic, love it! Thanks Steve, thanks Tom!
I love this, I had kind of a research project on this topic in uni and had a lot of fun discovering what encoding really is!
This one was hugely interesting and well presented. Thank you so much! Love em
I‘ve been through Linus, Marques and various others. GN is my new religion. The perfect balance of chill, quality journalism and entertainment. You deserve every Gbps of my traffic.
Fantastic video , well done , great explanation and great presentation 10/10 will watch again.
I ask so many program over the year to "compress" my video files and i never wondered how it worked. Now i now a little better
This series, with Tom Petersen, has been is fantastic. I would definitely be interested in more of this kind of thing.
Awesome content! Thanks, GN!
This was amazing, thank you for the video!
That was amazing. I feel like i just went back to uni and my favourite lecturer explained to us some of the most obtuse information in a way we could all understand. Bravo Tom, Steve and gamers nexus.
Fascinating video, Thanks Nexus!
This was really fun to learn about! Thanks for the great content :D
This is been a fantastic series, thanks Tom
This stuff is so interesting to me. In a past life, I did systems level and driver programming (back in the MS-DOS days). It's so interesting to see that the video compression stuff is hardware agnostic - it applies to all hardware - but then the video driver takes that information and makes it specific to (or translates it for) the hardware.
This is some nerdy stuff, right here! ❤
I love these types of videos! Keep the deep dives coming.
Please Please continue these, there is so Little technical information available online that is presented so well
these have got me watching your channel again. great stuff!
Thank you for existing GM & Tom ❤
Thanks GN
I've personally waited decades for this video. Thank you Steve, thank you Tom!
Great content, can't wait for the next episode and my own testing of PresentMon.
This is incredibly interesting, thanks tom
loving this series!! bravoo!
This and the other recent vids with TAP were seriously awesome, I hope to see more of him on the channel again!
Excellent content! Informative and understandable.
This stuff always amazes me! But then at the same time, it's not like it just magically came together all at once. Breaking it down step by step puts it into much much simpler and realistic perspective.
Thank you guys. Great presentation and follow-up questions by Steve.
Quite interesting information, especially for people with CS background who may need to code or interact with something like that one day.
I love these videos. I definitely don’t understand all of it, but boy is it fascinating.
This is just what my brain needed today. I'm absolutely serious about that. Thank you so much for providing content like this for us.
it was really spontaneous to know all these information about pictures from a highly tech person. thanks to both of you.
It is actually quite interesting to have someone, who is an engineer explain video compression, as someone that has recently learned about how it works on a technical level for my job.
That presentation he showed was insanely helpful and well designed