How File Compression Works
Vložit
- čas přidán 23. 01. 2021
- In this video I cover how file compression works
₿💰💵💲Help Support the Channel by Donating Crypto💲💵💰₿
Monero
45F2bNHVcRzXVBsvZ5giyvKGAgm6LFhMsjUUVPTEtdgJJ5SNyxzSNUmFSBR5qCCWLpjiUjYMkmZoX9b3cChNjvxR7kvh436
Bitcoin
3MMKHXPQrGHEsmdHaAGD59FWhKFGeUsAxV
Ethereum
0xeA4DA3F9BAb091Eb86921CA6E41712438f4E5079
Litecoin
MBfrxLJMuw26hbVi2MjCVDFkkExz8rYvUF
Dash
Xh9PXPEy5RoLJgFDGYCDjrbXdjshMaYerz
Zcash
t1aWtU5SBpxuUWBSwDKy4gTkT2T1ZwtFvrr
Chainlink
0x0f7f21D267d2C9dbae17fd8c20012eFEA3678F14
Bitcoin Cash
qz2st00dtu9e79zrq5wshsgaxsjw299n7c69th8ryp
Etherum Classic
0xeA641e59913960f578ad39A6B4d02051A5556BfC
USD Coin
0x0B045f743A693b225630862a3464B52fefE79FdB
Subscribe to my CZcams channel goo.gl/9U10Wz
and be sure to click that notification bell so you know when new videos are released. - Věda a technologie
![How are Images Compressed? [46MB ↘↘ 4.07MB] JPEG In Depth](http://i.ytimg.com/vi/Kv1Hiv3ox8I/mqdefault.jpg)
just put your hard drive in a hydraulic press
yes
yes
I alredy did that with some floppy disks
Meanwhile on another channel :
"This futa hentai is really dangerous and could attack at any moment, so we have to deal with it"
@@egg5474
w h a t
"squish that cat"
HI I CANT GET THE VIDEO TO LOAD IT SAYS I AM NOT CONNECTED TO THE INTERNET ...WHY ARE COMPUTERS SO COMPLICATED...WTF IS THE INTERNET ANYWAY, I JUST WANT TO LEARN HOW TO COMPRESS MY FILES I HAVE THOUSANDS OF PAPERS EVERYWHERE AND NEED TO COMPRESS
@@geist453 squish 'em papers
Aha!!!
@@geist453 you sound like you are not experienced with computers
Xd
Huffman encoding: the cat has ten million hairs, four legs, two ears, two eyes. Here's a tree with all the parts necessary and this string is the instruction to reassemble it.
@Anôn Imo pissed off much?
@Anôn Imo nobody asked
HI I CANT GET THE VIDEO TO LOAD IT SAYS I AM NOT CONNECTED TO THE INTERNET ...WHY ARE COMPUTERS SO COMPLICATED...WTF IS THE INTERNET ANYWAY, I JUST WANT TO LEARN HOW TO COMPRESS MY FILES I HAVE THOUSANDS OF PAPERS EVERYWHERE AND NEED TO COMPRESS
@@geist453 you can thank Steve jobs and the UX for normalizing computing for normies 😌 otherwise IT professionals would be out of a job.
@@gickygackers sure.....
Compression is really quite incredible. I had a high resolution heic image and it was only around 70MB. I exported it as a tiff and it was almost 500MB
heic is new kind of compression algorithm which is also being used for videos but it is always lossy unless specified as lossless. Another kind of such lossy compression is the djvu format which is capable of downsizing entire scanned thick books into multipage image file with several hundred pages. Converting that djvu to pdf will show that effect of 70mb djvu file to 500mb pdf.
Sadly heic and djvu are not widely adopted and used yet
heic sucks
avif better
Are you lot even speaking words?
dude that picture is bigger than half life blue shift
"In some rural areas you still can't stream 4k video" In entire countries man, entire countries
I find it really funny that you can do it in a Brasilian fabella but can't in an Australian city
@@cahallo5964 Cause all the kangaroos stomping on the internet cables, and the damage causes the cables to be less efficient, and in return, slower.
My country be like
@@TtEL bruh the kangaroos hate technology
In my country the highest quality is 1k on mobile, and HD on Pc, it's very normal
loving the thumbnail buddy. keep it up!
hahaha me too
all his thumbnails are cool, not like ur typical teacher trying to use a 2011 meme unironically.
@@yekh why is this so damn true
@Zonno5 Nice haircut
Yeah but...... I wanted a meme.....
1:27 “whereas the lossless one clearly there is some information lost” just wanted to point it out
Just making sure
Did I lose this part of the video?
Apparently
@@alexismandelias *compressed
I genuinely appreciate how you explain concepts with clarity and entertainment. You’re one of the better sources for understanding tech concepts.
Because most people who talk about this kind of stuff actually have no clue what they're talking about FYI.
Jpeg's use the discrete cosine Fourier transform for all you math nerds
Thanx ᕙ( ͡° ͜ʖ ͡°)ᕗ
I appreciate that
talk about the wavelets too...
Thanks
i prefer french cosine oui
This is the first time I see the XP background in wide aspect ratio. Didn't even know there are other hills in the background.
Nobody:
Torrents: **Puts a .zip in a .zip in a .zip... in a .zip**
At that point you are just adding meta data to the file and making the file larger
iso to zip to rar to tar
Those kinds usually hide malware
@@someonestolemyname the metadata isnt too big of an isue. but you cant get any more compression anyways from doing that
Why are you trying to download 42.zip?
JPEG XL is an upcoming compression method that will solve most of our problems with image files. It has a lossy mode that is way smaller than JPEG. It has a lossless mode that is smaller than PNG. And has support for all the kinds of metadata that you could want, including high color depth.
But the same is true of JPEG2000, and WebP, and HEIF, and FLIF. It's not enough to be better: Any replacement has to be so much better it would overcome the convenience factor using a universally-supported format like JPEG.
Situation: there are 5 competing image formats
That's ridiculous, let's develop one that covers all use cases!
Result: there are 6 competing image formats
@@fatweeb1545 Linux distros in summary.
@@dIancaster no because u dont need one distro only
@@snowwsquire you don't? My mint isn't enough?
another thing with compression:
images;
when the area has a slight deviation in color but still enough to look weird if it is all the same color; instead of storing all the 24 bits to get 255:255:255 color, you can just take a pixel and relate all other similar colors to it. so if there is barely any difference, you can express that in far fewer bits.
and about text
you can overlap selection.
so the phrase
Harry Potter and the philosopher's stone
contains several frequent words, that you don't need to repeat
letter combinations*
Another interesting form of compression is huffman coding. The jist of how that works is that it orders the different characters by order of probability of occurrence of a file on a binary tree, then it uses its placement on the tree as a stand in for its value. The effect is reduced number of bits for common characters at the cost of extra number of bits for less-used ones. A very cool technique and is very complimentary with other compression algorithms, and is also used commonly over channels.
There are 2 alternatives: arithmetic coding (e.g. CABAC) which compresses way better, but is way more complex, and there's Asymmetric Numeral Systems (ANS) which compresses just a bit worse than CABAC while being only slightly more complex than Huffman, making it a perfect deal and being used by lots of tech giants, and also in next-generation JPEG XL image format.
its almost impossible to tell the difference between the side-by-side images on my crappy rural connection
@@weakspirit_ bro starlink will change everything, my ISP wont know what hit them
1mb/s gang
@@afallingtree9114 elomegaluln musk
@@xfied you bet
There was a time I download torrent over ppp connection, 12~64 kbps
And this was 2008.
Really taught me patience.
Lossless
“As you can see some of the information has been lost”
Wait w h a t ?
The visible pixels.
i don’t understand anything about your content but i like your personality and you’re funny so im going to sub
Do you know why you don't understand it?
For me its like half on half. Sometimes I get it but once he rambles on I just lose it but I love the video
@@yniekac8851 a rambling teacher is the best teacher. And that's real btw
same. Some I get Some I don't get
Story of my life haha 😂😅
"Whoops, didn't mean to delete everything there."
Based and catpilled thumbnail
forsenCD ✌️
THE TWO TIME BACK TO BACK 💿
Imagine actually having to concatenate stuff.
that profile pic didnt age well
do I *look* like I know what a jay-peg is? I just want a picture of a god-dang hot dog
Jpeg*
Whips out mini wd40
@Tech Help Portal Extras yeah thanks for that. He definitely wasn’t misspelling for comedic effect or anything
I'm looking for a tap & die and some wd40. Do you know what an hammer is?
@Nem Gam Boi stf redditor
Yo I just want u to know I appreciate u so much man. I started studying computer engineering at a uni and the lectures just made me loose all most all interest I had in CE. But ur the guy who bought back my passion. Thanks G.
KGB: Throws cat into the garbage compressor.
Unpacking the poor thing would make a big mess though...
I think that would be a lossy compression. End result: minus one cat.
@@robdom91 end result: 27 cat parts
ok that went really dark
*Tom scream*
Bro that hit me way too hard, it took me four days to compress a 5gb folder
Zooming in the Bliss image shows evidence of 8x8 JPEG compression artefacts, one problem with lossy compression is that not only does it discard some things, but it can add things that weren't there before, such as those JPEG artefacts or some quite nasty effects if MP3 is pushed too far.
Recompression is particularly bad, if you compress to JPEG, and then recompress the result again to a smaller file, the quality will be worse than if you started again with an uncompressed or at least best quality original, as the second compression will be counting the artefacts of the first as part of the image.
raw to jpeg actually gets rid of a lot of info, including the dynamic range and editing capabilities, because it has all of the info from sensor, so you can edit the white balance perfectly
Hence, lossy compression
Edit: Yes, I know.
@@sanjacobs6261 That's not really what lossy compression means though. You lose the same info when converting to PNG which is a lossless format. The difference is that with a PNG you only lose this kind of metadata information whereas with jpeg you lose actual image information. Although how much and whether it's even noticeable very much depends on the settings since you can configure how strong jpeg should compress an image. But with high compression, you will very much notice artifacts on jpeg compressed images and for images with plain backgrounds/colors and text, it's much worse and much more noticeable even on high-quality settings.
But removing metadata has nothing to do with compression. There is no compression whatsoever going on there. You are just deleting information. It's like saying "I'm compressing an image" when you delete the lower half of it. The amazing thing about lossy compression is that it removes information but is still able to reconstruct an image that ideally looks exactly the same to the human eye from the remaining data.
@@sanjacobs6261 You know what ?
@@kiyu3229 1vader pretty much summed it up
@@sanjacobs6261 wow you replied very fast
i only compress isos onto a flash drive to distribute to my school mates
1:27 I think you mean the lossy compression type. You said lossless when explaining both examples.
Compression aka that conversation between the hardware and software for finding the data in ram and hoping we don't end up fusing all our memory cats together.
Anyhow I am a sucker for these types of niche videos and their surprisingly relaxing to watch.
Kenny keeping the thumbnail metagame high, as always ! Great vid sir
Thanks, you explained compression to me and I understood you.. That's impressive 👍
You need to put a magnet on your hard drive to draw all the data into one place and nicely compact it to make more space. Make sure to move it around a bit in case some data is stuck
This actually sounds convincing lmao
The thumbnails on this channel are amazing
>whether you're loading the data off a CD
_laughs in uncompressed Red Book audio_
*Laughs in FLAC*
Great videos I think you touched most subjects!!!
"like a boomer"
*looks at my CD collection* "its okay i still lvoe you"
My vinyl collection..
I already know all this but for some reason I still want to watch this and your other similar videos
1:27 Lossy*
o
i
i
v
going to computer information college and kept reading compression and didnt understand it my ADHD was frustrated from not knowing XD thank you for the informative video and your time
1:17 dam lossless and lossless have so many differences thank you for this information!
Thank you for this video! i was really confused whenever my friends said "compress the file", i tried to compress it by throwing my Hard drive out the window but my performance was a little worse after that so i decided to look on the internet for a new SSD and I saw this video! i will remember this for about a month, bcs i still need to test if throwing a SSD out the window saves storage capacity, Thx!
You did it the wrong way, you need to smash it to flatten more of the surface before you throw it.
Thank me later 👍
@@averagehololiveenjoyer8496 thanks! that really helped, although, idk if it will work yet bcs i've got to find the missing pieces to reattach
@Technophilius Pulcherax i tried that but i just ended up with a low-profile ssd
See what you need to do is launch it into space at a black hole, perfect compression to just a few bits of info
@@Music-nn9mi i'll met it around the other side of the wormhole, thx!👍
Absolutely loving these videos
Please note the difference between a picture and a pitcher...
So, compression is basically just refactoring code but for content. That's actually really interesting and I'm kinda surprised I never thought of it like that. In a way, a .zip file is more like a blueprint for how to build the file than just packing the file really tightly.
Compression is definitely crazy stuff. A couple examples I've seen: a steam game with a download of around 7gb, and an uncompressed size of around 12gb.
Another example: I exported one of my songs in the WAV format and it was over 50mb in size, while the MP3 version was less than 5mb.
Also, about "some rural places not being able to stream 4k," before I got starlink, which gives me a download speed of 100+Mbps, my old internet had a download speed of under 12mbps on rare occasion.
Just causal awareness of packed steam games vs raw.
Converting mp3 to wav is a sin.
1:26 - guess you wanted to say "lossy" 🙂. Just pointing that out, great video overall. I learn a lot from your content that's why you are in my recommended channels 👍
Genuinely interesting, thanks for the video.
The thumbnail is just great
Very informative, I learned some stuff!
Talk about single GPU passtrough on Gentoo, there's a reddit post on r/Gentoo with the title: "gentoo single gpu vfio passthrough scripts"
Yes please do
Compression mostly takes advantage of similar colors being side by side. It can encode the variation from the initial shade in fewer bits. Repetition only appears in artificial images like screenshots that contain exactly the same shapes.
I got clickbaited by those kitty files...😂
Same
its incredible how this works! program repacks are especially interesting to me, its just amazing how we can take a game thats ~100+ gb and repack it, losslessly, into a 40gb package!
I've seen an example of this where people will put a lot of a single letter in a file, zip it, the file looks tiny but when unzipped it crashes your computer or something because it's many terabytes in size and the computer doesn't have enough storage space
The irony of showing a before and after compression picture and then run it through compression before presenting it to his viewers.
Wow it's amazing how we can compress information and send it through electricity and microwave. imagine how quantum computing will compress information?
I can't. A quantum computer is not just a much faster desktop computer
@@celticwinter is it a brain of an ia?
top tier thumbnail game as always...DNA would be a good comparison for cat compression
Nice vid! You should do a part 2 and include audio compression
Audio compression knowledge is a very professional nature of knowledge and video compression is at least 4-16x more technical.
Knowing this information is only useful to music producers/distributers, and commercial industrial movie makers; for reasons like managing how much truth you can fit on a Blu-ray disk so your customers get more bang for their buck, instead of paying full price for a product soo thick that the dvd player struggles to play it without being over worked.
Knowledge of picture compression is much more relevant to poeple because of how many people actually do graphic design nowadays, and have to actually manage quality control on their finished product.
For video compressions, it can be a nightmare considering how far commercial practices have gone to allow only exact settings to be applied to work on a universal scale. For example, for your dvd players, you have to encode in mkv aac, otherwise it absolutely will not be recognized by any common dvd movie player. That being said, you can't just burn mp4 files to a dvd disk and assume that because the files read on your computer, that your Blu-ray player will look at it the same way, because odds are that your br player won't even know what it's looking at. I think this problem low-key comes from 2 things. 1. Combatting bootleggers (glhf). 2. Making it harder for the smaller entrapaneurs to succeed. [Same as it ever was.)
Companies like Nero, Adobe, and Sony have their software suites that are made to do the heavy lifting of ensuring you are compressing in exact commercial standards that are decided and agreed upon by lawyers, manufactures, and filming companies together. BUT, you may notice how expensive it is buy these programs/tools.
This is a definite example of how it pays to be smart, if you are a DIY hero.
on CZcams, i often have to put the video in 1440p to view it in real 1080p... also something funny i noticed is that in the past the limiting factor to compression was processing power! since they could compress it further but decoding it in real time was harder on some machines. tho i do not miss the day where i waited a week for one 90 MB blurry anime episode to finish downloading...
I have been using tiling wm for so long that now I feel uncomfortable when I see that strange thing above each window.
I don't watch cat videos. some how the algorithm decide to give me cat videos recommendation after watching how file compression works! Thanks u 2
Love the Videos 👍🏻❤️
thank you for such a wonderful thumbnail
Awe, I thought you were going to compare the algorithms (zip vs rar) and how they work. :D
Still a great video
One correction I'd make is that the difference between a raw photo and a jpg is actually significant. Very significant. Much more than just Metadata. Raw photo files have much more data than can be seen by just viewing a side by side preview on a 8bit monitor. If you've ever edited raw files from a professional camera and also jpgs side by side you'd emmidiately understand how much data is within those raw files. You can take a photo that is visually pitch black and lift exposure to find detail. You can pull highlights down from white. Color banding is pretty much nonexistent if you're editing in 16bit as well. Now once it's edited it will look more or less the exact same to anyone not in the know. Jpgs are great for presenting or even printing nowadays. But not for editing and not for storing all a photos potential
I 100% clicked the video due to to the thumbnail ngl, I still liked the information given by the content of it
I don't know why I watched this video but thank you for making it
You mentioned at the beginning of the video that reading a lossy compressed image like a jpeg does not involve decompression. I would argue that decoding any jpeg image involves a sort of decompression. In order to display a jpeg image on a screen, it needs to be decoded which involves computing every single pixel value in the image. Lossy compression just means that the decoded image result will not exactly match the original, but decoding a jpeg still seems like a sort of decompression.
love your vids
Legit a better explanation than my 60,000 dollar cs course...
Way better explanation than what my teacher did lmao
They should have included this technical part in the sitcom it would have been great and exclusive content for nerds shows amount of hard work that goes into building these things
They could have taken wav to mp3 as a use case
Showing that compression in not literal sense but how excess bandwidth is reduced to give the same experience which isn’t detected by the ear
Dude how do you output so many videos on a weekly basis, im geniunly curious, they are super high quality and informative, by all means, please keep it up, but hoooow??
Theres many videos to make
Thanks for sharing, now I understand this better ,
But what about vectors they are scalable with no lose in details I think that due to the facts it's geometry based . but if we convert images to vectors will we lose details or we have to do some shapes .
2- about youtube decompression the buffer how they made it I think they have used their encoding with mp4 format but how they managed huge amount of bandwidth in more depth ?
3- in technical terminology some film makers use encoding format to retrieve all recording data like they can change camera shutter speed and stuff and edit it after shooting what this called?
4-I know about video resolution is not determining fact about the quality ( video bitrate is more important) is there any AI encoding algorithm can retain the data by Artificial intelligence fixing and if so is there artifacts ? how it's compared to ground Truth data ?
I have kgb compressor which was able to minimize a whole anime (12 episode) to only 10 megabytes.
is it possible that the tool it's self have different results ?
can we speed up the process by AL ?
and finally some websites has it's own technology to which can speed up the download I want video about it .
thanks
This guy literally uses the same Vim theme I do and explains JPGs via the Bliss wallpaper. How did I stumble onto this near perfect channel?
Man thanks for the video. Probably I have crappy speakers but the sssss and zzzz are way worse than other videos on line.
Ah yes, my favourite tech video thumbnail in youtube
does the green eye of the cat on your profile picture stand for gnu/linux mint and the purple one for gentoo or is it coincidence? i love your content btw man keep it up stay woke
I love how he uses bliss as an example at 6:23
The worst types of compression actually both happened in the 90s: hard drive compression only achieved ratios similar to ZIP and would grind any CPU at the time to a halt. The other was "SoftRam" that was said to compress data in memory to free up space when in actuality it faked the memory space without doing anything. Compression used for ill gotten gains in both cases really...
I love how he says "In fact some rural places still can't stream 4k"
I'm from Germany, have live in a small city, have one of the best internet connections in a 20km radius and I sometimes can't even stream 1080p...
I love Germany...
Basically, compression is like making a pancake. Flattening all the good stuff up and remains delicious!
Saw a screenshot of the thumbnail on tumblr and I just had to come see this video because of it
I'm expecting for that but the explanation is good
that thumbnail is incredible
1:27 He never talks about the Lossy one. It's always the Lossless one!
nice explanation!
It's 2021, games can easily take hundreds of gigabytes and we still use JPEGs to save 10 kilobytes
not all games are bloat tho
@@igorswies5913 i know, but most of them are still heavy as fuck
@@iskamag games you download once and you know it will take a long time, meanwhile when you load a website the amount of users that click off is correlated to how fast it loads. A megabyte is a lot when you think of the web
@@igorswies5913 it's not that much heavier if you compress it good enough _looks at apple_
@@iskamag Not gonna lie tho, there are many games compressed from 30gb to 4gb because they are just pure chonk
*AMAZING THUMBNAIL!!!*
dood luv ur tumneil
I demand a link to that meme, compressed or not.
P.S. nicely explained, very comprehensive.
Finally black luke uploaded
Thank you so much
This guy knows how to make a good thumbnail. I clicked because cat
Great video
As we all know, every time a file is compressed a cat is mercilessly squished into a box.
The reason why they're are so many formats for each nature of data presentation, is because different types handle different natures of data differently. Gif handles black and white pictures best, png holds data very accurately and is this very heavy normally, and jpg is like the middle ground that is like the minivan for pictures. It's kind ugly but it can carry a respectable amount of accuracy in relation to the original source, without requiring a ton of gas.
Same goes for videos like mkv, mp4, avi, wmv. When it comes to music or audio, I like my absolute favorite songs to be in hefty flac, instead of mp3 so I can squeeze all of the quality, that the producer meant me to hear, out of it. If it's a normal song 192-320kbps mp3 is plenty smooth /clean enough. If it's an audiobook or old-school game music, 96-128kbps is functionally plenty. Funny enough, audiobooks are commonly marketed out into 96kbps, BUT at 48000khz range instead of the common 44100 in order to retain the depth of audio that can easily be lost when compressing into the fuzzier/blurrier lower density bit rates.
This is usefull to know if you like to share files online and want to shrink enormous files to cut down on a file size, transfer times, or if you feel chained by a mysterious 8mb file size limit.
Rule of thumb for simple folk would be that full quality (lossless) is only completely utilized if you have the equipment for it. If your device is low quality, you will have a harder time noticing the fine difference in lossless vs lossy compressions, and you will be wasting time and energy on something out of reach.
Last note, all compressions have xy or z amount of options to perfect quality vs file size. I recognize wav as having minimal options for audio compression and being rudely hefty in data, compared to flac with more flexibility.
I like mkv, png, jpg, gif, mp3 and flac.
I hate avi, bmp, and wav the most.
"pishure" i like the way you pronounce the word "picture"
Thank you
came for the thumbnail, left educated
Tom Scott's video on text compression gives a much more detailed look at how it works. Titled: "How Computers Compress Text: Huffman Coding and Huffman Trees"
RAW > JPG definitely loses a LOT of data, but the difference is how much that data matters. If you're just viewing it, doesn't really matter at all. But when it comes to people that need to edit photos, having RAW or higher bit-depth data drastically increases how well you can apply certain effects to an image or do color grading.
For example, I took some pictures today with a blue sky. However, my camera by default saves to JPG and is inherently 8 bits per pixel. The result is that when I tried to do some color grading on it, I started to reveal some banding on the blue sky. I can use certain algorithms to deband it, but it would be nicer if I just saved it in RAW and was able to manipulate the data that way, despite the files being significantly larger and slower to save into the camera.