Video není dostupné.
Omlouváme se.
Lossless vs. Lossy (Part 2: ABX Testing)
Vložit
- čas přidán 14. 08. 2024
- More information about ABX testing and lossless vs lossy audio!
● Links and further reading:
en.wikipedia.or...
en.wikipedia.or...
nwavguy.blogspo...
Thanks everyone for clicking the Like button and subscribing! :) See my channel for the things I recommend the most. / lachlanli. .
facebook: / lachlanlikesa. .
twitter: / lachlikesathing
The track used in this video, 'Aria' performed by Kimiko Ishizaka is from the Open Goldberg Variations and released under Creative Commons Zero License (Public Domain).
The download page and links can be found here: www.opengoldber...
The licensing terms are as follows:
creativecommons...
Notice of Fair Use:
All graphics and visual elements in these videos are used for educational and instructive purposes as provided for under United States Copyright Law. www.copyright.g...
The graphics and visual elements are used for educational and instructive purposes.
They are of sufficient resolution for identification but lower resolution than the original graphics. Copies made from them will be of inferior quality, unsuitable as artwork on pirate versions or other uses that would compete with the commercial purpose of the original artwork.
We believe this constitutes a 'fair use' of any such copyrighted material as provided for under US Copyright Law. www.copyright.g...
My ears are so fine tuned that I can tell if a burger is cooked rare, medium, or well done by tone-tapping the bun.
But the bun isn't the burger.
Woe, Is [S]unjay
It is part of the burger.
I think when one struggles to hear the difference, what's the point? Just get a decent file and some decent equipment and just enjoy the music.
you'd think that, but that proves your normal and reasonable....something that the died in the wool audiophile is not!!
I'm pretty sure I can't tell the difference between MP3 320 and FLAC
Nor can I
Only issue with this test is why did you choose to use a song without much going on in it.. not a lot of dynamic range or frequencies being used.. Everytime i see comparison videos like this which use vocal solos, Classical music, or soft music used for these tests, i always have to shake my head.. In my opinion the best music to use for these tests are electronic/industrial/ebm type music and metal music. Music which has multiple instruments happening simultaneously through a wide variety of frequency ranges, everything from sub bass/lows to mids to highs, and a lot going on which mucks up with compression and more to notice.
For metal music with it's wider variety of instrumentation (2 guitars, vocals, bass, drums (and synth in some)), frequencies, and range, you'll be able to tell a difference between a Lossless and Higher bitrate MP3 files even up to 320kbps, especially with certain metal artists with lots of cymbals ontop of drumming, guitars, bass, and loud growling/screaming vocals, as typically the cymbals take a sound quality hit between 192kbps and 320 kbps (they distort slightly or don't sound as crisp). Children of Bodom, Fleshwrought, Draconian, Ensiferium are good examples where this happens, but pretty sure a lot more.
Then electronic/industrial music where you even have a more drastic variety of simultaneous frequencies, lots of sub bass ontop on more bass, on top of electronic synth noises in every frequency range, drums, cymbals, harsh/electronic vocals, etc, same thing, the higher frequencies take a hit or the clarity of the music track muffles slightly between even 320kbps and lossless. good examples which you can hear a difference are The Prodigy, Psyclon Nine, DeadMau5, Nine Inch Nails, etc.
So encoding a hi-res file removes hiss, its an alternative Dolby noise reduction solution for the digital age.
Yes, that's my take-away. If I have an old hissy recording in FLAC, I can remove the hiss and leave the music by compressing. :)
did 14 attempts with Chord Mojo + Ether C and got 50/50... my life is a lie. Thanks for these videos really good reality check. :)
You make a very good point. I'm the one that didn't get it. The difference probably didn't have any thing to do with the relation of the recording to the format but rather the recording quality period. One recording was great in one format and awful in the other. Most likely it didn't have anything to do with format.
Many thanks for the video. Balanced, factual approach to an old discussion which I've had and heard since I graduated engineering. Its the digital vs analogue recording in a different form. There was even those who claimed gold plated audio cable connectors could be distinguished from the more plebian variety.
What I really liked about your two peices is that it acknowledges that the process from generation to hearing of sound is, in practical terms (i.e. the real world) complex and dynamic and always has noise pollutants seeping in from a miriad of sources . It is impossible to avoid. This is what most "sheet readers" forget. What is measured in an audio lab is only indicative of what an individual will be able to hear in a normal setting. Taken out of the lab , it arguably defeats many expensive refinements in equipment and coding due to the increase in the local background noise alone .
The piece on cognitive bias is usually missing from most discussions - nicely included . Would love you to expand on it in the future .
FYI My old lecturer (who was a ex Philips audio designer and lectured in systems analysis) had a no nonsense response to the discussion. Having always claimed that regular ear hygiene for over 30's (given the unfortunate but normal deterioration of the frequency reception of the ear due to age ) would be more beneficial than listening to flac etc. over a reasonable quality MP3 / AAC .
However one credible argument to date for lossless I have heard is for historical archiving however this does not negate your points.
A test where both the subject and person administering the test are blinded is called a double blinded test.
Nice video-tutorials mate :)
It's better and wise to do ABC testing between 128 kbps MP3 vs 320 kbps MP3 files, because no one can feel the difference between 320 kbps MP3 and FLAC :) I do not believe people saying that they can hear the differences. It's just a placebo effect .
I will suggest to do ABX testing between 8bit and 16bit audio also. Let's see who can differentiate :)
Conclusion:
I think that the best format for listening is 256/320 kbps MP3 file.
thank you very much for booth of the videos.
i used to think that i am able to determine flac or not, but it turns out every time i compared, i did the clever hans think.
i definitely hear a difference anytime but it is so marginal that it is very difficult to tell which tune is better, that is the main difficulty.
if you are on the go, i think a good mp3 quality is far enough, if not the only rational format considering the limitations of disc space on your device.
Smart fella! Subscribed.
It's hilarious how ABX testing is like kryptonite for "audiophiles". I don't even bother arguing with them any more. Let them waste their hard-earned money and enjoy their delusions.
For abx test, you definitely need to somehow do away with clickety-clack sounds of relays, switches and plugs being unplugged/plugged that are easily distinguishable...
This video was different but was very interesting I think it largely depends on the quality of the original recording, because if you have a poorly recorded file at 96khz, it could actually sound worse say an 320kbp file and vice versa. I agree that it highly unlikely someone would notice a difference between lossy and lossless file, in an noise environment, the abx test was good because it gives a way to finally settle the growing argument, between the to file formats.
Good video, can't wait to see the next one.
Actually if high bitrate compression is transparent, the lossy compression will not make the bad recording sound worse, it will just sound the same. Clearly in this test there was a difference, but if I tried with AAC or with 320kpbs v0 MP3 it may have been a different result entirely.
It will save you money from the Apple store if you can't tell the difference!
Also works for wine… drink cheap!
Not sure if this is worth mentioning, but wouldn't it be more interesting to also do another ABX test where you determine whether X sounds BETTER (or enjoyable) than Y, as opposed to determining whether X or Y is lossless? Cuz some music may actually 'benefit' from compression, ending sounding better to some people of the population. I think such a test would benefit music you don't listen to a lot or haven't heard before (you don't even know what the lossless version even sounds like, thus you are testing for which one really sounds BETTER to you).
This is so true. I listen to trance music and have quality copies of all of my favorite tunes. Yet, there is a radio show called "A State of Trance" that I listen to on Spotify as well. I have listened to some tunes that I know very well both in the clean album version on Spotify and in the radio show which obviously is run through a compressor when broadcast and recorded. I listen to both versions on extreme quality setting on Spotify which is known to be 320kbs. I noticed this a few months ago that when I was listening to the radio show, there was something that was just satisfying about the sound. Its obviously a thicker, while surely muddier sound, but there was just something about it that was pleasing and just sounded "good". There were obvious details and spaciousness lost but it was still very satisfying and when I checked it against the original album 320kbs version the loss of that extra thickness and impact that the radio version had made me go back. So now I have a playlist of a lot of my favorite tunes in the radio version, because sometimes the character of that compressed sound just "feels" better, even though it's lost some clarity.
I agree most of your conclusion, but to me, I can truly tell the difference between the 320 kbps mp3 and lossless even when I'm on the coach and using iPhone to listen to music ( probably because of the good isolation of my headphone), and when I use the speaker, this difference will be much more obvious. On the other hand, piano might not be a good choice for testing, try something more complexed, which might be easier to hear the difference.
I think you should consider doing the ABX test to actually verify whether or you do truly hear a difference :)
Actually I have done a test with my friends before, it's like, I had the same songs for two versions: 320 kbps mp3 and WAV (lossless), I played the songs for random choice and turned my laptop to their back, as I finished playing two songs, most of my friends could tell there's a difference between them.....
I know it's not that much difference compared to their size, but there's truly a difference.
Alan Wang The problem with doing a test that way, as I explain in the video, is that you already have a 50% chance of getting it right. Did you ever play the same file twice? Did they ever say they heard a difference when it was the same file? Even if you were to do things like playing the lossless file twice or the MP3 file twice and asking if they heard a difference, the problem with asking that way is that you can unconsciously influence the answer the same way Clever Hans' trainers did.
If you do not perform the test in a systematic, blind fashion after several trials, you are most likely just going to end up confirming your own beliefs. If you are confident that there is a difference it's worthwhile trying the test, even just for fun. It's also worth doing the test in AAC, which is harder than MP3 to pass an ABX test on.
***** What most people don't realise is that if you get your lossless and lossy files from 2 different sources, (online download for example), the only different between them might not actually be the compression. They may actually be a different mix or master. The only way to be sure you're using the same source (hence making the only difference the compression) is to convert that specific lossless file into lossy and ABX the two. I'd also like to know what listening equipment this Alan Wang bloke uses, because imo, it's VERY difficult to tell the difference between high quality lossy and lossless. If you look at spectrographs of lossless vs lossy, you can clearly see where data is lost, but whether or not I can hear it, I'm skeptical. Although, upon doing an ABX tests, I usually guess lossless vs lossy correct most of the time, only on some music though.
Flac vs. MP3 (these formats as an example, I have no experience with other formats)
Personally I get a sort of listening fatigue with MP3’s. A friend of mine has the same. It is not that the sound is bad but there is something ‘wrong’. A time ago a mutual friend who is a doctor showed us an article in a magazine for neurologists. Most of it was too complicated for us to understand but the conclusion was very clear: some people have something different in their brain wiring and are sensitive to a certain distortion in MP3’s. This is no joke and the source was a respected magazine for neurologists. Is this the main reason for the fact that some people don’t like MP3’s apart from snobbery and other reasons? I would like to emphasis that for my friend and myself MP3’s does not always sound bad but something irritates us while listening them.
I want to check if a have the same problem with other lossy file formats. To try out in the future.
+Rob Dequeecker Honestly, I think you should blind test yourself about that "mp3 fatigue". Maybe the fact that you know that you are listening to mp3 is a biased and thats why you get fatigue. Im not saying the fatigue isn't real, I'm saying that MAYBE it arises because of your knowledge about the file.
To be sure of your statements, I would ask you to listen to a long session ("long" being whatever amount of time is typically enough for you to get fatigue) of music that you like, and I would NOT tell you if it is mp3 or FLAC. Then, I would ask you if you got fatigue or not and reveal the file to you.
Four possible results follow:
Maybe you don't get fatigue and it was mp3.
Maybe you get fatigue and it was FLAC.
Maybe you get fatigue and it was mp3.
Maybe you don't get fatigue and it was FLAC>
To make sure, the test should be done at least 10 times. Perhaps it should be done at different times of the day, at different conditions (like being hungry or not)....And most interestingly, doing a couple of tests where I actually tell you the file-format BEFORE the session and then ask you if you got fatigue or not. After getting your answer I could reveal to you that I actually lied about the format....
Certainly you wouldn't be up to such tests...but scientifically speaking there are many variables that should be taken into account.
I usually listen to my music when I'm on the go and not listening to music in a quite room which would likely make me fall asleep. For my purposes it's not worth the extra drive space wasted for the little clarity you get which likely I won't noticed because of the environment I'm in or the road noise when I'm in the car.
Where lossless does count for me is when I'm watching a movie and I am in a quite room. I want to hear the sound effects and to make the experience as real as possible.
For some lossless is important. This are usually for the die hard music listener and I really doubt try are listening to dub step, rap or pop music like I do.
For most people, lossless quality doesn't apply to them plus you need high-end listening devices to hear the differences or to take advantage of the high quality file.
There is also improvement just from using a different software player. for example. Using Jriver in ASIO driver mode and an ASUS Essence STXii sound card produces a few layers of audio improvement in my experience.
Very nice video about comparing 2 áudio files! This video will be very handy to show for some friends of mine that overtalks about flac files. Especially cause reinforce my lines that you have to have a dogs hearing to noticie a diference between an flac, and a 320 kbps mp3!
Hey! Awesome videos!
Is the sampling frequency of your sound card adapting each time you play a different file in the ABX test?
Vorbis 500kbs like lossless
Thank you for a tutorial on ABX. It allowed me to do accurate comparisons of 44/16 vs. 192/24 audio. I did two tests with two sets of files. Got the same result both times.
"Total: 12/16
Probability that you were guessing: 3.8%"
Doing tests is indeed fatiguing.
That's interesting! What files did you use? I'm curious to try them myself.
I used the first two albums from soundkeeperrecordings.com/format.htm
Using ADL GT40a DAC with Grado RS1.
Jenya Burstein I'm impressed, especially since you did that with an open headphone. I tried an ABX using the first sample and got 5/10 and 4/10 across two runs. I checked the files out in Musicscope too and there's no funny business. May I ask how old you are and if you work in audio professionally, or as a musician?
I'm 45, don't play any instrument, never worked in the audio industry. I'll do another ABX run with the same music later. There wasn't much of the outside noise, though I could hear the laptop's fan.
I was very confident during my first ever ABX run. While I was statistically satisfied with the 12/16 score, I thought I'd be able to do 100% next time. My wife gave it a shot with a laconic summary that she does not need a computer to tell her that she can't tell a difference. I, then, had a few abysmal (quoting one of the guys on your video) tries with results displayed immediately. I realized that this requires concentration, and, after a short break, braved myself to do another 16 sample run. The immediate results were, of course, turned off for both of the 16 sample runs. Using key mnemonics helped a little. . That way you can keep eyes closed.
Jenya Burstein Hmm interesting. I'll give it another go, but you my friend may just really have golden ears! Where are you hearing the differences?
The problem I have with a ABX trials are that they miss the whole point. To me anyway it never was a question of one format being better than another. It' s more a question of whitch format does the best job of bringing out the redeming values of the original recording. Some of the original recordings were atrocious hence loosy files improve the sound. Now as a nostagic old fart some of the vintage music has a soul to it that would be sacriligious to tamper with. And that's where it gets difficult. For myself the imperfections of a live recording are a part of the essence of the music of the times and should never be edited.
The point of this video is not to say that lossy files introduce certain imperfections (of course they do), it's to encourage people to test whether or not those imperfections are at all perceptible in the first place. Whether the compression makes it sound better or worse is not particularly an interesting question to me (because everyone will have a different opinion), but the question of whether the compression makes the file sound different AT ALL is interesting. By all means there's no harm in keeping lossless files on a hard drive and listening at home given that storage space is cheap, but for a portable player it seems largely redundant.
You can't compare lossless and lossy files with iphones or any mid-fi
equipment like basic computer sound/headphones through the computer sound. If you don't have a truly great hi-res setup you can not ABX
test because the system your listening on and comparing with can't show
any of the improvements over mp3/aac. I've been lucky to hear files on
some really great hifi systems and in most cases the mp3 sticks out
like a sore thumb.
yeah, sure
DO take a bit of care when comparing 44.1 and 96 KHz music - Windows might be doing some resampling internally.
The case is, that the sound card can only run at one sampling rate at a time. Windows (also OSX, Linux) however must be able to play your 48k audio track on CZcams video along the 44.1k MP3 and the ringing of the Skype call, all at the same time regardless that those audio sources have different sampling rates.
The only way it can do this is to resample them all on the fly to the same sampling rate. So if you are comparing two 44.1 KHz files it doesn't really matter because both will be treated the same way. On the other hand, if you are comparing 44.1 with a 48 or 96 KHz, you might listen to one of them resampled and the other one not resampled at all (it is probably the 44.1k version that is going to be resampled, and 96k going straight). That might affect your results.
Apparently Foobar can be used with ASIO sound drivers which take exclusive control of the sound card and configures it to match the source sampling rate. Or at least pushes that decision to sound card which might resample it anyway.
I should probably do an ABX test on that anyway :)
he did it with the wasapi thingy
Hiding your results shouldn't really make a big difference. The only thing that I can think of that affects your choosing with results on is that you know that X is a random file, so if X was A lots of times, there's a higher chance of it being B the next time, however, this isn't totally true. Since you already made a choice when you see the result, this isn't really a clever hans situation.
I agree it shouldn't make a huge difference. Here are the reasons I can think of to do so though: the possibility that someone will abandon the trials or do less trials when they see that they are getting a poor result or a better one. It's also possible that the stress of doing poorly may reduce your ability to differentiate things as well. It's seems best just to eliminate that distraction.
Paco Verschut Confidence might be a thing. If you think you can barely hear it you might start to guess at one point or if you start to get confident you might drop your guard and guess wrong while otherwise you would have kept paying attention. I think things like these don't matter too much if you keep them in mind though.
So it's ok for me to use 350kbps ogg files in my sansa? :D
Somewhat true, that environmental noise overpowers quantization noise... but, it entirely depends on both the music and the kit. If in a quiet environment, listening to music with deep bass, with high quality headphones - or, on a very high powered sound system at a party - the difference is noticeable.
Lossy audio compression cuts lower frequencies. Although you made a great many points around confirmation bias and ABX testing, and explained compression algorithms - I'm disappointed that you didn't include in your discussion anything about MP3/AAC compression algorithms, and the specific frequencies they cut. Nor anything about their history; MP3 being created for the film industry to store audio clips of sound effects, none of which occupy the entire frequency spectrum.
Perhaps you should create #3 of this series, with a truly unbiased approach, explaining the benefits of both lossless 44.1/16 audio files and lossy compressed files - their benefits and drawbacks.
Sky Makai "Nor anything about their history; MP3 being created for the film industry to store audio clips of sound effects, none of which occupy the entire frequency spectrum."
What's the source of that? Isn't it historically documented that Suzanne Vega's "Tom's Diner" was used during the development of that format?
Yup, i can tell but it needs good audio system. I use 24bit 192khz version of Phil Collins song Hang in Long Enough and convert it to 320 MP3. In ABX test the "Hi-hat" hit in about 30 sec in that song sounds far wider (wider echo) and hit is sharper and cleaner on FLAC version. Headphones are Audioquest Nighthawks trough Audioquest Dragonfly.
I honestly find no difference between flac or 320/lame tracks, granted the music if from the same source ie. downsampled from the same file. There are questionable uploads up there specially if you're just grabbing pirates.
But psychological effects are quite powerful on some people. If it makes them feel better about what they're listening to, by all means, more power to them.
Just make sure you take HeadFi advice with a bucket full of salt. That place is where one opinion is posted and everyone passes it around like it was their own experience and where post count = credibility and absolution.
I do understand how you make a difference in WHERE you are listening to music. If I am on the go, AAC would be fine, especially considering the headphones, road noise and other factors. However, at home with my stereo (which I am a bit proud of... not going to spout specs) I can absolutely hear a difference. The best way I describe it is that the lossless files seems more alive. Sort of the same between analog and digital sources. Analog sources like a turntable, or tape just seem much more alive and responsive than digital ones. You WILL get a higher end hiss as a result, but this is something you can filter out easily without clipping the overhead too much.
So I would agree with you for average listening, but make exception for a specific stereo listening setup (depending of course upon how good that setup is). So several factor determine what you can hear... I have mild hearing loss in my right ear from shooting in the military and hearing things go boom, sometimes at close range... so that is another factor, as well as age etc.
So A and B is constant based on what you uploaded? Only X/Y will be different?
Alvin Tan Depending on the settings you choose. Some software can be set to swap A/B as well.
***** thank you! I saw another video a guy using the same thing was playing all four of them ABXY and got a little confused.
Alvin Tan Usually you just do an ABX test, no Y.
+lachlanlikesathing. So If I Download A Video Off Of CZcams ) | For Example A Jazz Beat |
Could I Turn The Downloaded Video Into A Wav File An Still Get The Same Sound Quality As Is, On CZcams?
is there a Mac version of ABX?
app store has ABX test
One thing to note is that the app store app is only limited to 5 trials, which isn't really great for statistical significance unless you sum multiple runs. Also I have no idea how the app locks onto different sample rates / bit depths, unlike WASAPI which resets audio output for each file type. So the app works but it's not ideal.
***** that's true! I wasnt aware it was limited to 5! that's kind of lame. should be as many as you like
can you spell sibilance?
I appreciate this video but it would have been better if you're having random songs or at least random parts of the song cued up.
Boring video cut it to 5 minutes to make it watchable
Yawnnnnnnnnnnnnnnnnn