Audio Blind Tests and Listener Training
Vložit
- čas přidán 4. 03. 2021
- Let's settle the confusion among audiophiles on what listening tests are, what it means to be trained and produce valid audio comparisons. Results of blind listening tests are presented and as well as published research.
Text article on long term listening tests:
www.audiosciencereview.com/fo... - Věda a technologie
Amir, I think most audiophiles would be saddened to learn that critical listening skill and audio preference are two different things.
Frankly I’d be scared to have the power to pick up on such anomalies for fear of forgetting how to enjoy my music. In this case ignorance can be bliss.
That’s why we have you sir!
Very interesting topic though.
Indeed.
You should listen to more vinyl. You can sometimes hear a bit of mistracking but still enjoy the music.
''That's why we have you sir!'' to the man who considers 100dB SPL a comfortable listening volume...
i guess Im kinda off topic but does anyone know a good website to stream new series online?
It's amazing what I can learn when I move my ego out of the way. Thank you for sharing your knowledge.
:) That is a very kind way to put it.
Inspiring, Amir. Could you pls start an online course on critical listening? We (ASRs) all will be grateful to you..!
Are you sure you want to be a trained listener? You will start to hear artifacts that maybe are not bothering you right now. :)
Evening Amir
I am definitely interested if you could teach and guide me
And even would love to pass on the knowledge later on
@@AudioScienceReview it would be amazing, you should absolutely do it!
@@AudioScienceReview A bit of; Be careful what you wish for?
@@AudioScienceReview I thought about this a few years ago. I decided I didn’t want to know, I’d rather just enjoy music.
I can't thank you enough for the stuff you do for the audio.
Amir, this is a great video. I really enjoy the long format you have here. Thanks for your efforts! You're a gem to this community and I'm grateful you're doing this work.
I subbed to your Patreon. Worth it! You just saved me $100 on that amazon amp. I was just about to buy it when I checked your site and saw you review it.
Ah, that is very kind of you. I am glad you find these longer formats useful. I am so worried about them getting long.
@@AudioScienceReview A long educational video vs one that say "trust me, this is better cause I say so". I'll take the long one every time. Keep up the exceptional work!
@@Adrian-jp2kt Amen to that! Amir does a good job of getting the info out without pandering which is fantastic.
Amir has the gift for being understandable. Very educative.
Thank you kindly.
the amount of knowledge of your video is insane. I love all the debunking videos and definitely love all the informative information you put out there.
I'm soooo glad that I came across your forum but I'm pissed that it didn't happe 10 years ago :). Thanks million times for what you do!!
It’s really a blessing that I’ve found your website. Thank you for putting the time and effort into sharing your knowledge.
Awesome - what is great is how you explain complex statistical testing in lay terms. We need more!!!
Each of these videos is a delightful ride of entertainment, knowledge and insight. Totally enjoyable and I’m always learning.
And it is delightful for me to read such comments. Thank you for sharing.
Amir, this is one of the most comprehensive presentations on this subject that I've ever seen. Thanks for sharing!
This was an eye opening explanation on hearing audio differences. I think the main point I take from your analysis is that there are differences in certain situations but if they are so hard to identify do they make a real difference in the real world? I think that you have given me great confidence to enjoy my system for what I expect it to do well without listening for minuscule, inconsequential differences that would only detract from the enjoyment.
Please keep at it. All your videos are great, ASR as a whole is a fantastic resource informing many purchases I've made. We need objective science in high end audio and your methods and stringent standards are a breath of fresh air in a cloud of audiophile mystique.
Super. Thanks Sam.
Thank you Amir, illuminating once again! This educational videos are GOLD
So educational Amir, what a great channel!
Thank you. I am glad you are getting something out of it.
So telling.Thank you for this. I was wondering about this for years now. And glad you took the time to offer many of the exhaustive variables that can be the case with abx, training the ear and the pertinent and specific ways to discern differences that are dependent on said variables. Very useful “rant” (more like an informative and much needed lecture) 👍🏼
Your videos are fantastic. Thank you for posting!
Thanks for watching Mathew.
Such valuable knowledge that you are sharing, Amir. Thank you!
You never fail to fascinate me!!!
:)
Very informative video Amir. You explained something which none of the Audiophiles take into account or are unaware of ! "Human performance limitations".
Because unless someone study in deep about the limitations humans possess, everyone thinks their senses are fool proof. I work in aviation and there has been alot of accidents in aviation due to humans believing in their deceiving senses and discarding the information instruments provide. At the end of the day instruments were right most of the times.
In audiophile industry this effect is not of a catastrophic nature like in aviation but again it can steal a lot of money from you.
As we saw in the tests you performed, human senses can be accurate given that the conditions they take the tests are pristine. A calm room with an in-ear monitor after a well rested night and analysing test subjects without delays in between using the full capabilities of humans such as echoic memory. But once surrounding conditions become anything less than pristine, our senses starts to become unreliable at an exponential rate.
Again unmatched content Amir. Thanks
Well said. As humans, we are optimistic beings, thinking we are right far more than we really are!
Great job Amir as always ! Keep up the good work.
Will do. Thank you Luigi.
Fantastically informative. Thank you!
My pleasure. Thanks for watching.
Fantastic video - so happy for clear thinking and DATA!
Thank you for the effort and also the explanation, it was a very enjoyable experience.
this is such an excellent video; thank you Amir!
Thanks for the kind remark and feedback. I need such input to decide what future videos to do.
@@AudioScienceReview while the debunking ones are certainly fun, the videos based more in education are the ones which i personally find the most appealing.
OMG it's THE Amir nice to finally see and hear you speak after reading on ASR for the past 6 months!
Amir thank you for being a honest enthusiast. I always felt that it is always in short listening A/B that I could identify any qualitative difference. You just reinforced my experience.
So happy to come across the channel. Great work Amir. I thought the cable myth had universal acceptance now. Genius job of marketing by those companies. "You can't hear the difference and you call yourself an audio-pile?".
Wonderful video! You are doing a great job. Thank you!
Ah, very kind of you. Thanks for watching Reiner.
Very useful info
More audiophiles should understand this
Amazing, i learned a lot ,certainly I didn't know long term memory was so unreliable,quite instructive, even 40 minutes of video felt short. Maestro! Gracias gracias,what a great teacher.
Thank you so much.
Great tips. No wonder why I could never hear any differences between 2 DACs. 😄
👍 Thanks Amir for this enlightening video. I really like your scientific way of approaching audio! Wish there were more audio guru’s like you around 😭
Amazing as always
Correct me if I'm misunderstanding but my take on this is that for normal 'pleasure' listening the difference in quality is in most cases so subtle that it is not really worth worrying about.
Yes! Amir passes the tests because he focuses on very tiny details and repeats them over and over. You are not going to do this in normal listening!
Thanks again for sharing your knowledge and experience. That was a very informative and interesting presentation.
Thanks Peter. Can't believe how many of you appreciated it. When I finished it I thought for sure I would get a lot of negative comments about it being too long, boring, etc.
This is your best video yet. My personal experience is consistent with your description of mp3 compression. I bought a Diamond Rio 500 when it hit the market and collected and shared Napster files, including my own recordings. Then, with a 128MB library constraint, I could obviously hear differences in compression. I also relate to your embracing of scientific method for better compression rates.
Diamond Rio? Man that brings back memories. i wonder how many people know that the iPod is distant descendent of that player.
I found this video useful, but also enjoyable. This is actually my favorite video so far in this channel.
I hope you'll do many videos on home audio products that are excellent performers at relatively low prices. I'd be thrilled to be able to look at this channel for the best of any component category at prices that regular working people can afford.
Thanks. Will do.
Thank you so much! It's been a great help for me as a mixing / mastering engineer to run both analog devices and plugins through an analyzer to figure out what is really happening and what to listen to in the sound. The pro audio market is also flooded with false claims and fancy descriptions by industry veterans how this piece of analog emulation software is a must-have in your toolbox and emulates perfectly this analog gear and yet it's just a simple bilinear transform without even harmonic distortion.
Indeed. Consumer hifi people have this impression that pro people 100% follow science. As you indicate, the same confusion exist there about what is real and what is not.
Fantastic! Learned so much here.
I am glad. Thanks for watching.
@@AudioScienceReview Was talking to my far more knowledgeable audio friend, and he remarked that even IF someone had golden enough ears to be able to generally hear a difference in the noise floor between 16 and 24 bits, the self-noise and natural distortion of most consumer-available audio equipment would overshadow that difference anyhow.
And as we already know, tracks at 96000khz, besides having content that's audibly imperceptible, suffers from the intermodulation problem, as fabfilter and Dan Worrall's wonderful demo shows:
czcams.com/video/-jCwIsT0X8M/video.html
Thank you teacher! Great work.
Hey Amir, thanks for making my suggestion a reality. Listening right now :)
Humbling. Thank you.
Great video and very instructive, this is science!
Thanks Denis.
Echoic memory on human is super fast, It lasts only 2-4 sec and then it's gone so fast ABX is needed like Amir says. So take like cable testing, normally its too slow, it takes around 2 minutes to switch cables. The testing is flawed from start.
if some cant tell the difference without a delay, then the difference must be so minimal it shouldn't matter what cable they are using as their wife coukd swap them out and make some money of them
@@I_drive_over_dogs_n_dont_stop thats the point. We are talking about so small differences that those wont do any good for you at the first place. Like DACs. Its the last thing you need to upgrade. A 200€ DAC will be good enough for the majority of system's. When you hit the last stop of end game. Like on headphone over 2000€. Etc.
@@Thevikingcam Honestly a headphone past £500 isn't worth it for sound alone providing its a well tuned one.
@@Jordonater what?? Are you kidding? Right? Hahah... Have you ever heard STAX 009 or ABYSS?
People literally cries when they hear their favorite song from those. And they are not audiophiles but normal people. Many of my friends are in disbelieve how headphones can produce sounds line that and they are looking around the room of someone is actually playing instruments in the room
This video is so informative.
Thank you. Appreciate the feedback very much.
Thx 🙏 Amir. Very interesting and usefull video
Amir. Thanks so much for taking the time and sharing your invaluable experiences. Considering the expensive rabbit hole of audiophilia, what you are doing is critical in educating music enthusiasts. Before anyone decides they want to go through training, they should be warned that once you develop a hearing skill, it's very hard if not impossible to turn it off. I won't go into detail of what happened to me as a child but, if I am in a room full of people having separate conversations, it's overwhelming. I can hear every conversation. It's an extreme example but, my philosophy towards my equipment for listening to music has always been, do I enjoy the music? Simple! Also, unless you have personally attended a recording session, you have no idea what the session sounds like live. It's a shame that there is so much misinformation and fraud in the audio industry. Keep up the great work!
Agreed and thank you so much.
For this interested in how audio codecs are evaluated in relation to transparency. Look up Mushra. This is an ITU framework for doing that type of evaluation. Harmon had a good tool for teaching and selecting critical listeners. Search ‘Harman how to Listen.’
As an interesting side point. For video this is much easier as we have more standards for capture and display. There are tools like SSSIM, VMAF that can look at a source file and compare the encoded file against it and it gives you a score for the delta in quality. It has some blind sides but it generally works. Where Mushra requires people to get scoring. Yet, there isn’t as sophisticated a psycho-visual model as the one for psycho-acoustics. In video (my area of expertise) we’ve stumbled into techniques that remove information without as sophisticated a model as audio codecs (e.g. motion estimation, quantization, etc.).
Just discovered this channel and it's quite a revelation. I'm not used to hifi being discussed in rational and objective terms. I like it.
Thank you Nick. These are topics I have discussed online in text for 10+ years but I am finding that video format is so much more effective in conveying them.
Hello Amir: I have a suggestion for a debunking video. Directional RCA. Now even the car stereo manufacturers are getting into selling such. The worst part is that there are ones that consider it as gospel. Please consider it.
Sure, I will do one on that. :)
Hey Amire, love your vids that you have been putting out so far. I was wondering if you could do a video about your whole testing process. Like how you plug in the machines, how you do your listing, or how you plan on getting dacs and amps to test. Like a "tour".
Ah sure. I was thinking of doing that. The only barrier is that I don't do any edits of my videos. It is one take and upload. This means there will be dead periods where I wait for tests to load and run. Wonder if people will put up with this.
you are 100% right Amir - the Sommelier is the best confirmation how right you are
Thanks for these videos. When I was young I was interested in Hi-Fi for as far as my budget would allow. Recently I got my attention again and I bought some better equipment. Not super expensive, but a step up. Started watching CZcams videos on the subject and fell into the Rabbit hole of watching videos of what seemed like experienced and knowledgeable people. They had nice graphs, seemingly well informed opinions.
Until I came across a video of one of them actually promoting audio grade Ethernet switches. I'm in IT and knew this was nonsense. Then discovered your videos with some more items debunked such as absurdly expensive power cables etc.
These videos explain quite well that this is nonsense. Even better to me is the explanation on where they go wrong and why they think they can hear a difference when there clearly is none. I now don't trust their judgement anymore on other topics. If they don't understand how this works, how I can trust their hearing when they claim some devices sounds much better? So I guess the veil has been lifted for me on those 'experts' ;-)
Your videos and reviews are very valuable: objective measurements, supported by critical listening.
This is a great video; Amir may be heavily associated with measurement-based judgments of audio equipment, but this video clearly shows that he has phenomenal ears as well.
First time I hear someone speaking about critical listening in a way that makes sense.
What I got from this:
No issue with me not being able to discern audio from Spotify vs lossless.
The difference is not always present and is freakishly small. If I wanted, I could learn to discern it is super controlled environment (listening to the same couple of seconds of a track for a while).
Going the extra mile (and $) for state of the art devices and lossless audio files will have make an objective difference I could train myself to hear.
Thank you Amir!
Very informative per usual. I will attempt to be less dogmatic about people being unable to hear the difference between various compression levels. If you're attempting to use the P values of 0.05 on the tests often seen in papers as being 95% confidence, you'd need to get 19 out of 20 listening tests done in the ABX. 9 out of 10 (90%) as you mentioned would probably convince most people the person was accurately telling the difference, it would not make the p 0.05% stat sig needed by usual standards. Been a long time since I did bio stats, etc, but I believe that's accurate assessment. I found the discussion the time period between them being essential especially beneficial and long term listening tests a waste of time.
Thanks for another great video, this is fast becoming my favourite channel. I'm curious as to whether you think 16/44.1 audio with dithering is sufficient or not as an end format. Personally I've always thought so on theoretical grounds (although I record and mix at 24 bit or more), and when you showed your 16 vs 24 bit results I was initially surprised but, as you yourself point out, cranking up the volume and doing the ABX test on a fade-out tail where there is only about 8 bits left in the 16 bit recording doesn't really prove much.
Was very interesting to listen to.
Brilliant!
This again is an awesome video Amir !
Any possibility you could teach me something please ?
I'm all ears ... 👌
Thanks for sharing Amir.
I have worked on thousands of sound systems. People regularly allocate their funds and effort incorrectly based on marketing and internet hype. There is a great gap in perceived value versus real value. I wish people would focus more effort on room problems and speaker performance.
I recently was helping a client who wanted better DACs when his speaker locations were poor. I doubt he could hear the DAC changes, but I am sure he would benefit from moving his speakers and subwoofer.
Indeed. I routinely get questions on the forum if someone should upgrade from an already great DAC to even a better one and my answer is always no. Speakers are so important.
@@AudioScienceReview so true. I point out that sound reproduction is about the room, speakers, electronics and media. The lowest performing part will limit your sound. It is amazing how many people are obsessed with electronics over speakers and the room when most people could get so much more from the room and speakers.
Everything fell into place when you mentioned echoic memory - a true eureka moment.
Exactly, for me too! I'm thinking for year's now that our hearing memory is really bad. I'm thinking of that every time i see a headphone review where the reviewer compares two or more headphones from memory. It's bullshit, and now i know why. Thanks Amir a lot!
@@PappLacc
That's nothing! I saw a CZcams video where violinists were playing their million-dollar Stradivarius blindfold verse a copycat that cost maybe $1,000 and they couldn't tell one from the other and these were very seasoned violinists!🤯🤯🤯 It was nuts. A lot of the times the chose the fake as their own $$$$$$$" violin.
Impressive, this is precisely what audiophiles require, rather than merely the opinions of Andrew and his spouse.
Very nice, thanks for this information...
Once done a test with a friend, same song 96kbit / 128kbit / 256kbit / 320kbit / ... flac. To 192kbit he could here the difference between the flac and the mpg3, but with higher rates very difficult. It was on a good system....
Yes. We tested audiophiles at Microsoft with respect to lossy compression artifacts and they did no better than general public and well below our trained listeners.
I hated MP3 128 encoded files in the "old days" because I could tell they sometimes had weird artifacts and some instruments seemed muffled (I'm no expert so I guess that's the compression on some audible frequencies). When I could, I would always chose VBR with the maximum quality possible and those were indistinguishable from plain CD quality.
I can’t believe I just watched a 44 minute video of Amir just talking. It was so good I rewatched watched it! I find audio reviews utilize impactful adjectives and typically conclude “that this (speaker, amp, DAC...) is a great value worth many times it’s price... “. Not the case at AudioScienceReview.
It is great to hear you tolerate 44 minutes of me. :)
Amir, I was testing this yesterday afternoon by changing the bit-rate settings for the audio output within Windows 10 and there sure is a huge difference between 8 bit (telephone) sound and 16 bit at any sample rate. But, the difference between 16 bit studio and 24 bit studio was slight. When I watched this video this morning I became so glad that I have been watching your videos and I very much appreciate your life time of experience. Thank you and please keep up your great efforts.
I would be very interested in seeing an O-scope recording of 2 channels monitoring the output of 2 items playing the same audio at the same time. I wonder if cables could be proven out this same way. Thanks again Sir.
Great to see you experiment. On your suggestion, devices can run out of sync quickly due to their clocks drifting so such comparisons become hard.
@@AudioScienceReview Very good point sir. Thank you. I far too often forget the finer details like passives drifting due to heat and load. However, I think a test like strip charted audio output for a side by side comparison would be the definitive proof of audio qualities and quantities.
I took Ethan’s converter test twice a few years apart and failed both times. This was the Soundblaster, Lavry and Delta test. I didn’t know you knew him.
Great stuff, Amir. I am a EE by degree, expert enough in filters/DSP to understand the theories and moderately (not very fancy) audiophile listener. To give you a flavor of what I listen to: my home theater system is a Denon AVR-1801 Dolby receiver and I have 5.1 channel audio in the living room two Marano speakers up front I’m a rush center channel speaker and two Boston Acoustics rear speakers in the walls) with a Klipsch subwoofer - so that’s not “audiophile” necessarily but it makes BLU-ray movie soundtracks sound really good in the room. In more critical listening situations I have about $100 or so cost Sennheiser headphones or I think about a $50 Apple earbuds that I sometimes will listen to my iPhone 11 Pro with ... and I’m listening to music or meditation type of music for enjoyment; I guess I would say I’m not too critical a listener. I have tried some ABX tests several times trying to concentrate totally and it is hard to tell a FLAC file apart from an uncompressed WAV file (really hard!). What I like is how you explain things, very balanced, just-enough-engineering terms. I like that you’ve done ABX testing and explained them well but you also know that you can be a critical listen there to tell 16 bit file from 24 but file with the fade-out/noise floor. In general I just love your channel Amir! 👍🏻😃
Thank you so much Curt. It is great to see other EEs finding value in what I have to say.
very interesting, that was enlighting. I would argue that the type of testing you are doing is by knowing what is being tested you zoom in on minute details that you expect to change and with your training you can discern it. In normal, recreational listening these differences are irrelevant.
I agree that it costs nothing to have the transparent quality. But then it should not cost more (sometimes a lot more) to get the 24/*khz files, because it didn't cost more to produce it. Let alone all the issues with high frequency noise etc that you have shown in your analysis of highrez files.
When Soundblaster introduced their stereo sound card I bought it. At the time this was the best sound you could have for computer gaming.
Thanks for this great videos.I would like to ask a question. Let's say you have 2 audio files to compare and they sound very similar. You can't spot and hear the difference with ABX blind test. "You can't hear the difference" doesn't mean that other people can't hear it,too. Hearing depends on where you listen, how you listen, how loud you listen, your hearing skills... etc and all these change from person to person. So here is my question :
You have 2 very similar audio files. You can't hear the sound difference between them and you want to know if the sound difference between these files have a chance to be heard by some other people. Is there anyway to make an assumption about the difference between 2 compared files is hearable or not ? Is there some kind of threshold any kind of analytic data or scientfic research where you can say this sound difference has a probability to be heard?
With audio analyzer softwares you can have rms, peak,lufs delta... like values or have spectrograms, spectrum analysis graphs... Is there any kind of information hidden in there to define a threshold for hearblity?
Note : I'm using deltawave audio null comparator software for this kind of comparison. deltaw.org/
Any kind of help with scientific approach is much appreciated.
I knew this before, nothing new LOL Amir
This is impressive...
Its interesting regarding the subjective quality of compression. With properly encoded CZcams music I cannot tell with real certainty that there is any significant loss when comparing to a high quality source recording it came from, even with good headphones (and good hearing). These algorithms are better than people give them credit for.
The artifacts are indeed small.
I remember seeing a blind test some years ago using a metal coat hanger in the speaker lead in one of the set ups, can't find it now. The audiophiles didn't like the outcome.
I researched that and turned out it was a fish story. It was just an audiophile meet and someone suggested to use a coat hanger and they did. It was not any kind of format blind test and such. Based on that though, I did test coat hangers and found them to be quite poor conductors: www.audiosciencereview.com/forum/index.php?threads/when-12-gauge-wire-is-not-12-gauge.3/
Fascinating and educational video. Many thanks for posting it. Perhaps we need to start with a more basic question: why is it advantageous to try to train yourself to be a "better listener" and be able to, for example, tell subtle difference between a Single ended tube amp and a state of the art Class D amp or an MP3 and wave file?
The idea of training yourself to hear differences begs the question: for most people, what's the point? If you can't tell the difference between an MP3 and a wav then the difference does not exist. It's not that you just can't tell the difference.....it's that the difference between the two *does not exist* (for you). To claim anything else is a-kin to claiming the color red exists for a person who was blind from birth, just because sighted people claim the color red exits. The experience of the color red does not and can not exist for a blind-from-birth person. Perhaps unlike the color red for a blind person, brining into existence through training the experiential difference between a single ended tube amp and a state-of-the-art class D amp (for example) seems to have little benefit or utility (except for people like Amir where it helps their job). I'd go so far as to claim that training yourself to hear such differences may be disadvantageous and thereby harmful for most people. Gear selection may become more difficult and expensive. Less enjoyment from perviously fine car radios (for example) may limit the scope of your enjoyment of music to certain places and times. Etc... Perhaps the fundamental claim I'm making here is: ignorance may indeed be bliss in this case.
just curious if you can differentiate 2 devices in a ab testing blind test ( not in focused ab testing leaving just a few seconds between the 2 devices but in a normal casual usage just sitting down and listening to music, meaning leaving at least few minutes between 2 test so you are not using short term memory) thanks
Amir this was an interesting video on blind testing and critical listening. As you stated you knew the weakness of the system under test, (i.e. bit depth), had the training, and experience to identify the impairment.
Do you know if anyone has ever done preference blind testing to see if there are significant statistical differences? For example, instead of telling the subjects what technology is under test, hi res vs. low res, tube vs, solid state, digital vs. analog etc. The subjects would be asked which musical recording they preferred, whatever better meant to them. We could then see if one system was preferred as a subjective experience, or no difference, without asking subjects to identify technologies or impairments.
Since music is a subjective experience could distortions, lack of distortions, or some combination be found more, or less pleasing to the total experience of listening to music, without being able to consciously identify specifics?
An example from visual perception is if you want to see the most accurate colors the light would be calibrated to the 5000 Kelvin standard, or what we would call daylight. However, humans have a negative emotional reaction to daylight lighting and prefer what is referred to as soft white in the range of 2700 - 3500K even though it distorts our color perceptions. I know If I warm my photograph's white balance viewers prefer them to “accurate” colors.
Amir, are hi-res streaming services, eg Tidal, Qoubuz, Amazon Hd, BS if they tout superior and better quality than normal streaming services?
Will that be the next test?
I have Spotify and Amazon HD and can’t hear a difference between Hi-Res and compressed music
very interesting. i would like to tell you some of my experiences as music producer and "Audiophile?" in my genre i am well known for the sound quality of my mixes and masterings. got an award from keyboards magazin for that. i was self educating and always in "science " to improve my production quality. a friend of mine has a well known and awarded high end shop in germany. so i had a lot of contact with snake oil and their "audiophile" users. i was alwas challenged to heal them somehow. i found out that they dont want to know. they want belief. and they dont want to listen to the original. they want it with sugar. some of you where talking about rme and marantz dacs. there is no good and bad in sound for a producer like me. there is only right or wrong. audiophile dacs and preamplifiers in most of the cases useing a wider stereo image and rolloff filters. in many cases the subjektive listener likes that. with no chance to compare to the original. as producer i can compare and i was shocked when i was first listening to some 2500euro dacs. i offered a comparison with my rme. at this time i owned a fireface. conclusion of the comparison was that the listeners did not like what they was listening to. the fireface had 8 out and inputs and was less than half the price of an high end dac. and it was the only dac in this comparison that gave an exact playback of what i have produced. all the others made mistakes on the soundstage and the roomsize. but the listeners didnt like the sound of the rme. i said, but the rme has no sound. i startet to search for arguments. and than i had the idea. one day my friend came with different audio cabels and wanted me to judge between them. not possible when you change the cable on your system and try to compare that. however few years later he came with silver cable and offered me endorcing. i didnt belief in cable, vinyl, i want to know. and so i found a test that seems to me nobody else did. i used a track that i had produced on my studioworkstation. i used my production interface rme fireface to playback that track. but not with an amp. the cables went direktly back into the rme for resampling. after that procedure i had 3 files on my daw. 1.the original, 2. one resampled with big crystalin silver connection 1.5 meter. 3. standart studio patch cable. i ve put these 3 files into cubase on three stereo tracks. loudness calibration. and than i was able to switch between these files in less than a second. do you want to know the conclusion? untill this day i had doubts about my fireface. i had some nice analog gear that i wanted to use for mastering. but the resampling process in that analog mastering eat the whole advantage of the expensive gear. on that day i found out that it didnt was the resampling that let my masterings sound metallic. it was the patch cable. i am aware to say that the silver cable makes the difference. thats maybe the wrong interpretation of a bad patch cable. the silver cable was also sounding a little bit different compared to the original. the bass was a little bit slendericed. but this effekt could also came from resampling. turning the phase of the tracks confirmed what i was listening to. since than i use a set of these silvercables for productions. for line and microphone. i ve got also two pairs interconnects for my audiophile soundsys. i use them between dac - preamp and amp. but i wouldnt have bought them. for hifi and studio it makes more sense to treat the room and, or to implement a room correction. . maximal effekt , half the price. the other topic was 16 or 24 bits. is it possible to hear a difference. from my experience shure, yes. but it depends very much on the production and the gear. once long time ago i was promoter for basf. i offered people a a/b test with cd and basf audio tapes. most of the coustomers where not able to seperate cd and tapedeck. these a/b comparsions between mp3 flac a.s.o. reminds me to that. the promoter was ordered to use 3 different tracks for this test which were selected by basf. with these tracks i was also not able to seperate them. let me choose the track and if your ears are okay and you follow my advice you will seperate them. but for a unexperienced listener and also for the most musicians its not possible without advice. even in my tests with mastering clients i was wondering that they prefer bader sound. they belief that loud masterings sound better. they love disortion. when these people check mp3 - wave they often like the mp3 sound more. the same effekt as 16bit,44,1 - 24 bit, 88,2 . one file containes mor information. to realise this more of information you need to konzentrate your sense also when you are used to it. also the masters fail when they get louder files. or easier files. i make all my production each single track in 24bit 88,2. sounds good. i give my musik a virtual room with convolution reverb. i like real rooms with reverb time fits to the tempo of the track. imagine , you use a wooden studio room with 8meter walls. you place all the musicians and intruments. the room is big. they have all space between each other. when you close your eyes you can see them. when the production is finished it comes to mastering and the track must fit on different media. so, downsampling for cd and in most of the cases also downsampling for vinyl! yes most of the vinyl companies work with 16bit. however, when my work is really good i hate this moment. all the work on the details like the room are nearly gone. you can still see them when you close your eyes, but the room in which they play has only 4 meter walls left now. and that effekt is obvious . when you now compress them to mp3 and you close your eyes, the musicians have reverb. on an average production the room is gone. try it for yourself. take a track with bigger soundstage than the space between your speakers. guess how big the stage is on which they play. compress it and listen again. thank you for your patience
Isn’t it amazing that a “better” performing product always costs substantially more than your perfectly adequate product you already own?
Very, very interesting
Thanks Amirm
I have 4 dacs, I can not tell the difference between them. 😅
Still enjoy Your measurements, reviews and scientific approach 👍
I tried mp3 320 vs FLAC with my friends, nobody heard a difference 🙂
Great video again! So one if the conclusions for me is, that most of "differences" between components like amps, CD-Players, Dacs.....(if they are not broken) won't be detectable in a normal listening environment. At least not via speakers. Any difference that could potentially be heard (if there is any) would be covered by e.g. environmental noise or you would be fooled by expectations and/or your memory meaning you are failing to do a real ABX at all.
The only components that can reliable be detected/judged is the speakers (of course here's where your personal taste comes into play) and how these "behave" in the room. So that's where the money should be spend objectively. Of course there are many other subjective reasons like look and feel that make me and you spend money on components. That's not a problem at all, but it becomes a problem when people brag about their latest 5k DAC they purchased that changed their whole system....
You need to go to a high-end stereo store and have some AB testing done and you will tell differences in amplifiers easily.
Really interesting Amir, thanks. If you haven’t already I would love to see a video on Bluetooth codecs and audio quality. Are they measurable in anything other than bitrate and latency?
I was hoping to do a deep dive on BT codecs but I am stuck not having a convenient transmitter that lets me control all the parameters. I have done straight measurements of them which shows bandwidth limiting and such: www.audiosciencereview.com/forum/index.php?threads/qudelix-5k-bluetooth-dac-headphone-amp.17386/
@@AudioScienceReview ah ok, thanks for the reply. Will watch it now. I’m currently waiting for my amplifier to be returned and then I’m going to try and blind test AAC, aptx and aptxHD if possible. Also going to compare vs wireless streaming from the same amplifier. I suspect I won’t fair anywhere nearly as well as you did here!! 😀
Amir, do you think it's possible to hear differences caused by various IC cable insulation materials (rubber vs. PVC vs. polypropylene vs. teflon) while listening to cymbal's sound decays in good quality headphones. I believe I kind of did when I had compared a few (shielded 22 AWG twisted pair) Belden cables with different insulation. :) However, that was just non-blind A/B test.
Btw Amir in a scientific context, if you take a significance level (alpha) of 0.05, You follow that religiously.
So 0.055 would mean that you failed that test. i’m not saying that you actually guessed, you would probably pass if you did more tries.
Hi, is the hidizs s8 still your top recommended dac amp in that same budget class? Just curious of an update from all the dac amps youve tested. Or maybe the s9. Is there a tempotec version of the s9?
Thanks
For a portable DAC, yes. There is a tempotec and someone was trying to get one for me to test.
@@AudioScienceReview thanks for the reply I appreciate it
another great video, can a graph show those subtle differences between 16/24 bit 44/88/96/192...?
Thanks for the video Amir! It's great to know that you worked on audio compression in the past as I was pondering about it recently, perhaps you could point me to some further reading. As I understand it, raw audio files are like a table of amplitude at each sample point. If so, how does removing certain frequencies deemed unnecessary affect the final file size? Wouldn't you still need the same amount of samples at the same bit depth? I assume the final compressed file is read a lot differently and I'd love to hear more about it.
No you wouldn't because the output is not an audio file at certain sample rate. It is just data that point. Is this what you were asking?
@@AudioScienceReview I see, I’m trying to understand what happens under the hood rather than the actual effect on the music. I guess I will need to read more about compression in general.
I had the same experience. Mp3 320kps vs FLAC or WAVE files, always think i could tell but never on a consistent basis.
Great work. Do you listen to speakers you review before you conduct measurements to avoid measurement bias?
Thanks. I should do a video on your question. :) Answer is no, I measure first, listen second. The reason for my listening tests is not to give you a controlled listening test of the speaker. That would need to be a multi-way test against other speakers. And at any rate, the test is sighted whether I look at measurements first or second (these non-measurement biases are far stronger by the way).
I listen because I want to verify that the measurements are telling us what they seem to be saying. Say I see a dip from 1 to 2 kHz. I listen to my reference tracks and form an opinion (without measurements in front of my by the way). I then apply an inverse EQ to fill that whole and then do an AB. I sometimes do this AB test blind. If the EQ improves the sound then I assume our interpretation of measurements is correct. If not, then I investigate more.
In other words, listening tests are not presented independent of measurements. They are part of the measurement suite.
I think the concept of someone giving us subjective listening tests of a speaker sighted without measurement is flawed. You are relying on them being god like in knowing everything about tonality of a speaker and giving you a perfect opinion. I am good, but not that good. :) Why would you trust my opinion of listening to a speaker anyway? What makes you put weight on that assessment? Is it a test of manhood to see if you can catch me saying something that doesn't agree with measurements?
With measurements, I am a compass. I know what the speaker is doing that is out of norm. The listening tests *and* equalizations are then there to confirm and pinpoint the audibility of those artifacts. Is it a perfect process? No. Per above, I am doing the test sighted so there are a number of factors which could pollute the results. It is just that I feel the confirmation via listening tests need to be there. Otherwise you all will be subjected to do the same if you buy a speaker based on measurements alone.
A bonus which comes out of my listening tests is equalization that you can then apply to the speaker without doing the work that I do, or the experience I have.
@@AudioScienceReview well you kind of say it above, "if the EQ improves the sound then I assume our interpretation of of measurements is correct, if not I investigate more." - in an objective world the measurements are gold or not?- why do we listen? it assumes flat frequency response and minimal distortions are the absolute sound. Not to be argumentative, I found the video very good. But it does make wonder what the value of trained listening is to the consumer, you have 20+ dacs in your list of measured dacs all measuring within a point of whatever metric you want to establish, SINAD, etc. Are they then all interchangeable? Or are there differences in sound? Is a purely distortion free and artifact free dac desirable or do some distortions sound better psychoacoustically? I can appreciate that this has been a moving target since the tech has changed enormously and maybe we have hit the point where there is not much more we can get out of a dac and all of our old comparisons that showed differences in sound were on dacs that measured far worse. And I'm happy to spend less!:) if I don't have to...thanks for the videos.
This is something the elitist audiophile groups should watch… although they wouldn’t as it threatens their self proclaimed ability to hear differences in absolutely ridiculous products that don’t make a difference, like a £10,000 power cable or a £4,500 “audiophile fuse”… yes really, I can show you exactly the brands sell those items for that price. Actually, I found a nordost pwoer cable which sells for £17,000…. Seventeen thousand pounds for an AC power cord.
Thank you Amir for a very humbling video. You are a gem to the HiFi community
Hello Amir, thank you so much for being wiling to share your knowledge. Do you find imperative to do these critical listening test with headphones or can a good stereo system do the job?
And by "good" I'm not talking about 10k $+ crazy expensive equipment.
In most cases, headphones are more revealing due to blocking of the noise, and generally lower distortion. Sometimes though bass is impacted and there, speakers are better for obvious reasons. In research headphones dominate for hearing small artifacts.
7:28 "...I stopped paying attention to what music and its beauty is"
Well at least you realize it
?🤔
Fascinating video!
On the topic of being able to hear differences between standard audiophile format bullshittery, do you believe one can hear the difference in absolute polarity? I have taken the absolute polarity test thing over at audiocheck I could hear an obvious enough difference to get 10/10 or 9/10 on every try. Is what I am hearing really a difference between a normal and inverted phase signal or is something else happening here?
I can attempt to show you proof of me passing this somehow, but it was very strange to me to be able to hear the difference as I thought absolute polarity did not matter in the slightest in terms of audibility. Similarly to how you were able to tell 16bit vs 24bit, there is just one particular part in the two files that I can tell the difference in. If you took that part out I don't think I'd be able to tell.
I have not taken any polarity tests so don't know. How as the polarity changed?
@@AudioScienceReview It looks like they just inverted the polarity of one sample and kept the polarity original of the other. I imagine that the two samples would sum to zero. It was to simulate a piece of gear with inverted polarity, which used to be strangely common and still happens to some products like the original Topping E30. You can find the Absolute Polarity Blind Listening Test at audiocheck, sadly youtube really doesn't want to let me post links here.
The way I am able to tell is the little pause after the fourth strum when there is a bit of a "slam" sound as if something was hit. It is subtle but in the inverted sample it is boomier and overdriven - I compose music as a hobby sometimes and it sounds to me like using a tube amp simulator and cranking the distortion too high. It sounds grainy and unpleasant, while the original sample (in that one moment) sounds leaner and cleaner.
But like I said, if you removed that one moment, I wouldn't be able to tell at all. Here's a recording I did a while ago to show what exactly I was listening for and how I could tell:
streamable . com/pn4nqv (remove the spaces)
What file format do you recommend for untrained listeners; 320 mp3? cd-quality flac? 24-bit? Are huge high-res files like 24/192 and DSD etc. a waste of time and space?
Thank you so much for making all of these videos. I hope they will help the audio community move away from things that don't matter (like overpriced cables) and towards things that do.
Hmm... are you stalking me? I just talked about echo memory today in one audiophile group, in this exact context.. It is good to have these videos, i keep posting them strategically, hoping that someone will learn something.
Nah. I am a mind reader in spare time....