Are ABX tests valid?
Vložit
- čas přidán 27. 05. 2018
- ABX blind testing is supposed to be the be all end all in testing but Paul helps us understand its strengths and its flaws. Have a question you want to ask Paul? Head over to our website www.psaudio.com/ask-paul/
I am getting close to publishing my memoir! It's called 99% True and it is chock full of adventures, debauchery, struggles, heartwarming stories, triumphs and failures, great belly laughs, and a peek inside the high-end audio industry you've never known before.
I plan a few surprises for early adopters, so go to www.paulmcgowan.com and add your name to the list of interested readers. There's an entire gallery of never before seen photos too. - Věda a technologie
That's crucial: not knowing what A is or what B is. Get rid of bias. Audio magazines will never dare to do this regularly. They have to fear that a 500$ cd player outperforms an 5000 $ cd player. If they let that happen, they're out of business.
ABX texts being uncomfortable is a bad excuse. You're doing the test for yourself, not for anybody else, just design it in a way that's not uncomfortable! You don't need a guy to stand there and pressure you, you can usually use an electronic switch or a computer to do the switching and listen at your own pace in your own environment (unless you're testing loudspeakers, but that's super-difficult with any blind testing). A blind AB test may be better than a sighted test, but it's still nearly useless if you don't have a tangible proof that you really hear a difference. And I'm saying this as someone who's done ABX tests in the past, still occasionally do them with amps and lossy codecs.
No, it is not bad excuse. Some people do become anxious when being tested, and it psychologically affect the way they do things. Some people perform better in a test, some people perform worse. The fact you didn't know that and think it's a bad excuse is astounding.
And he did say his company do ABX testing, and you still think he's making excuses.
@@limitlesssky3050 Yes, he's making excuses. But that's the business he's in, selling a fantasy.
People just don't want to admit the existence of human neuropsychology in the minds (aka brains) of blind test listeners. Whatever you're using A/B or ABX, the test result is only as good and useful as the listener's state of mind and capability. So the focus should be on how best to ensure the listeners are trained, prep and supported to give them the best chance to perform, and hopefully with scientific basis to support. Of the flooded discussions on internet about blind tests there are so few if any mentioning how best to set up the listeners to succeed in these kind of test. One example is that: can anyone cite any science based researches in neuropsychology that shows that if there is a correlation between a) knowing what to listen for- expectations vs b) not knowing at all what to listen for-uncertainty, or no correlation between a) and b) in terms of diff discovery????? If one trying to be scientific pls then be 100% scientific from top to bottom and from start to finish and show me the solid evidences from neuropsychology ..
happy to have found abx testing for different bitrates of music (lossless vs high quality) .. it proved to me that I can't tell the difference .. (someone else might) .. and I dont feel uncomfortable with that knowledge -- in fact i'm happy to now know that i can't tell, save me money since i wont have to pay for Tidal HIFI streaming and can settle for the youtube music that i already have.
I've done some pretty extensive abx testing with my own equipment at home by myself so there was no "pressure" to perform. (That's what she said) Anyway what I've learned is ABX is definitely necessary to overcome the "placebo" effect. If you know that you're listening to something that is supposed to be better you will definitely "hear" differences but as soon as it's blind the differences disappear, unless you're listening to something with actual differences in which case you WILL be able to pick out differences easily regardless of it being blind.
How can you do abx testing by yourself without knowing what you are listening to ?
there are automated tools available for eg foobar2000
@@wildcat1065 foobar2000 abx component
(Yes, 5 years later)
I think Paul is being generous enough putting out his opinion, experience and jugement on the ABX testing and other matters. I have seen many of his videos, and some of them, I can say, go against the direct “marketing” or “business” benefit of PSaudio. But definitely show his genuinity as a creator and an engineer. Whether I agree or not with Paul’s opinion on ABX, AB, or blind testing is secondary here... Some comments or criticism are just inconsiderate to the fact that Paul could have just thrown that question and many others in at the bottom of the pack and avoided opening that can of worms! and the criticism. HE DID NOT!
So, with all due respect to the rest the doubtful, Chapeau, and thank you Paul for giving us an insight of how you work, think and evaluate your products, and mostly how you feel about it all!
Of course, the main point was missed. ABX is one possible format for controlled ears-only tests. There's lots of others, BUT... unless it is double-blind, a listening test is not even vaguely valid. DBT is the important thing, not whether it's ABX or triangle or sorting or...
If you can't hear a difference without peeking, you can't hear the difference. All the blather is commerce-driven excuse-making. And indeed, DBT tests show that we have fabulous sensitivity to frequency response, level, localization, dynamics, data compression... somehow, the excuses don't seem to apply.
Did you actually listen to what he said? He didn't miss the main point. He literally said if he knows either one of the equipment, the test becomes inaccurate to him, if he knows he's being tested, the test is also inaccurate to him, and if the pace of the song is not controlled by him, the test will also be in accurate. In fact his testing method is superior, since A, he didn't know he's being tested, B, he didn't know the equipment at all, C, he can control the pacing so he's more relaxed.
Possibly the lamest excuse I have ever heard.
Possibly the lamest excuse I have ever heard.
Possibly the lamest excuse I have ever heard.
Possibly the lamest excuse I have ever heard.
Qualified Listener is just as important as a "trained listener". If a listener cannot tolerate an ABX test then they are not qualified to give anyone else their opinion since they closed themselves off to being truly neutral and scientific in their analysis. Of there is truly a difference between components then a qualified listener will always hear it no matter how randomly the sound samples are. If the listener cannot hear the difference in an ABX test then the systems are either better than the listener can resolve or the systems are not perfect so the difference because subjective (i.e. they are so similar that the difference is seen more subjective than usual).
A "standard AB test" does not exist. We ruled out the ability of that method to work accurately back in the seventies because the second person (changing the system from A to B) invariably influenced the test, even if it was inadvertently. Humans are inherently opinionated and selective and cannot perform a neutral test, they simply can't.
ABX testing reflects the current state of the art in scientific method, if you're not going be scientific about your purchasing process then there are going to be "high end audio" manufacturers that will rip you off. Just think about it.
The flaw of a test is the test itself. If the testing is so accurate, why not camouflage the testing procedure, and makes it even more accurate? If you think interaction between human testee and human tester and the circumstances of being tested do not affect the variables in a scientific procedure then I think you do not qualify to tell people what is scientific and what is not scientific.
@@limitlesssky3050 That actually has been done many times, in various ways, for decades and the results support a well run ABX test. In fact my favorites is when there wires from a high end systems were substituted for more reasonably priced systems. The Engineer leaves the facility and the people go to into listen to the system and they invariably extoll its virtues, even though the entire system was behind a sheer curtain in a room with dim lighting so the listeners did not know what they were listening to. The results were rather surprising in that the $100,000 USD system, played at the same volume, could not be differentiated from a decent but modestly priced system, so we already have disguised the testing and we already know the answer.
So, ABX testing is similar to an annual visit to the optometrist: "Which is better: 1 or 2?" :) It then gets narrowed down using subsequent A-B compares.
I don't agree with you here. I think vision is a much more easily objectifiable thing than sound quality. What your eyes see is either distorted or not, blurry or not, veiled or not, missing colors or not, whereas audio pertains to many more variables, including some that are similar to the ones used in optometry. And, as you kind of mention in your comment, an eye exam resembles more an A/B test than an ABX test.
David Perkins Optometrists are doing so. They match several pairs of glasses, down to a precision of 0,1 dpt. The test is repeated, the result measurable and reliable. So what?
Or hearing test
One thing I have to mention is that people judge sound differently when they see what they're hearing than when they don't. And I don't mean that they can or can't tell things apart when blind. I'm talking about things which pretty much no one doubts. (Like speakers.)
I love my Stellar GCD, so whatever you're doing is working great.
It is interesting and funny that Bose marketed its systems in ABX manner. They would put inactive large speakers and people thought that is where the sound is coming from, and were surprised to see that the sound was actually produced by the tiny speakers behind the audio grille. But when those same people knew it was the tiny paper cones they went crazy and started calling out Bose as a scam. LOL
Yes, you must be able to choose A or B. Just as you stated Paul, to be able to choose which is best, you have to be relaxed & comfortable. Best being, best for you, of course.
When we know we’re listening to different speakers we are afforded the freedom to proclaim differences and nobody can call us wrong. In abx we don’t know whether we are listening to the same speaker or the other, so if we proclaim differences and it’s the same speaker then the experimental method shows this. If we can’t tell whether we are listening to a or b in an abx then there is not a statistically significant difference between a and b. That’s why abx is a nice method. I think it’s more stressful because we can actually be held accountable for the opinions we make.
One thing I've noticed is that listening to the same piece of music on my Stax headphones can reveal things I never heard on my Sennheiser HD598 - but when I put those 598's on again I sometimes hear what I missed before. Just knowing the sound is there helps me hear it. I think this is one of the kinds of things that make A/B tests difficult.
I'm chiming in with BOTH camps here. So on the one hand, you can't tell under pressure or in just a few minutes all of the subtleties of a system or a piece of equipment. I'll buy that.
And then I say; so those differences are SO subtle that you can't even hear them from one minute to the next? Okay, then for a huge majority of people, even audiophiles, those differences probably aren't worth losing a lot of sleep over. I know, I know - audiophiles aren't normal people. And I can also see a designer like Paul M. being VERY concerned about those subtle changes - it's his job!
So while this video pretty much confirms the position of the ABX fans (like me) it also explains why ABX isn't necessarily the absolute end of the argument.
Thank you Paul - you struck a blow for tolerance today. (I imagine that's the sort of thing Paul McGowan likes.) I will appreciate the fact that there are things that aren't really proven by an ABX test the next time someone waxes poetic about "living with the spirit of a system" before judging it.
I was involved with an acoustics project as to why people preferred one thing vs another. This was done by a very prestigious University and an associated Audio Group. Think the Boston area. We used the ABX box test not as a go no go, better or worse but simply as part of a selection of tools to help start with which bits and parts had the most influence. If the Golden Ears could clearly guess right every time which X was then that was test 1. If the Golden Ears got it right 50% of the time on test 2 then test 1 was the item to be looked at first. Test 2 was not eliminated but you look at the big chunks first. ABX testing was never used as a pass/fail or better/worse tool.
Audio designers use ABA tests as a tool. They have nothing to gain by fooling themselves as that costs money and it leads to nowhere. I did over 10.000 even knowing what A and what B was, There were many instances I was utterly convinced that a new solution, call it B, was better that A and listening test clearly show otherwise. I agree fully with Paul.
I might add that the occasional blind ABA test by others are very worthwhile and even necessary in this process. This as a reference point in the process, sometimes you are on the wrong track, yes that happens too.
The fairest, truest and most reliable, tks Paul!
I can see why ABX is considered ideal but sometimes it just isn't practical as it requires more time and people to set up. I do most of my work alone, so it's just very basic non-blind AB and that is enough for bigger differences.
As you said Paul, when you're trained everything is easier... and AB is the right way. ABX is more that way ''to guess'', and AB way ''to examine''. Peace
How do people volume normalize? Is it done to a specific tone/frequency?
Paul is quite correct when he says the Raters must be trained.
Two key components of testing methodology is validity and reliability. If you don't have inter-Rater reliability then the test results could be valid but not be generalizable. No subjective test is perfect, just accurate to within a pre-defined parameter such as 5% or 10% within one standard deviation from the mean.
I find that with rapid (or even not so rapid) A B switching, you tend to focus on small changes in tonal balance above all else. Given the luxury of time, I find it much more informative to substitute the component under evaluation in a known system, and live with it for at least several days or a week. Then go back to the original component.
gotham61, I think you hit the nail on the head. I always find that it is much easier to just listen to a change over a period of time and then go back to the original to compare. And the significance of the tonal changes will also fade too. The trick seems to be spending time doing the check in your own system that you are comfortable with.
tru
You can do ABX testing the same way you do AB testing (play A, now let's got to X, now B, Ok, back to X...). Double blind AB testing is good, but it can still make you hear a difference where there is none. Let's say both A and B are the exact same system, and the AB switch is not connected to anything. It may still be possible to convince oneself to hear a difference.
I wish there was an equivalent of a magnifying glass for sound...
Here is an alternate take. As a person of science and mathematics, ABX is scientifically solid and legitimate. Science admits no personal biases and rightly so for it is truth we are after. Music and audio equipment, on the other hand, are extremely personal and there is no question of truth - we all are equally right in our choices of likes or dislikes in music - and there is nothing scientific in it - and none is needed. Hence, we all should simply go by what we like in equipment subject to the constraints of our finances, availability of equipment, space in which it will be listened to - and tweak it more to our liking.
Rajendra is the smartest guy in the room ; )
Why would you be on guard? Whats at stake? Worried lets about the result? Other than pride, what is at stake? If not worried then the result shows 100% that something is an audible difference or not.
Hi Paul: Somewhat like an eye test; "better or worse - better or worse". I don't think I have had an ophthalmologist/optometrist do an ABX test. As you say you have to know what you're looking for (listening for). In my case I have extreme astigmatism and I look at the test image as a whole, but specifically look at the sharpness of horizontal edges and vertical edges and go back-and-forth: ABABAB . . . concentrating on a single parameter until I have the very best. If the test was BBABAABBBABAABABX I think I might throw-up and certainly would be confused. This seem to work well as without my glasses my site is crap and with well corrected lenses it is 20/17. I would think an BBABAABBBABAABABX test for very subtle differences in sound, taste and sight would become white noise (if I can use that term in this contexts).
Cheers,
Mark
*************************************
Lots of black and white answers to a problem whose answer is probably a shade of grey. On paper Abx testing is better and necessary to avoid placebo effect but A to B tests only show up differences which are immediately obvious to the particular listener, other effects such as listening fatigue are more insidious and can creep up over a period of time. I have been involved in ABX testing with amplifiers and I struggled to relax enough to assess the differences reliably wheras at home in the familiarity of my own system with my own test tracks the differences are very obvious. Unfortunately most humans are not listening machines so we can easily get it wrong for a while particularly under unfamiliar circumstances. Abx tests are also very difficult to perform thoroughly as changes need to be seemless, not take too long, be volume matched, not resort to using switching boxes or other components that will degrade the sound etc etc.
100% agree. I spent a lot of time testing gears and you have to have actual experience in testing to say what you said in this video. Annoys me slightly when i read comments from people about ABX test as the ONLY correct way especially if they have never spent time testing anything.
je t'ai pas oubié Thomas , Quadraspire dorment au gaz avec mon meuble que j'ai commandé , genre ça fait 2 mois que j'attends. C'est sur qu'apres avoir testé un rig a 300k y'a pu grand chose qui va te donner une larme a l'oeil LOL !!!! , mais on va avoir du plaisir quand meme !!!
Merci!
Let those people speak. Think of it as counselling for them. Keep up the good work with your channel!
Some people use the excuse of ABX being the only way as a means to justify that they don't spend much money on audio gear. It's an excuse. Doing a quick ABX test doesn't give the listener enough time to get a good enough sense of what A sounds like vs B. especially since you have to shut down a system and replace A with B. Imagine changing speaker cables. The only way to do it properly, is you have to detach from the speakers and the amp and you have to shut down the amp. Some amps take a while to get back to the same level before they were shut down. Some amps HATE being shut down and then turned back on shortly after. So, it can take a long time to change out product A to product B and then back and forth, etc. etc.
The best way is to live with a product until you get used to it. it can take a while because some products have a break in period, some products simply take time to get used to what they do well and what they don't, etc.
I first do a ear fatigue test because if I'm testing a new product and I get ear fatigue within a short period of time, then that product is a failure and I take it out immediately and try something else. Ear Fatigue to me is unacceptable for a "high end" system, even if it's a budget high end system.
Again, ABX testing is typically setup for listeners to fail when it comes to listening to Audio gear because it's just the way it is. Why would you setup a comparison to get people to fail? What are you trying to prove? That spending lots of money on gear is a waste of money because you can't afford it?
Here's something to think about.
I can listen to a Bose stereo of my favorite recordings and enjoy it for about a few minutes, but I couldn't enjoy it after hours and hours of listening to it over the course of several weeks, months or years. Short term is not the goal of buying a piece of audio gear. If that's how you judge what to buy, then you will never be happy with what you buy unless the only thing that makes you happy is how little you spent THINKING that it's just as good as something else that's more expensive.. You have to like it enjoy to LIVE with it. If you don't get enjoyment for whatever reason and you don't spend time using it, then it's not suiting your needs. You can't tell with only a couple of minutes of listening in a ABX test.
Please, ABX testing is more of a joke than it is a method of dealing with the reality of figuring out if you like a certain piece of audio gear. Some piece of gear sound great or bad with other products and you don't know unless you bring the product into your room and give yourself a chance to either like it or not.
I don't know why you are so obsessed with trying to conjure up some "simple" test to disprove high end audio, when it's just not the best way to judge high end audio gear. Maybe you shouldn't buy high end audio and simply walk away and not get into any discussions with those that do. You'll never understand if you have this notion that the only valid method of testing is a ABX test.
Some rely on both subjective and objective measurements. I know some mfg. do both and rely on both. If they can't measure enough of a difference in a product design, then the design is rejected. Some of these companies do measurement tests and then they have a group of people that do subjective listening tests and then they compare the evaluations along with the measurements to see if there is a direct correlation.
Excellent.
You are so correct
Right on Paul! Excellent explanation. The way I see it put another way is ABX is an intellectual exercise vs an emotional engagement of our senses, which is, after all, what I'm after.
Good vid Paul, I did an A B test using the same model of amplifier with two different types of op amp when I was in college, 3 people tried it out but couldn't hear any difference, I came to the conclusion the power amp was not revealing enough to show the difference between the cheap an cheerful op amp and the high spec chip
Lol The "I need to be relaxed!" excuse. Hilarious.
Blind tests are the only valid test in fact - and you only need to see how the Highend World runs from it to know for sure it is. What its key is to maintain the same volume though, this is the most important part that many people forget. You have to compare the systems are at absolute equal volume.
First off, everyone has different levels of hearing and listening skills, so that's the first part. Secondly, in short term ABX tests, if the listener has only a short period of time to compare, then they are only going to be able to tell differences that are 2dB or more difference. anything that's more subtle isn't necessarily going to be detected in a short term listening test.
Now, other factors that will impact a listening test are as follows, but not necessarily limited to.
1. Recordings one is using. A recording may not be able to give the listener adequate information to hear any differences in what you are comparing. Some recordings have lots of distortion, or distorted instruments, audio compression, limiting, etc. etc. and some recordings are just awful to use when comparing two different pieces of audio gear.
2. Volume level. It's best to not have the volume level exceed a certain level, typically above 85dB as that will cause short term hearing damage in a fairly short period of time.
3. Bad room acoustics. Your room could be a limiting factor on whether or not you hear any differences, especially at higher listening levels.
4. Unfamiliarity with any aspect of the room, system, music, etc. The less familiar you are, the more likely you won't be able to to focus on what differences there are.
5. Added switches, cabling in comparing two different products. NEVER add anything to the system that wouldn't be there under normal circumstances, otherwise any added components may neutralize any difference between two pieces of gear you are comparing.
Here's another thing to think about. If you are going to take a test for a course, if you are unprepared, then you are probably going to end up guessing and not being accurate when taking a test on material you haven't had any time to study. By the same token, If you have plenty of time to study, then you'll more likely get a better score and not have to resort to guessing.
If someone wants you to compare two pieces of audio gear in a ABX test, the fair thing to do is give the listener enough time with the room, system, content so they feel comfortable in determining if there is any difference between two pieces of gear in a specific system. In some cases, there isn't and some cases there is. But it's dependent on whether the listener has enough time to determine.
The other method is simply conducting harmonic structure measurements within a listening room. Having a MLSSA system can help facilitate determining if there is a difference between product A and product B in a specific system/room. Why guess, just measure, it will at least tell you how they are different and give the listener the ability to listen to those differences that are, in fact, measured.
Blind fold tests aren't the definitive method since most people don't listen to music blind folded and what you are doing is putting someone in a state of unfamiliarity. using it as an excuse to dismiss high end audio gear is being ignorant and immature.
I have my own tests I conduct with certain pieces of gear that I feel produce a good end result. Unfortunately, it takes about a day minimum to determine if a piece of gear will pass one of the first tests I perform when I evaluate a piece of gear. I have successfully weeded out products using a certain test method and it has nothing to do with ABX testing.
Oneness100 Of course, one should have all the advantages, familiar place, music, you can train with your preferred pieces, take all the time in the world, do it yourself at home, whatever. And there's no need to do it blindfolded. We live in 2018, there are other methods. It doenst need to be ABX, a sinple AB would work also. The only key component is external measure of absolute volume. When people detect differences in amateur AB tesg these usually are simple volume changes. At the same exact volume detecting differences is very difficult, even for audiophiles. These are the common results, wheter we like them or not.
I will keep fascinated with high end audio, i just love the designs, the engineering, etc but i know at equal absolute volume differences are very slim and mlst people cant tell the difference between a 10.000 € dac and a 500 € one.
I don't care what most people can tell or not tell the difference, I only care about what I can hear or can't hear.. Most people can't tell if someone is playing an instrument out of tune unless it's at least 2 cents out of tune, but others can tell a smaller difference since we have more experience/training on pitch.
Either way, my biggest concerns is whether a component is going to cause ear fatigue when listening to music at 85dB or lower. I have found cables to be the biggest concern for me.. I have found some cables that consistently don't have ear fatigue causing distortion, while most cables (at least the ones I have tried, do). That's what I have found to be a reason for buying specific cables that have consistently not had distortions that cause ear fatigue. It's just something I've had to live with and I have a heightened sensitivity to certain distortions.
Again, people have different things that we can and cannot live with. Ear fatigue is my biggest issue. I like to enjoy listening to an entire album all the way through without having to turn off.
Oneness100
Sure, but even pros with perfect pitch have trouble telling apart mp3 192 from mp3 320.
czcams.com/video/YgEjI5PZa78/video.html
If you're able to tell apart cables you should do the Randi test and win a million bucks. Easypeachy.
Roberto Calvo Perfect pitch has nothing to do with mp3.
@ 0:26 "Thanks man!" LOL! :-)
👏👏👏👏👏
My ability to tell stuff apart increases when I am on my guard. If you swapped all of my music for MP3 128 during a listening session I probably would not notice even though I can tell it and FLAC apart in blind tests.
They are as valid/invalid as Patrick WANTS them to be. Do we ABX test ice-cream or amusement parks ? I completely *get* that its important when you're trying to compare two components/formats etc without expectation bias or whatever, but relying solely on the outcome for your purchase decisions is about as intelligent as relying on the vendor's marketing. I'm all for objective testing - what I'm not so keen on is the smugness that accompanies many of the results. No-one would be happier than I would be IF the Objective-2 was all the headphone amplifier I ever needed - it would have saved me a couple of thousand dollars on the purchase price of the Auralic Taurus - but sadly that just isn't the case regardless of measurements. Would I pay 10K for a headphone amp if I had the money ? No, but that has nothing to do with ABX/DBT testing. DSD/MQA are a much better test case IMO, but again you can spend an awful lot on gear to be able to categorically say 'yes, there IS a worthwhile difference between this and Redbook' or otherwise. Thanks for the vid.
I try listening to things to see how they sound.
Weird, I guess.
This kind of testing drives me crazy !! Honest ....!
Again the human ear comes into play here.... I ask this question :- why BOTHER doing A/B tests at all...? Audiophiles are their own worst enemy you know. They all tend to be far too subjective...they are !! You must bear in mind this.... you are all dealing with what a call "artificial sound" .. it's NOT the original .... that stops the moment I hit the record switch at my studios here. Testing speaker systems has too many variables ... and psychoacoustic tricks like ABX does nothing more than confuse or bamboozle is..! My dear old dad ( who was a music lover ) said to me once .." if it sounds 'right' then it IS 'right' ... Us human beings are all as different as there are types of loudspeaker ... subjective testing is ok for manufacturing but OBJECTIVE testing helps keep us from going insane lol !
I remember once walking into a friend's house and telling him that I thought he had a gas leak because I was getting whiffs the odorants used in natural gas. He could not smell it, nor could his spouse or visiting friends. Honestly, the smell would come and go, so I could not follow it to its source. Now, if we were to have used blind testing, the conclusion would be that there was no leak, so they should not have worried. Fortunately, they remembered the event and when his spouse smelled something about two weeks after I did, they had their furnace checked. It turns out that it was leaking a fair amount of gas. Blind testing has its virtues, but basing conclusions solely on subjective responses and negative results does not always lead to the right answer.
You may not be able to determine what’s best for everybody, but at least you can determine what sounds better to you, and save yourself a bunch of money while at it, and believe me… there’s a lot of snake oil out there.
AARRR ABX stir up a right bloody hornets nest those beggers.
There is extensive literature on double-blind testing in audio and most of it agrees that double-blind tests are meaningless in audio. Double-blind tests fail to reveal known differences, it's too bad most skeptics who insist on double-blind testing aren't aware of its inherent limitations.
but there's a lot of variables involved what can change or give you a false reading of something, you need a controlled environment when testing temperature wise and electrically wise. Just take the mains coming in will change on a hourly basis, and the person who's listening cognitive at the time and fatigue as well all play apart.
Just do it at the comfort of your home, take your time, pour a good cognac, do it with your favourite music, etc. Just make sure volume levels are identical, thats all you need. Please, come back with the results :)
Roberto Calvo but as I say there's variables involved temperature causes expansion will change the result the electrical mains will change its voltage, maybe running through a battery pack will sort this out. And use another battery pack to charge that battery pack connected to the mains. Humidity plays apart has it been raining we'll play apart. And don't forget is the system run in
Those external variables doesnt matter since they affect equally to both AB systems
While I'm willing to do an ABX test I know it usually doesn't work. the test box itself destroys the ability to hear differences. I can tell as soon as the test starts that nothing is sounding right.
ABX testing has good uses, but often when it comes to audiophile level performance differences it takes weeks (or at least several days) of listening to "experience" the differences in real use. And that is not going to work with any ABX setting.
Say you are a boy again, dating girls... You have two you find interesting. So, what you do? You have an egg timer.
Set it for "soft boil." Now... every time it rings? One girl leaves and the other immediately takes her place. That way? You can decide which one you really prefer? BS! AB testing, at best... only lets one hear there is a difference. The only true way to know what component/speaker you may prefer? Is to live with it for a while and listen to all kinds of music. Comparison testing. Not rushed AB testing... its a sales gimmick in my opinion.