Why "Audio Objectivists" are so wrong.

What is an Audio Objectivist.

If you've ever visited any audio forums you'll probably have noticed  a plague of "Audio Objectivists" (AO) . The percentage varies according to the forum, with some being close to 100% AO. (one famously does not allow any subjective comments at all unless they are backed up with double-blind tests!)

The basic AO philosophy goes something like this :

·    

• Measurements are a superior (or the only) way of determining how audio gear performs (compared to listening tests)
• Listening tests, if sighted, are worthless. Only proper 'double blind' tests are considered valid.
• All similarly measuring devices will sound the same. This can apply to amplifiers, DACs, CD players, power cords, interconnects, etc, etc. The only thing that is universally agreed that sounds different are loudspeakers.
• Much of what is sold in the audiophile world is useless junk and close to fraud.

This position is seemingly bolstered by the apparent numbers of scientists and engineers among the AO fraternity.  In a similar vein, the subjectivist camp often seems to be composed of untrained crackpots. I can say with some certainty that the maddest crackpot I ever knew was an extreme subjectivist audiophile. Also probably the second and third maddest as well..

So this post attempts to debunk the Objectivist position - I do this because I believe there are genuine mysteries in high-end audio. I'm not saying that audio gear implies some kind of new-physics, but any decent scientist knows that there are emergent effects that are not well understood in any field.

My journey

I was first exposed to a good audio system when I was a young teen, and I was an instant convert to the quest and have been an avid audiophile ever since ..  when I could afford to be..

But I have to admit that for many years I was an audio objectivist. It's natural I guess if you studied electronic engineering at university and spent your career as hardware and software engineer.

I thought it logical that such things as interconnects, power cords, vibration control devices and so on made no difference.  And I figured that electronics didn't have much influence either, especially digital gear.

My journey away from this position was gradual; firstly accepting that amplifiers could sound very different.  Now I am willing to believe just about anything - if the evidence supports it.

OK so let's take the points above and refute them..

Measurements

AOs love their measurements.  However, the measurements that they are able to perform are so rudimentary as to be virtually useless. In nearly all cases, the measurements turn out to be :

•     Total Harmonic Distortion normally using sine waves, often 1kHz
•     Signal/noise ratios of various types
•     "Jitter spectrums" using  JTest
•     Intermodulation Distortion using 19khz and 20khz sine waves
•     Frequency response using sine waves

This is not an exhaustive list, but you get the idea.

There are huge problems with this approach; sine waves are simply not music; they are the easiest thing for any gear to reproduce. So in one sense it's not surprising that all the gear they test sounds the same - playing sine waves.  This also matches with the real-world observation that many systems can reproduce simple music but fall down when presented with complex music (eg full on symphony orchestra).

Determining audio system performance using these kinds of tests is like trying to figure out the real-world performance of a race-car by measuring fuel flow, damper rates, dyno output and so on, rather than seeing how well it actually performs on a track.

AOs will claim that music is all composed of sine waves anyway. Now this is not actually true as it's a misapplication of Fourier theorem. But even if you accept that, it's like music is thousands of sine waves playing all at the same time, not one simple 1kHz wave.

The second thing that so-obviously-wrong about the AO approach to testing is down to vibration. AOs don't consider this important, so even the tests they normally do are done in silence. So the gear is not bathed in slightly-delayed vibrations of the signal they are reproducing. But for a sine wave this would likely not make a huge difference unless the gear was particularly prone to 1kHz resonance.

But for real music it makes a huge difference; as there are so many frequencies present, music will stimulate resonances across the whole audio band. And dealing with vibrations in various ways makes easily-audible improvements.

In a similar vein, AOs ignore or downplay other second-order effects like EMI (to give one example). 

The big problem with this kind of measurement regime is that the results do not always correlate well with what people find sounds good.

Listening Tests

When I first considered starting Mad Scientist Audio I have to admit that my listening skills were not that great. Although I'd been an audiophile for almost 40 years I simply had not practised analytical listening enough to be that good at it. My wife had more luck than I did at blind tests.

So I resolved to do something about this if I was serious about Mad Scientist. I started making it a habit to spend 2-3 hours every day doing analytical listening (i.e. listening tests) rather than just "playing music". This is a habit I've had for about 5 years now, 7 days a week,  365 days a year (well, almost..). This comes to over 4000 hours of listening tests.

As you are no doubt aware, focused concentration and repetition are the keys to learning or improving skills, and this worked great for me.  In that time I've collected a wide range of test tracks across all musical genres. Some are chosen because they sound great. Others because they don't ! (these usually are hard-to-reproduce tracks, often very complex). Each is like a tool in my audio toolbox - for example I have sets of tracks that tend to be over-sibilant on vocals, or tracks that have great bass, etc.

Before I actually started the company I needed to prove to myself that I was not imagining things, that the products I had made really did make a difference. So to this end I designed a series of blind tests, comparing real Black Discus devices against dummy ones that were same size and weight.  The tests were blinded by the devices being wrapped  in small envelopes by an assistant.

The point being that I needed the tests to be properly blind, so that I was completely unaware of the device under test. I was not going to start a company that sold useless items, despite what you AO folks think!

The results were conclusive - I could pick the real and fake Black Discus in almost 100% of trials.

Audio Objectivists like to point to numerous double-blind tests that have been performed and documented  by various audio societies, magazines, websites, etc.  Nearly always they show the desired result - that the amplifier/cable/snake oil under test cannot be determined correctly by the participants (although you will often find a few folks that could apparently tell the difference, but that is written off as statistical chance, and the results are usually given as statistics).

Also nearly always : the tests are performed using an unfamiliar system, in an unfamiliar room with unfamiliar music. Is it any surprise that they get the results they do. Bear in mind that it only takes one person who can reliably tell the difference between, say, various interconnects, to disprove the theory that "interconnects sound the same" (This is akin to the black swan argument;  it's also a case of experimental data trumping theory).

I'm not sure that I would be able to be successful in an unfamiliar setting.  Let me give an example to explain why..

Example : Wire Directionality

This is one of those hot-button topics for the AO folks.  

Suggest to them that wire can and does sound different in different directions and it's likely to set them off: " Wire can't be different as the signal is AC (you dummy) so half the time it's flowing one way, half the other" and so on.

I did not believe in wire directionality for many years - it was simply obvious to me that it could not make any difference.  But then I sat down and actually did some experiments.

For analyzing small differences like this, initially I do sighted listening; two sets of cables, two different directions.  The idea here is to figure out if I can hear a difference, and if I can, to isolate the sonic signature of the differences.  Also to find some test tracks that highlight the differences so as to make it easy to pick which is which.

Once I've done that part I can do blind tests. 

But not before. I simply wouldn't know what to listen for, and trying to figure that out in a small amount of time would not be easy, especially with a test track that did not highlight the differences very well.

A while back I performed a blind test of wire direction as a demonstration to a cynical friend. 

In the normal production of our interconnects, the wire (and carbon fiber) direction is tracked. In this case the 'destination' end was marked inside the RCA plug, so you had to unscrew the barrel to find which end was which.

So I had 10 of these cables - they were digital SPDIF cables - and I proceeded to test each cable each way and "guess" which direction was preferred, all while my friend looked on. 

(Yes, cynics - wire direction is important for digital cables as well. Quelle horreur!)

By this time I was pretty good at picking the correct direction. The "trick" was knowing the sonic signature of each direction. 

I used a test track that had two things : a solid bass-line and a centrally located singer. When the direction was correct, the bass-line had more drive and the singer's voice was more tightly focused.

During the test, I tried the first two or three cables both ways, and picked the direction after around 10 seconds of music. For the rest of the cables I could easily pick "right way" or "wrong way" on first listen, often after just 2-3 seconds. 

Needless to say I got 10 out of 10 correct. That was never in doubt.

But trying this test on a strange system with previously unheard music could likely be beyond my skills.

While on the subject of wire direction : I have to admit to being baffled by this. (One of many things that baffles me about audio).

In one sense the AOs are right - this makes no sense. But you can't deny experimental results. Where would we be if people had declared that photons simply cannot pass through both slits..(Actually quantum mechanics is a case study in things that fail common sense.)

So the 'scientific' thing to do is to look for reasons why, rather than just deny the effect exists. I believe anyone who is serious can hear wire direction differences with some practice.

I don't have a good theory of why the directional effect happens. My best guess is that it's to do with the microscopic mechanical properties of the wire interacting with the current flowing and/or the various vibrations that the wire is bathed in. I know this doesn't hold a lot of water, but the effect is beyond doubt in my mind because of the numerous blind tests I've done.

I wish wire was not directional. It a nuisance having to test each new batch of wire I receive to determine the correct direction.

Blind vs Double-Blind vs Sighted Listening Tests

One favorite tactic of the AOs is to decry any listening tests that are not strictly double-blind, which are the gold standard for medical/drug tests. It's done to prevent any conceivable source of bias in medical tests, which are frankly much more serious than listening tests, which are hardly life-or-death situations. But crucially, it's fairly easy to do in drug trials as it's easy to make placebos. 

Double-blind means that nobody involved in running the test knows the identity of the device-under-test. This can be done sometimes, but it's really difficult to do for a lot of things.  

For example, you could get an assistant to change between two types of interconnect. But he would see them and would know which was which. There are ways around this but things get cumbersome very fast. How does this work with amplifiers?

The reality is that for audio tests, the majority of the time it's almost impossible to conduct true double-blind tests outside of a research environment, and then you get all the unfamiliar-system problems.

However it's often possible to conduct single blind tests. I'd encourage people to try this as it's an instructive experience. It really is much harder to pick things blind; some is no doubt confirmation bias, but some is down to pressure to perform, etc.

The reason that AOs bang on about double-blind testing is that it gives them an excuse to reject mere single-blind tests, even though double blind would be unfeasible. I wager most of them have never performed a true double blind test in their lives.

In reality, most listening tests are performed sighted - there simply isn't enough time to set up blind tests for everything. The AOs will tell you that confirmation bias completely invalidates sighted listening. Perhaps it does for naive or untrained listeners, but for people who are well practiced, and can repeat tests multiple times, I say that they have validity.

If sighted listening was all about confirmation bias,  you'd never be surprised during sighted listening. 

And I often am surprised enough to exclaim "you've gotta be kidding me" and similar less family friendly retorts.

Confirmation bias is very real though. For instance if you believe that interconnects all sound the same then if you test them you will (usually) fail to hear any difference. But for some reason, AOs forget about their own confirmation biases.

Similar Measurements = Similar Sound

This just doesn't hold water in my opinion. A well-known blogger routinely tests various DACs and unsurprisingly they all measure pretty much the same. The differences are written off as impossible to hear. So his implication is that pretty much all DACs sound the same.

I have to wonder about such folk. The first thought is "Are they deaf or something?"

Really, it's not so hard to tell the differences between DACs. Not like picking cable direction. 

Often there are gross differences in tonality, presentation and dynamics that means you don't need to resort to figuring out micro-differences..

The same sort of thing applies to amplifiers. In fact it's fairly easy to produce an amp that measures impeccably on sine waves, in fact it may measure much better than say a tube amplifier.  But the tube amplifier might be preferred by all listeners.

What's going on here? 

The AOs will tell you that these stupid audiophiles actually like the sound of distortion, that they think it improves the sound quality!  

Right; so the extra distortion is responsible for ambient hall sounds now being audible, the two singers that were previously  blurred together now being separated,  and all the other effects that we can hear. 

Bullshit!

It's much better  to believe that the measurements were simply not measuring the right thing. Distortion and/or noise will only ever reduce the information content (see : 2nd law)

(The latest version of this I heard is that these silly audiophiles prefer the sound of upsampled DSD to normal PCM because of the differences in ultrasonic noise - DSD has more - and that this extra noise works just like tube-amp distortion. They will also tell you in the next breath that you can't hear ultrasonic noise anyway!)

Audiophile Products = Snake Oil

The logical conclusion of the AO's arguments is that many audiophile products are fraudulent (that would include just about everything I make and sell). Of course I completely reject this.

If I come across such a person in real like (as opposed to online) I normally do the following demonstration for them:

1.       play some music for them

2.       replace the power cords and interconnects with standard items

3.       remove the blackpod footers from everywhere

4.       play the same music again and defy them to say it sounds the same..

Normally it's not necessary to replace the "tweak" items again. Doing steps 2,3 and 4 so ruins the sound that even non-audiophiles are easily able to hear. 

Another thing that is strange - no seller of actual snake oil would offer trial or money-back guarantees. But we offer that, and so do many other audiophile companies.

In the end I think the AO phenomena is a lot about ego, the need to be right, and most of all, a patronizing way to feel superior to all the stupid audiophiles like us.