You Can't Blind Test Cars

[firesho]

An excellent article.

I have a preference NOT to buy asian products, and most euro products. That is my bias. But even I have to admit, each and every product out there brings something unique to the table.

I have always been a Chevy/Pontiac fan, even when they were at their worst. Yet, I have also owned other brands... Studebaker, Volvo, the dreaded FORD. When I go to look for a vehicle, first i have to ask myself, which ones meet my needs at this time. Perhaps its a mini van, or in my present case, an SUV, that I need.

Then, i look at overall performance as it relates to why i am purchasing this vehicle. Go fast?MPG? load? people room? cargo room? By now I have pretty much whittled the contenders down to a few. I have satisfyied my left brain now. Next I go see which one is the more enjoyable ride. This satisfies my right brain. Strange as it may seem, there are always both US and 'other' products in the final mix.

Ultimately, i end up with what I hope will be a decent vehicle, and all my biases have been satisfied.

Automobiles are an expense, and consumable items. When I leave one behind, there isnt a drop of juice left in it.



Thank you for GMI's insightful and informative articles such as this one.

[DanCBJMS1988]

That's the best thing I've read in ages. Keep it up!

[SobeSVT]

Originally Posted by banzai79

I don't really have a problem with car mags having biases towards certain makes and models, I do too, so I can't blame them.

I may be wrong but I don't think you are an opinion-forming-mass-media-outlet. So, yes I blame them. One thing is to allow you biass to permeate to your media, another is to consistently trash one particular brand. This fact is consistently whined about in GMI, how can it, all of a sudden, be . . ."fine".

[jwrebholz]

I'd imagine a bunch of people test driving cars blindfolded couldn't do any more damage than the average morning commuter...

As long as you realize there IS bias present, you can recognize it and tune it out when necessary.

I'll admit, I have my own biases. Everybody does, it's part of our nature. However I try to not let those biases cloud my judgment when I'm looking at a car.

[hunchman]

Originally Posted by GotAWD

i gotta go shave and im only at the first half of your editorial. thank god you dont write up more stuff here dude. ill be back to finish the rest. good work though grank.

LOL.......do you think Grank took a breath or did the whole thing in one shot

[mkaresh]

Great editorial. I'll post my take on this topic, with a link to this thread, in my blog today or tomorrow.

But I don't see myself as downplaying the "right brain." I just write about those topics elsewhere, especially in my Epinions reviews.

In general, I feel that right brain stuff comes down to personal taste. I don't think it's possible to objectively conclude which car drives the best, is the most fun to drive, etc. I've written about the value of comparison tests as a result here:

http://www.truedelta.com/pieces/comparison_tests.php

I see TrueDelta as showing how much a car's extra right brain goodness will cost you in terms of price, reliability, and other quantitative factors. I see people driving cars, decided which they prefer, then trading these qualitative preferences off against the quantitative factors.

The problem with CR is that the way they report their results makes these trade-offs imprecise or even impossible, because the absolute size of the differences between cars are unclear.

[regfootball]

CR has become largely irrelevant, IMHO, to anyone whose decision making process includes reason, emotion, or taking responsibility for your choice. CR assumes you only buy a car based upon data, which is insane. Like choosing a spouse, You might have specifications for what measurements you might want her to have, you might make a value judgment on whether she is 'reliable' . But people who overemphasize data in life are hollow. I think this is our problem with marketing as a profession. Too many people who know nothing about how the product they are working for gets bought and sold and who their customers are. The only shred of credibility they have to fall back on to claim they know what they are doing is 'marketing data'. I could go on, and on.........

People need to look at cars, and test drive them. Use data as ONE factor in carbuying. Not the only factor. Most folks are too lazy or indifferent and simply default to letting CR and all the other 'data miners' tell them what to buy. SAd really. Would you choose a spouse that way?

The left ultimately needs to recognize that vehicles are more than just than value statements, they are emotional statements. Of pure factual sources, TrueDelta.com has been unique in offering an editorial on the intangible value of a great vehicle. When vehicles can’t be blind-tested, the left needs to recognize this in evaluating itself, and certainly its vehicles.

The right ultimately needs to recognize that not all vehicles are designed for them, and that they serve more needs and tastes than their own. Favorite qualities can be found in more vehicles than just the favorites.

love it. said well.

[jlmartin99]

Greetings Ghrankenstein,

A well written opinion and I happen to believe the right and left opinions are almost a perfect set of bias opinions that used properly provides valuable information in regards to cars. I found all of the publications valuable when I performed my review and evaluation of cars for my purchases. Once I included my requirements and I would not classify them as bias, but simply what I prefer in my car products. These different points of view allows insight on the product otherwise might be loss and many varied opinions are powerful, such as safety test results are never discussed in Car and Driver or any of the traditional enthusiast magazines, but that is not a problem, CR, government safety and Insurance Institute is where I can source opinions. I use all of the information and add it to my personal evaluation, with weighting factors included and I rank the cars based on my requirements. I found the experience very rewarding and a clear eye opener into the various car products.

Again, a well written opinion.

JLM

[5.3Avalanche]

Very good article. You should work for one of the big car mags.

[ciciusss]

One of the better editorial pieces that has appeared on GMI. Nice job. Since subjective factors were a part of the editorial and Consumer Reports name came up, here is what one of their test engineers wrote about when CR evaluates subjective factors:

"A note about subjective measures: just because something is evaluated subjectively doesn't make it open to the whims and caprices of the tester. Instead, you can evaluate something subjectively within a set framework of scoring that is identical for every car, looking specifically for certain attributes within an evaluation. In other words, the person becomes a measurement device.

Take seat comfort: the 8 people who score it are looking for quality of support, shaping, lateral grip, headrest comfort, range of adjustments, and numerous other factors. To that end, most subjective scoring (and even some of the objective tests where we get a number) are usually performed by multiple people, so the scoring averages out the outlyers."

So, CR would argue that as far as scoring subjective factors is concerned, they have minimized the opinion of testers to where they can score subjective categories nearly as objectively as the objective categories. They are working from a certain frame of what they are looking for where various attributes of that category can be scored objectively.

Of course, one could argue that the criteria that CR has set up for these various attributes to be scored in the subjective categories are based on their opinion of what is best and not what others believe is best. At best, the results are a compromise between those who scored the subjective category.

My opinion is that many times these subjective factors are overstated. For instance, CR complained about the Saturn Vue's front seat comfort. However, go to the various forums of those who own the Vue, such as saturnfans, and you will not find complaints about front seat comfort. There have been complaints about other issues, vti was a hot issue awhile back, but not front seat comfort.

The vast majority of new vehicles are pretty good. That was a point that Edmunds' Karl Brauer made in a blog several months ago. He wrote that over the past 5-6 years he could only say that there were only 5 cars that he would consider bad. It is getting to the point that reviewers are arguing about how hard the plastic is in a car (most people don't care, for them it's about whether the cabin is attractive or not). Car and Driver wrote a paragraphs on how the Aura's trunk only opened a "few millimeters, forcing you to wedge your formely clean fingers between trunk and bumper to lift it." Truly petty and honestly, how many people would this even be an issue with?

[mkaresh]

Posted my site's perspective in a blog entry:

http://www.truedelta.com/blog/?p=17

Everything about a car cannot be reduced to objective scores. But for those that can...(resisting urge to riff on the MasterCard tagline)

[FStephenMasek]

Your article was good to read, but most of the "Right" vehicles you list are front wheel drive, and front wheel drive is wrong wheel drive. I still think that much of the popularity of SUVs came, and still comes, from the fact that most are rear wheel drive midddle or full-sized vehicles, most with V8 engines optional or standard.

[asim]

Thanks a lot Ghrank! You're 100% right. I just wished more people realized these things, people in the industry, fans, and the public. In matter of fact, I am going to try to get this posted at autoblog.

[Ghrankenstein]

Thanks for the kind words. I'm glad that you guys realize that I wanted to fault no-one, and emphasize that opinions aren't right or wrong (though some are poorly thought-out or uneducated), and I'm adamantly NOT faulting any publication for its perspectives. They're just part of the landscape, just like GMI members and their opinions.

I'm trying to editorialize more, since I usually just write car reviews, and I'm still completely blind about how they'll be interpreted and received. The DTS editorial was pretty darn jarring. Tangible vs. intangible aspects of vehicles, though, is interesting and worth writing about again.

And yeah, I did write it in one shot, over the course of a hockey game. As usual, not letting it get any longer was the biggest challenge.

Mkaresh: thanks for the vine, and a great read yourself.

[mkaresh]

I personally didn't read the editorial any other way.

I wouldn't lose any sleep over the DTS piece. If you want to post material worth reading, you've got to expect some criticism. I'd be more worried if you don't receive any criticism.