Consumerists: The New Car-Salespeople

[Ghrankenstein]

Consumerists: The New Car-Salespeople


An Editorial By Ghrankenstein


Introduction:

This is one of those articles that I’ve written many times before, but haven’t been able to finish. It took a rant in a recent GMI thread to get me rolling, and to really get what I’ve been thinking to work itself out in a single, continuous coil.

Understand that, in my own interests, I don’t care who finishes where in any of these “quality” surveys beyond what I can use and what I need to overcome. That’s because they’re all crap when I read them, and when I actually try and decipher what they’re trying to say, it's “Buy ME,” or "Visit MY website for what you want to hear," or "Use MY results in your advertising."

I’ll finish introducing this by saying that I’m going especially hard on Mike Karesh, and TrueDelta. There’s a reason, and it’s a compliment.

For those of you who are looking for a breezy and irreverent read full of abstract references, I can only offer you this:

We live in a society in which the individual ego and the group dynamic are in constant struggle.


The Article

When the ever intelligent subscriber to the Book of Easy Answers and Quick Fixes sees nearly identical import-branded products as "better" and pays more for used Corollas than their equivalent Prizms, for Proteges over their equivalent Escorts, and Mazda6's over their roomier Milan and Fusion conspecifics, there really aren't any excuses that the resale market can offer over the surveys. Very little science is actually involved, and every kind of marketing toward buying insecurities, preconception, and their resultant web traffic is involved. Until it can prove itself otherwise, the Automotive Research market is nothing more than a two-pronged Book of Easy Answers and self-designed OEM marketing tool.

I can debate the IQS, Long-Term Dependability, unscheduled visits for service, Consumer Reports, whatever else all day long. We're talking about customers, and I ignored stupid things in my Malibu and Camaro that didn't impact their quality or reliability because I really liked them, and other customers do too. My current beloved Cavalier isn't "quality," but it's quality.

I can also comment on the reasons that I've gotten bad CSI's (i.e. that go to JDPower). Customer got a scratch on the vehicle after a week, and, while we tried to fix it anyway, it didn't look right. Customer got 8 mpg while towing a 40 foot trailer in a Yukon Denali, when I told them they could get less. The notorious HomeLink and now LEAR universal transmitters, which I have made house-calls to set up because many well-to-do can't read. Poorly PDI'd dealer transfers and out-of-state purchases (I have swapped bumpers with vehicles for out-of-state buyers who didn't want Missouri front license-plate brackets, and -have- gotten perfect CSI's). Finally, I've had/seen megacomplainers who just wanted to get out of their vehicles and have made multiple complaints when nothing was wrong with their vehicles (i.e. Lacrosse steering "too heavy"). Those things all go into the surveys. I know firsthand. It's rare, but it costs me a $75 fine each time it happens, and that means that I absolutely do not forget.

In their current forms, which have benefitted me in the past by affirming that GM quality is on par with everyone else's, quality surveys are presented in simple, stupid averages that from what I can tell are as influenced by outliers as by "average" customers. There are far more factors dictating these "outcomes" than just the vehicle, and many documented problems have not affected the scores of some (i.e. Toyota), while silly and superficial "problems" have affected the scores of others (Mini, Pontiac). The fact that these surveys are a grab-bag from year to year makes them nothing but advertising bait and lies otherwise.

If these surveys of "problems" are to truly be worthwhile, we need to know what the real problems are. If Land Rover is at the bottom of any quality list, we need to know why (electronic suspension? fuel economy? cupholders not right?). If a "top-quality" vehicle should have, say, a problem with broken camshafts, it shouldn't score better than a competitor with problems associated with fixing parking-lot sheetmetal dents, especially when the latter has problems with unreported fuel economy.

If OEM's are struggling to understand the difference in what buyers want, and what they say they want, they should be struggling just as hard to understand the difference in what they don't like and what they say they don't like. We salespeople know the difference, at least in hindsight. The time-tested phrase: "Buyers are Liars." I've dealt with enough disgruntled malcontents, and customers whose daughters I've dated before getting surveys, to know that what goes into those surveys isn't always the real story.

It's still the fairest comparison, since the factors are supposedly equal on all grounds. If Ford can make it through this gauntlet with gusto, then all the glory to Ford. I'd rather get hit by a Ford than drive a Toyota. I'd rather get hit by a dolled up Fusion than a dolled up Camry. I'm a GM guy, but when it's a question of Ford or Toyota, any Toyota I might consider would have to go through any Ford first, and any DCX second.

Good for Ford.

What JDPower and Consumer Reports do NOT report is any kind of standard deviation or confidence interval for initial or long term quality. What is the industry confidence interval for initial quality, or long-term quality? Who is notably above that confidence interval, and who is notably below? Who is statistically average, and who is really better and really worse?

Just as important: Which manufacturer's buyer responses are not normally distributed (i.e not along a "bell curve," indicating bias)? Judging normality in itself is a debate in this segment. Do some/all/no response sets qualify as normal? Does Consumer Reports test in the same Kruskall-Wallis style that it reports? And if so, how does the entire industry compare? As if from the OEM’s themselves, it’s a product yet again. It’s “The Truth About Cars ™.” It drives me mad.

Wait, no it doesn't. I already was mad. Freaking crazy. For thinking that these so-called reports had any kind of actual meaning beyond whims. Maybe they do, maybe they don’t, but I have no way of knowing. I'm insane, but I'm also smart, and I don't have any idea how JDP, CR, or anyone else actually collects or analyzes their information.

TrueDelta publishes something along those grounds, I think. I'm not sure. I have a background in science, sampling, statistics, and interpreting statistics, and I can't make heads or tails of TrueDelta's information in the time of an average consumer's website visitation. Not in the same way that a diehard Consumer Reports "enthusiast" can goose-step in a matter of seconds, or that an OEM can capitalize on a JDP whatever survey or Most Recognizeable award for something or other.

As an index of reliability, TrueDelta uses successful service visits. That sort of works with me, since it theoretically neutralizes both the habitual complainers and service department issues that OEM's must correct. Since it’s the customer reporting, it’s still a “problem,” whether or not it’s actually a problem. If TrueDelta, CR, or JDP, has a way of distinguishing, “Rides stiffer than my old Park Avenue,” “Radio gets really loud when I get on the highway,” and “Battery went dead (because I left my interior lights on or I didn’t park my Corvette in reverse),” from some kind of actual problem, I have yet to know about it. Good luck with that. Or better yet, post the most common problems/ "problems," with the model report.

TrueDelta posts confidence intervals, which is also good. Darn good, since I have some idea of what I'm looking at. The problem is that I have to dig darn deep in order to find them, beyond the "best-worst" scale somewhere in the middle page that is dominated by Hondas. That's no better than the other surveys, when the same data could be displayed just as easily as a graphic confidence interval superimposed to that of the industry or the segment. I'm not convinced in a world of supposed consumerism that looks to be convinced of what it already believes, and others in its perceived correctness, in seconds. Maybe I just woke up on the wrong side of the rock.

On first glace, these data look run-of-the-mill. What disappoints me is that when I take a while to look at them, they're not, and even then I don't trust them. The questions that Karesh raises in the research of TrueDelta are noble, but, as much as I admire the effort, they are not adequately addressed, or presented in a digestible graphic. Mike has commented on Consumer Reports’s inadequate and deceptive graphic representation of its data, but hasn’t offered any real alternative for the casual browser and its short attention span.

TrueDelta’s results suffer from small sample sizes. Karesh asterisks products with small sample sizes as not usable for research, which I’m guessing is somewhere around/under 30, because that’s what statisticians say is the boundary for a statistically significant sample. That’s partially good, since the in-depth data show year-to-year anomalies among such beloved vehicles as the Honda Odyssey. This is our problem, and TrueDelta’s problem; more owners need to respond to the survey and register their vehicles. Even here, I’m still not convinced.

With sample sizes, and sampling in general, like this, I’d like to see just how likely the responses are to being unbiased, i.e. “normally” distributed. In statistical terms, this test against normality can be stated as a “p” (probability of statistical significance) value, and the confidence intervals that TrueDelta publishes are subject to this confirmation of validity, or at least notation of different analytical methods. I’m not remotely knowledgeable of skewed-distribution statistics, but p-values indicating the possibility of skew would lend a lot of credibility to the statistics Karesh is publishing. Similarly, potentially skewed samples should be displayed as such, and asterisked in extreme examples (i.e. p<0.10 for standard normality tests), if they exist. That’s assuming I knew what distribution TrueDelta was using (t? If I knew the sampling method?). The problem is that no analytical product will ever be willing to admit the limit of its own validity, and if TrueDelta can/will prove me wrong, then a freaking medal to Mike.

If there’s a description of statistical methodology on TrueDelta, I haven’t found/seen/read it. Even if the general public doesn’t touch it, a description and explanation that is open to evaluation and criticism would be beneficial. In short, I want to see the likelihood that the data I’m reading are biased, and I want to know that what I’m reading is the real deal if it actually is. Variability in the TrueDelta surveys suggests that it isn’t. What I respect is the fact that TrueDelta is willing to report that at all.

I’m ready to state my conclusion.

The automotive research survey reports are the modern equivalent of the car salesman: Incomplete, deceptive, and bent on generating profits/sales/web traffic/advertising revenue/subscriptions for themselves over providing the information they claim to give.

TrueDelta has been, and hopefully will continue to be, revolutionary in providing commentary and criticism of a well-established consumer-advocacy industry, just as print and publications criticized and vilified the frontline sales industry before it. We’re still dealing with car salespeople, and the stakes are even higher for these demagogues. The standards should be just as high. Nobody faults Edmunds for misquoting an invoice or a so-called “True Market Value,” but if I have to look up the “bonus cash” on a particular trim of a particular model out of the 40+ models I sell I am somehow less helpful. I would no sooner believe a Toyota salesperson who told me, “That Chevy will fall apart in a year,” as I would believe a website “salesperson” with thousands of hits to gain or lose and millions of leads to sell. This standard by which I live should apply to these individuals who claim to be the “real” car salespeople.

I have been absolutely brutal on Mike and his efforts in this article, but the reality is that TrueDelta has opened the forum for criticism of today’s pseudoconsumerist “salespeople.” Starting with Consumer Reports and numerous angry and hate-filled authors and websites, they have created a market for themselves with no accountability other than those of us who represent the actual product on the front lines. I criticize TrueDelta today only because it’s the only publication that even tries to explain its methods, publish its actual results, or attempts to explain the overall picture in an industry that only it is willing to elucidate shortcoming.

Yes, TrueDelta is flawed. Yes it’s hard to read, and yes it doesn’t tell even close to the whole story. Similarly, TrueDelta tries to do better once you actually get inside, beyond the knee-jerk “best-worst” ratings at which short-minded visitors only glance.

TrueDelta has been willing to expose its own flaws in order to raise criticism of latter-day “car salespeople,” and to seek a better, more honest, and accurate form or research.

As they sit now, it’s all meaningless arm-waving and web/publication-oriented salespersonship. If someone out there really wants to publish meaningful quality survey information, they need to:

• Actively distinguish what an actual problem is, and include frequent problems in their analysis. If a make or model has problems, what are they? Otherwise, it’s worthless.

• Actively compare model, segment, division, company, and country of origin results, with graphic overlap, of each other and the industry in general whenever appropriate. How much better is the "best" fromthe "average," and the "worst?" Otherwise it’s worthless.

• Post their sample sources, sampling methods, blank surveys, and analytical techniques so that they can be scrutinized and criticized for everyone’s benefit, even if Average Joe doesn’t understand it. How do you establish your grounds to report this information? Otherwise, it’s worthless.

• Post sample sizes, confidence intervals (see 2. above), and skew probabilities for their collected data so that consumers can have real confidence in their results. Have a consistent policy on detecting/dealing with outliers, even if they're simply included in the survey. How good are your data? Otherwise, it’s worthless.

• Write reviews as reviews and opinions, not as demagoguery. Publications and publication websites should offer multiple opinions, as many already do.

Mike has done a good job in setting these wheels in motion, especially on the analytical side of matters. TrueDelta is just beginning a new movement to hold these latter-day salespeople, and their ulterior motives, responsible for their content. He’s been willing to expose his own work to criticism and evaluation in a way that other consumerists have not.

In my critical evaluation, which has targeted TrueDelta in particular, I can only conclude by saying that noone else was even worth the effort.

Sincerely,

Ghrankenstein

[TKR]

As a recent car salesman I offer my for a well thought out and written commentary. I look forward to other comments from posters on this site.

[samkung]

well written, its like reading an article;p

[darknight9]

Ghrank: Excellent, thought provoking and insightful article. Easily one of the best I have seen that tackles the senseless and shameless antics of Consumer Reports, TTAC, Edmunds, et al.

There are scads of consumer lemmings out there that just go and buy whatever these organizations tout as the "best" choice in their respective categories without doing ones own hands-on research. Are there no skeptics left in the US? Nobody left to question anything?

Mabye it's just me, but somewhere in my life I realized that not everything you read or hear in the media is truthful. Sorry folks, it's just not. Today's age is about money, site hits, leads, selling your personal info and recycling every technique in the book to make a quick buck. Don't think for one second that if Consumer Reports or TTAC could garner more revenue for stroking GM or Ford, that they wouldn't do it? These supposed consumer advisory sites are nothing more than the high-class call girls of the internet. They put on a pretty dress and star quality makeup to reel in the suckers who, otherwise would not have a chance or better yet, a clue.

My advice to the car buying public is this: Question everything and really question anyone who says that their product is outright the best anywhere.

Likely that's the reason that car salespeople typically dislike me until I plunk down my offer check. I ask the tough questions. The questions that the average spineless Joe won't ask because he's too afraid of losing a battle of wits with the salesperson. Do you really think that people in a Honda or Toyota factory don't have bad days and misalign a door trim or two? Are their lives so perfect that they never worry about anything while expertly assembling your xB? Believe me, some days their "Kaizen" is half empty instead of half full. I really do think that much of the car buying public walk around harboring these silly ideals. These are the same lamers that would rather poke away at a BlackBerry waiting in line for coffee than acknowledge the next person in line and strike up (for Pete's sake) a conversation or simple "How are you today"?

Stand up and make that Toyota salesperson tell you why the Camry is better instead of taking his wheelbarrow full of garbage research that is: "Well sir, Consumer Reports says it is, so it must be, right?"....

/off soap box..

[unkillsam]

Originally Posted by darknight9

Ghrank: Excellent, thought provoking and insightful article. Easily one of the best I have seen that tackles the senseless and shameless antics of Consumer Reports, TTAC, Edmunds, et al.

There are scads of consumer lemmings out there that just go and buy whatever these organizations tout as the "best" choice in their respective categories without doing ones own hands-on research. Are there no skeptics left in the US? Nobody left to question anything?

Mabye it's just me, but somewhere in my life I realized that not everything you read or hear in the media is truthful. Sorry folks, it's just not. Today's age is about money, site hits, leads, selling your personal info and recycling every technique in the book to make a quick buck. Don't think for one second that if Consumer Reports or TTAC could garner more revenue for stroking GM or Ford, that they wouldn't do it? These supposed consumer advisory sites are nothing more than the high-class call girls of the internet. They put on a pretty dress and star quality makeup to reel in the suckers who, otherwise would not have a chance or better yet, a clue.

My advice to the car buying public is this: Question everything and really question anyone who says that their product is outright the best anywhere.

Likely that's the reason that car salespeople typically dislike me until I plunk down my offer check. I ask the tough questions. The questions that the average spineless Joe won't ask because he's too afraid of losing a battle of wits with the salesperson. Do you really think that people in a Honda or Toyota factory don't have bad days and misalign a door trim or two? Are their lives so perfect that they never worry about anything while expertly assembling your xB? Believe me, some days their "Kaizen" is half empty instead of half full. I really do think that much of the car buying public walk around harboring these silly ideals. These are the same lamers that would rather poke away at a BlackBerry waiting in line for coffee than acknowledge the next person in line and strike up (for Pete's sake) a conversation or simple "How are you today"?

Stand up and make that Toyota salesperson tell you why the Camry is better instead of taking his wheelbarrow full of garbage research that is: "Well sir, Consumer Reports says it is, so it must be, right?"....

/off soap box..

The only problem is that Toyota salespersons can't tell you why. Their training consists of "It's a Toyota so you are dumb if you don't buy it, and we don't negotiate price!"

That's why my sister bought a used malibu for $14,000 over a used Yaris for $15,000. She was able to see the better deal for what it was, more car for less money, something Toyota does not offer.

[Ghrankenstein]

I appreciate the compliments, but I expect my share of broadside responses, and they'll be good. I feel like I did after the many times I brutalized Samuel Adams/Boston Brewing Company in past editorials, while enjoying the results of a market they single-handedly created. That's to say, not entirely positive about discussing the negatives of an overly positive effort.

I'm still just some dude with a keyboard, and a pretty grody one at that. I might be the only dude with a keyboard who sleeps in a tub of KY Jelly and ground-up vitamines, but when all is said and done, that's still what I'll be.

[civilzues]

Originally Posted by Ghrankenstein

We live in a society in which the individual ego and the group dynamic are in constant struggle.

i stopped reading right here, that is from a Geico commercial, which can't be taken seriously...

[jwrebholz]

Originally Posted by Ghrankenstein

I might be the only dude with a keyboard who sleeps in a tub of KY Jelly and ground-up vitamines, but when all is said and done, that's still what I'll be.

That's possibly a little more than I really needed to know.

Otherwise, great article. I didn't understand most of it, but it was written well. (About halfway through something in my brain snapped and I reverted to caveman "FIRE BAD! HORSEPOWER GOOD!" mode.. )

[regfootball]

quite honestly, all the car reliability data bores me to death these days and anyone with a shred of analytical thought knows that every data collection method regarding cars is flawed and suspect in some way. To base your purchase decision solely on all that questionable process is moronic. The fact that people rely on this data and surveys as much as they do is incredibly stupefying.

that does leave the reliability paradigm as a figment of people's own imaginations mostly.....a workmate unloaded his troublesome Jetta last week and bought a new camry. I'm sure he bought the camry because it was the 'best car' but look at toyotas today. sludge, shredding camshafts, etc.

FWIW, medical research is highly questionable as well.

[Ghrankenstein]

Originally Posted by civilzues

i stopped reading right here, that is from a Geico commercial, which can't be taken seriously...

I try not to take anything I write here -too- seriously, part of that demagoguery thing I talked about above. Outside of GMI, I'm a chronic overthinker and hand-wringer, and I do an excellent job of doing a lousy job of making that invisible on GMI. The only person I'm trying to make laugh is myself, and I write a lot of stuff I think is hilarious that goes unnoticed. I'll take all the comic relief I can get, and if it's cryptic all I do is hope that -somebody- else gets it.

I'll throw out that the Enclave review I'm working on will seem a lot funnier to people familiar with early-1990's Chris Elliot.

[Ghrankenstein]

Originally Posted by jwrebholz

That's possibly a little more than I really needed to know.

Otherwise, great article. I didn't understand most of it, but it was written well. (About halfway through something in my brain snapped and I reverted to caveman "FIRE BAD! HORSEPOWER GOOD!" mode.. )

I hear you brother. Fire definitely bad. Horsepower definitely good. And you know what else good? Bread.

[goblue]

It's a strong editorial that mirrors some of my own thoughts.

My problem with all of these surveys, is they do not in any way account for driving style. I will never be convinced otherwise that how you drive the vehicle has an effect on long term quality. However, no one has a good way to control for that - and I don't either.

My biggest issue with CR and now TD (True Delta) as well - is the sampling methods. As someone also with a strong statistical education, I know the quickest way to bias a study is to engage in non random techniques. From what I can tell, JD is the only one who has a true random method, and a high enough take rate to be considered legitimate. That type of random sampling with a reasonable take rate controls at least partially for the huge variance in driving styles. What also controls for this is huge sampling rates. 30 is not enough. Take into account driving conditions - and the sample rate goes well into the thousands to get a reasonable p value.

What I like about TD is the pricing guides. The commentary and reviews leave alot to be desired in my opinion suffering from import bias - I've read a number of his epinion reviews, and see a similar theme.

That leads to a huge problem, one CR suffers from as well - perceived bias. The same company processing the stats are writing reviews. Does this have an effect, with CR its almost certain. With TD - we'll see. When customers trust a reviewer and then review their car on the site, they are preconditioned to report issues. Thats a problem.

Bottom line, to get any accurate information, you need a huge, truly random sample over a multitude of demographics, driving styles, and climates. JD by nature of sheer volume is probably the closest - but TD is right, they have serious issues with their questions - and this is one area TD has an edge. JD however doesn't review vehicles and doesn't condition their respondents - a huge plus in perceived and actual bias.

[tamatt27]

To be honest, I didn't read most of the article. However, after reading the paragraph regarding CSI, I've decided to whole-heartedly agree with everything you say!

[Relik86]

Originally Posted by unkillsam

The only problem is that Toyota salespersons can't tell you why. Their training consists of "It's a Toyota so you are dumb if you don't buy it, and we don't negotiate price!"

That's why my sister bought a used malibu for $14,000 over a used Yaris for $15,000. She was able to see the better deal for what it was, more car for less money, something Toyota does not offer.

My sister and her husband were at a Toyota dealership looking to buy a Sienna, because her husband had bought into that same mentality.

Then he asked about low new vehicle financing rates, and the salesperson said "we don't have to do that, our vehicles are just better". So they left and bought a used Impala

[Ghrankenstein]

Originally Posted by tamatt27

To be honest, I didn't read most of the article. However, after reading the paragraph regarding CSI, I've decided to whole-heartedly agree with everything you say!

I wouldn't go that far.

I've been wrong before, and will be again. I don't write to be agreed with any more than Mgescuro does. I write to say what's on my mind, which is usually a combination of something that's been bugging me for awhile and Futurama.