Consumerists: The New Car-Salespeople

[eaton53]

that does leave the reliability paradigm as a figment of people's own imaginations mostly.....a workmate unloaded his troublesome Jetta last week and bought a new camry. I'm sure he bought the camry because it was the 'best car' but look at toyotas today. sludge, shredding camshafts, etc.

Except for a few bottom feeders chances are the car you buy will be pretty reliable. Paying high prices to get rid of 17/100 of a defect is completely retarded, IMO.

What I mostly look for is how much up front and how much to maintain. I've had imports and domestics... import maintenance has always been more expensive, sometimes a lot more with no additional benefit that I can see. In fact, I've been taking my Miata to a GM dealer for routine maintenance because it has better hours and lower prices.

Resale is another one.... Honda buyers love to argue that they are great for that reason. IMO, paying extra to get high resale is another example of fool's gold.

For example, a Pontiac G5 GT is over $3K less out the door than a comparable Civic after all offers. The difference is exacerbated if financing is involved... now the Honda buyer is paying interest on the difference.

Even if there's no financing, the Honda buyer has thousands wrapped up for years in a car, while the Pontiac buyer's thousands are in investments earning money. The longer you hold the car, the larger the cost.

If you keep your car a long time the resale argument becomes moot... most of the money is lost along with the money that could've been made off of it. This can easily add up to hundreds, even thousands more if that money is put toward retirement.

Those who argue resale tend to ignore opportunity costs they incur to get that resale.

On auto reviews, I don't really worry about them. I drive the car myself and I'm the only one who has to like it. Well, me and my wife.

[mkaresh]

Ghrank:

Just stumbled upon this. A head's up would have been nice.

You're confusing skew with subjective bias. They are two totally different things. "Skew" as used by statisticians is not the same as "skew" as a synonym for subjective bias.

All "skewness" means in the statistical sense is that the data do not form a normal distribution, which is essentially a symmetrical bell curve centered on the average. Everything in the universe is not normally distributed. Many things tend to be, and the field of statistics has found that it's very useful to assume that just about everything is, even when you already know it is not. All a test for skewness determines is how far from a normal distribution the data are. It says absolutely nothing about whether or not responses were subjectively biased.

In the case of my data, individual responses are not normally distributed for the simple reason that most cars these days do not have problems within a one-year or shorter timeframe. You cannot have a normal distribution when the mode is zero and negative numbers are not possible (you cannot have a negative number of repairs).

My analyst has used bootstrapping to test whether the confidence intervals are valid despite the violation of normality. This determined that in many cases the lower confidence intervals I have posted are about 0.1 repair trips per year wider than they should be. So to the extent I have distorted results, the intervals make the data look a bit shakier than they actually are.

With the sample sizes I have, it would make no sense to post the actual problems people had. In most cases only one or two people had a particular problem. I wouldn't feel comfortable posting specific problems unless they were experienced by at least five people, and preferably at least ten. If the problem is experienced by ten percent of all cars, we're talking about a sample size of at least 50. Which I do have for a few cars, but not for many yet. This will be done in the future.

What might be the main point of your critique is especially frustrating for me. You want me to include more detail, but present it in a way that lazy people will fully understand it. You want me to say more, while also saying less. This is a common request. And I'd love to do just this. I just haven't figured out how to do it. It's the whole, "I want a full-size SUV that goes from 0 to 60 in 5 seconds and gets 50 MPG." Maybe it's possible, and I'm always hunting for ways to improve what I do, but it certainly isn't easy.

That said, if you or anyone else has a possible solution to the problem of how to report my results, I'm all ears. I started reporting problems per 100 rather than per car this time because many people suggested this, to avoid the complication of decimals. Apparently many people get confused by decimals. I really wanted to stay with the decimals, but when people have specific suggestions, and they're persuasive, I adopt their suggestions.

I used to put the CIs, ugly as they are, on the main page, but that page was getting very long and no one seemed to be reading them. So, at the suggestion of a number of people, I put them on a second page linked from the main page. To date absolutely no one has emailed me wondering where they went, and those results have been viewed about 5,000 times.

What you haven't said, but I suspect is one thing that was heavily on your mind when you wrote this piece, is that GM vehicles didn't do especially well in my results. Which pretty much matches how well they do in JD Power and CR. Similarly, Ford did well in my results, and has been doing well in the others. Sorry, there's no big conspiracy here. We're all measuring the same population, if in different ways, so our results are bound to track together.

In none of these surveys do result vary wildly from year to year, as you imply. You do appear to have noticed the major anomaly in my latest results, which involves a couple years of the Honda Odyssey, both of which are asterisked. These are two results out of 65. The fact of the matter is, even with good sample sizes you're going to have a small percentage of results that are off. And the two in question ARE asterisked to denote that the sample is insufficient.

I've said all along that the benefit of my approach would not be a different rank order in the results, but that the size of the differences would be much clearer. You can say, "The Aura did worse than the Camry and Fusion, that's awful." Or you can say, "But the difference is only a couple tenths of a repair per car." I wouldn't say the former. The latter is a major part of what I hope to get across.

[M7Mayberry]

What an excellent post! This is the reason I signed up with GMI. Straight talk, honest discussion. And, so far, no name calling, etc. Two personal notes:
I recently bought a Saturn Vue. When I read forums complaining about all the problems with Saturns, I wonder if I made the right choice. So....I quit reading the complaints and my Vue runs just fine, thank you.

The Chevy S-10 always received "fair" or "below average" in safety ratings. And yet, every time I read about an S-10 being in an accident, the occupants suffered only minor or no injuries.

[mkaresh]

Originally Posted by goblue

It's a strong editorial that mirrors some of my own thoughts.

My problem with all of these surveys, is they do not in any way account for driving style. I will never be convinced otherwise that how you drive the vehicle has an effect on long term quality. However, no one has a good way to control for that - and I don't either.

My biggest issue with CR and now TD (True Delta) as well - is the sampling methods. As someone also with a strong statistical education, I know the quickest way to bias a study is to engage in non random techniques. From what I can tell, JD is the only one who has a true random method, and a high enough take rate to be considered legitimate. That type of random sampling with a reasonable take rate controls at least partially for the huge variance in driving styles. What also controls for this is huge sampling rates. 30 is not enough. Take into account driving conditions - and the sample rate goes well into the thousands to get a reasonable p value.

What I like about TD is the pricing guides. The commentary and reviews leave alot to be desired in my opinion suffering from import bias - I've read a number of his epinion reviews, and see a similar theme.

That leads to a huge problem, one CR suffers from as well - perceived bias. The same company processing the stats are writing reviews. Does this have an effect, with CR its almost certain. With TD - we'll see. When customers trust a reviewer and then review their car on the site, they are preconditioned to report issues. Thats a problem.

Bottom line, to get any accurate information, you need a huge, truly random sample over a multitude of demographics, driving styles, and climates. JD by nature of sheer volume is probably the closest - but TD is right, they have serious issues with their questions - and this is one area TD has an edge. JD however doesn't review vehicles and doesn't condition their respondents - a huge plus in perceived and actual bias.

So much here is off-base.

Driving style and long-term quality: I review every repair response personally. Most things that require repair simply cannot be affected by driving style, most notably electrical problems. Does how a car is driven make the power window motor die?

Driving style could well affect major engine and transmission repairs. But it won't be the cause of most of them, and these are rare to begin with.

That said, if I was reporting the rate of major engine and transmission repairs, then it might be important to try to get at driving style. But currently I'm not. I'm measuring ALL repairs, and the percentage of ALL repairs that might be affected by driving style is low.

Sample size: a recent round of JD Power's VDS (might have been 2004) had a target sample size of 225. No one has a minimum sample size of 1,000, much less "in the thousands." As noted in my response to Ghrank, even with my very low sample sizes and the fact that I don't measure the same things or ask the same questions they do, my results are tracking with others.

Bias in my reviews: I don't think there's a manufacturer I haven't praised in some reviews and criticized in others. Just because you don't agree with my reviews doesn't mean I'm biased. If I liked everything GM made,--or anything anyone made--then I'd be biased. Like everyone else I have specific things I like and dislike in cars, but this doesn't mean I have an import bias.

My general take on all car reviews:

http://www.truedelta.com/pieces/comparison_tests.php

"That leads to a huge problem, one CR suffers from as well - perceived bias. The same company processing the stats are writing reviews. Does this have an effect, with CR its almost certain."

I'm far from a booster of CR. But "almost certain?" Do you have any evidence for this aside from your desire for it to be true? And, no, "my friend's uncle did this" doesn't count.

In general, your arguments start from the conclusion you desire, rather than start from evidence and lead to this conclusion. This is the very defintion of bias.

[mkaresh]

Originally Posted by M7Mayberry

What an excellent post! This is the reason I signed up with GMI. Straight talk, honest discussion. And, so far, no name calling, etc. Two personal notes:
I recently bought a Saturn Vue. When I read forums complaining about all the problems with Saturns, I wonder if I made the right choice. So....I quit reading the complaints and my Vue runs just fine, thank you.

The Chevy S-10 always received "fair" or "below average" in safety ratings. And yet, every time I read about an S-10 being in an accident, the occupants suffered only minor or no injuries.

Forums tend to make problems seem more common than they are. News stories are even worse. Ghrank mentioned the Tundra camshafts, and wonders why this sort of thing isn't reflected in Toyota's reliability ratings. The reason is simple: 20 failures in 20,000 or so trucks is enough for a news story, but it's not enough to affect a reliability rating. It's one failure per 1,000 vehicles.

On the S-10: the great majority of accidents in any vehicle result in minor or no injuries. The issue is your chances of survival in a truly severe crash. And though the difference between vehicles is probably large (I'm not an expert here), it certainly isn't the difference between "certain death" and "walks away with just scratches." Driving style also varies from vehicle to vehicle. Actual death rates are likely much lower in a minivan that scored poorly in crash tests than in a sports car that tested well. Most S-10s are probably driven more like minivans than like sports cars.

[regfootball]

wouldn't the world be a better place if we put the effort into our jobs and our personal realtionships was the same as the effort we spend analyzing a car purchase?

[M7Mayberry]

The S-10's poor safety ratings were mostly due to leg injuries, according to the NHTSA or whatever, at the time. I will take body-on-frame over unibody anytime. (Pretend I don't have the Saturn)

Originally posted by Mkaresh: Most S-10s are probably driven more like minivans than like sports cars. You've never seen me or my daughter drive!!!!! LOL

[goblue]

So much here is off-base.

No - just because I don't believe everything you say doesn't mean you're right.

Driving style and long-term quality: I review every repair response personally. Most things that require repair simply cannot be affected by driving style, most notably electrical problems. Does how a car is driven make the power window motor die?

Drive a car aggresively over averege to rough roads and you will cause electrical systems to fail, creaks and rattles to appear, etc. Where did I get this little tidbit - a QC engineer from a Big 3 company. Driving style affects almost anything in a car.

Driving style could well affect major engine and transmission repairs. But it won't be the cause of most of them, and these are rare to begin with.

That said, if I was reporting the rate of major engine and transmission repairs, then it might be important to try to get at driving style. But currently I'm not. I'm measuring ALL repairs, and the percentage of ALL repairs that might be affected by driving style is low.

I disagree, and others have as well. Drive a car aggressively, consistently, and see what happens relative to one driven by a little old lady.

Sample size: a recent round of JD Power's VDS (might have been 2004) had a target sample size of 225. No one has a minimum sample size of 1,000, much less "in the thousands." As noted in my response to Ghrank, even with my very low sample sizes and the fact that I don't measure the same things or ask the same questions they do, my results are tracking with others.

I didn't say what sample size JD uses, or anyone else, what I am saying is thats what it would take to get truly accurate results.

Bias in my reviews: I don't think there's a manufacturer I haven't praised in some reviews and criticized in others. Just because you don't agree with my reviews doesn't mean I'm biased. If I liked everything GM made,--or anything anyone made--then I'd be biased. Like everyone else I have specific things I like and dislike in cars, but this doesn't mean I have an import bias.

My general take on all car reviews:

http://www.truedelta.com/pieces/comparison_tests.php

I read many of your reviews, thats my opinion. I saw a trend against American manufacturers. Nothing as severe as CR or anything, but thats my opinion.

"That leads to a huge problem, one CR suffers from as well - perceived bias. The same company processing the stats are writing reviews. Does this have an effect, with CR its almost certain."

I'm far from a booster of CR. But "almost certain?" Do you have any evidence for this aside from your desire for it to be true? And, no, "my friend's uncle did this" doesn't count.

Just read what even Toyota's VP said about it. Seriously, it comes from the sample bias. Also, you might note many auto manufacturers have spoken about against CR for these reasons. The basis for the claim is a sound one. Thats why I said almost certain, I - no one has certain proof - they would have been shut down if that was the case.

In general, your arguments start from the conclusion you desire, rather than start from evidence and lead to this conclusion. This is the very defintion of bias.

No, acc'd to Webster's one defintion of bias can be defined as : systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others. Sounds very germane to this discussion.

[mkaresh]

I'm not saying that driving style and road conditions don't have an impact. I'm just saying that the impact is smaller than you imply. FWIW, alignments and brake jobs are excluded from my analysis because they are two repairs that tend to be heavily affected by these extraneous variables. Brake jobs clearly have a significant impact on some others' results.

The brands that you'd expect to benefit from driving style are those driven by the oldest drivers: Buick, Lincoln, Mercury. Do you really want to argue that the standing of these brands is due to driving style? Or does this variable only apply when it suits you?

It's simply false that a sample size in the thousands is necessary for accurate results, at least not with regard to what I'm measuring and how precisely I'm trying to measure it. If you wanted to know the rate of A/C compressor failure within one percentage point then, yes, you should probably have a sample size in the thousands. But I'm not doing that.

Would I like to report more detailed information and include more variables? Absolutely, and I plan to. But that's a goal that must be worked toward. I am very consciously working within the limits of my methods and sample sizes.

In general, my reviews track with others out there. And there's no big conspiracy where reviewers all get together and decide how to review a car. As I said before, I have not consistently praised or panned the products of any manufacturer.

CR's survey is more susceptible to their reviews than mine is. They let people decide if a repair is serious enough to report. If they give a car a positive review, and people tend to like it partly as a result, then problems could well seem less serious.

My survey has people report every repair that is not in a list of wear items. It doesn't provide that sort of wiggle room.

Still, the effect will generally be in one direction: under-reporting of problems that occur. If the cars simply don't have problems, then there's no impact. People aren't making up a significant number of problems.

I've been recommending the Ford Freestyle for a couple of years now. Has this affected survey responses? In my results, the 2005 has a moderately high repair rate, while the 2006 has a low repair rate. My review didn't change from one model year to the next. I don't think people drive the 2006s less aggressively or over smoother roads, either.

What I do think happened is that Ford really got it's act together on this car after the first model year.

I wasn't saying that there was only one definition of bias. You'll never like anything where GM doesn't look great, and that is the simplest form of bias.

In general, people with strong biases tend to perceive me as biased. It doesn't matter what their personal bias is. I'm been called biased by fans of import brands as well as fans of domestic brands. Strongly biased people want all of the results to come out their way. And reality is rarely if ever that clean.

[mkaresh]

Originally Posted by M7Mayberry

The S-10's poor safety ratings were mostly due to leg injuries, according to the NHTSA or whatever, at the time. I will take body-on-frame over unibody anytime. (Pretend I don't have the Saturn)

Originally posted by Mkaresh: Most S-10s are probably driven more like minivans than like sports cars. You've never seen me or my daughter drive!!!!! LOL

But I have seen my father-in-law drive

I will grant that many of these trucks were probably bought by young drivers because of the low price, and the driving style of these drivers might not be the best.

[mkaresh]

Originally Posted by regfootball

wouldn't the world be a better place if we put the effort into our jobs and our personal realtionships was the same as the effort we spend analyzing a car purchase?

I've actually made a similar point for years. Many people expect their cars to be far more perfect than ANYTHING they personally produce.

I've also accused CR of presenting results in such a way that people will exaggarate the size of differences in quality. Though there are many problems with their methods, this one I see as by far the largest.

I do not expect my methods to yield a much different ranking than theirs. The size of the differences will simply be much clearer. And often this size is small enough that the implication is that people really should not be worrying as much about reliability as they do.

[CountryME]

Great article Grank. MKaresh seems to be taking the article as it was intended, to try and improve the consumerist web pages.
I believe that the research that someone has to do has increased in recent years to ensure that the second largest purchase(behind a mortgage) they make is not a mistake. During my most recent car buying experience, I was amazed at the ineptitude of the sales people about their own product. If I walk into a dealership and for example say " I want a car with factory remote start and a bench seat" the salesman doesn't even know on what trim packages these options are available. When was the last time you talked to someone who ordered a car? The salesman's role is to move product off the lot, not find the exact car a customer wants.
Thus, consumers are being driven to become more knowledgeable using internet research. This research isn't just limited to the options and specifications of the purchase, it now needs to include informed research into the actual quality of the vehicle being purchased. This research needs to eliminate the lemming mentality that has emerged as a result of CR and people believing every television commercial that they see. As Grank and MKaresh implied, people need to be making these decisions based on real data, not the sensationalized story on the evening news. I may not like that data shows a certain manufacturer has better quality or reliability scores than the one that I would prefer to buy, but if the data is true and unbiased I can at least purchase the vehicle I prefer knowing what I am getting.
While I do not want big government, an independent oversight of the consumerists may be what is needed to ensure the information being provided to consumers is statistically correct and unbiased.

[asim]

It looks like a couple people just picked up their statistic books. Ghrank, you use (p<0.10)? Thats nice, I usually determine it at 0.05. Anyhow I am glad people like you Ghrank and Mkaresh are questioning these traditional surveys that the big companies have. They don't report on their ways of finding their information, which I believe is imperative. The people should know how these results were founded.

Also, thanks Mkaresh for giving a confidence interval, it really puts things in perspective. =]

[Ghrankenstein]

Just stumbled upon this. A head's up would have been nice.

Sorry about that. It kind of plopped out, and then I went to sleep. The accuracy of my hindsight is uncanny. I've been angry with myself a lot lately over the quality of the content I've been working on, most of it being crap and all, and the result was rushed and jumbled in my eyes. I decided against consulting you before posting because I'd rather be incompletely informed or outright wrong than be accused of writing a cheery, "Here's how TrueDelta does things the RIGHT way," article with help from Mike Karesh.

Also, see my Samuel Adams/BBC reference above. I knew you'd see it as as an attack, and yet not something personal. I also know that CR and JDP have other things on their minds, and at best consider me a contributor to a "GM Fansite" that isn't worthy of a response.

In retrospect, I doubt if I'd have been happy with the final product either way. That's par for the course at Ghrankenbriar. More often than not GMI is willing to cut me more slack than I ask for or even want.

You're confusing skew with subjective bias. They are two totally different things. "Skew" as used by statisticians is not the same as "skew" as a synonym for subjective bias.

No I'm not. My commentary was NOT about subjective bias. In this case I was purely questioning how rigorous the data actually are. I questioned the survey responses just as much as the methodologies.

All "skewness" means in the statistical sense is that the data do not form a normal distribution, which is essentially a symmetrical bell curve centered on the average. Everything in the universe is not normally distributed. Many things tend to be, and the field of statistics has found that it's very useful to assume that just about everything is, even when you already know it is not. All a test for skewness determines is how far from a normal distribution the data are.

I know I'm treading on thin ice because I can't pull the entirety of statistical assumptions out of thin air, but random sampling from a normal population is standard for all but a few specialized tests. You've addressed this below, with the exact response I was hoping for. Ultimately, I was a little frustrated because I wanted to see more about your methodology, even if I'm grateful for what you have already shown. As I said originally the more information you provide on your methodology, the more of a credibility advantage you'll enjoy over the other surveys.

My analyst has used bootstrapping to test whether the confidence intervals are valid despite the violation of normality. This determined that in many cases the lower confidence intervals I have posted are about 0.1 repair trips per year wider than they should be. So to the extent I have distorted results, the intervals make the data look a bit shakier than they actually are.

That's what I'm talking about. I've got a couple of days worth of resampling training, and no desire to show off my level of "knowledge," but I hoped you'd fill in exactly as you did. Understand that you can't provide too much information about your methodology. As a parallel, I take most of Mgescuro's posts with the perspective that if he's saying it, other people are thinking it. The fact that you -have- a methodology that you're willing to discuss has opened the door for criticism of JDP and CR.

With the sample sizes I have, it would make no sense to post the actual problems people had. In most cases only one or two people had a particular problem. I wouldn't feel comfortable posting specific problems unless they were experienced by at least five people, and preferably at least ten. If the problem is experienced by ten percent of all cars, we're talking about a sample size of at least 50. Which I do have for a few cars, but not for many yet. This will be done in the future.

I empathize, because you can't work with data you don't have. The project is still in its infancy. It sounds like faint praise, I know, but when I say the data don't mean much yet, I -do- think they will, and the fact that you've not really hidden the data's limitations makes them more valuable than CR or JDP.

I -still- want to know if the "Best" are actually better, and if the "Worse" are really worse, even on the most liberal statistical terms. You imply that they aren't (i.e. 0.1 problems per car type-statements), when CR and JDP unabashedly say that they are, but don't provide any evidence to back up their arguments. It throws the whole industry of consumerism into question, which doesn't do you any good, but it does a great deal of service by holding the consumerist industry to the same standard that we front-liners are held.

Real reliability information has to go this route, or it's just more marketing for some reason or another.

What might be the main point of your critique is especially frustrating for me. You want me to include more detail, but present it in a way that lazy people will fully understand it. You want me to say more, while also saying less. This is a common request. I just haven't figured out how to do it. It's the whole, "I want a full-size SUV that goes from 0 to 60 in 5 seconds and gets 50 MPG." Maybe it's possible, and I'm always hunting for ways to improve what I do, but it certainly isn't easy.

Yeah, I -have- thought about this a lot, but no it wasn't the main point of the article. I'm not surprised that you took it that way, but no, it wasn't the main point. It -was- a point.

Simple graphics that say a lot don't just create themselves, and I feel (and criticized the fact) that shoppers are only looking for the "best-worst" continuum that you already provide. You're providing what John Q. General Public wants to see, and it's good for you/CR/JDP, but frustrating for me, because I'm held to a far different standard.

I have a beef with the "Best-Worst" carat over the meaningless rainbow bar. Keep in mind that it's still way better than CR or JDP, but it doesn't tell the story that the more deeply embedded confidence-interval pages do. If it were me, I'd replace that carat with a bar reflecting that confidence interval, assuming that the rainbow bar reflects the industry distribution. I'd also include another bar, below, for the appropriate vehicle segment. The reason is that if there's a bunch of overlap, I want to see it.

I do have a simpler format in mind, but need a larger number of more precise results first. That format is here:

Can't get the link to work, says "click OK three times" and that's it. There's no "OK" to click.

I used to put the CIs on the main page, but that page was getting very long and no one seemed to be reading them. So, at the suggestion of a number of people, I put them on a second page linked from the main page. To date absolutely no one has emailed me wondering where they went, and those results have been viewed about 5,000 times.

Fair enough. I'd still like to see at least segment-specific confidence intervals. As before, it's a matter of the "Best" really being better, and the "Worst" really being worse. Otherwise, it really doesn't mean anything.

One thing I get especially tired of on this forum is, "This person doesn't thing GM products are the best, so they're clearly biased." Not only this forum. On a Chrysler forum someone said I was biased because no one in my family works for Chrysler. Crazy.

As you know, I've never accused you of that. Nor do I think it. The thing is that nobody on GMI accuses me of not being fair and critical of GM vehicles in my reviews. I haven't seen that type of response directed at you on GMI, other than once in this thread; the key is posting opinions as opinions. They are biased from the beginning, regardless of any disclaimers, and you know my stance on this.

What you haven't said, but I suspect is one thing that was heavily on your mind when you wrote this piece, is that GM vehicles didn't do especially well in my results.

Again, that's not the case. I was aware that my timing was very convenient, and I'm willing to take the criticism. I wanted to post an editorial at a different time, but the thread came up, the rant got started, and that "convenience" factor wouldn't have changed for another year.

Knowing what I know about ulterior motives in consumer reporting, which are supposedly equal across the board, and my questioning what I've questioned about ulterior motives in consumerism, the article wouldn't have been any more or less valid in another month or another six months. The fact that I didn't get off my butt and do it a month, six months, or whatever ago is my problem. I have to hide behind the fact that I'm just a dude with a keyboard, and I'm writing about a passionate subject and not for a living.

In none of these surveys do result vary wildly from year to year, as you imply. Even with my small sample sizes the results have been consistent from quarter to quarter. You do appear to have noticed the major anomaly in my latest results, which involves a couple years of the Honda Odyssey, both of which are asterisked. check out the four years of the Mazda3 while you're at it. Every single one is either a 0.3 or a 0.4 Check out the improvement I show in the Ford Five Hundred from 2005 to 2006, and note where it placed in the latest IQS.

I did notice the Odyssey, like many critics would, because it was a sore thumb. Possibly a dumb move, since it -is- an anomaly when I've been talking so much about trends and overlap. I also implied wildly varying results because this year's JDP IQS was so markedly different from last year's. Yes, I'm criticizing TrueDelta in this case because such results -do- bring into question the rigor of the data and because they are still reported albeit with an asterisk.

I'd like to think that if your data showed a similar anomaly with a domestic vehicle, I'd have pointed it out just as readily. You either don't have one, or I didn't notice it, so it's moot. Just keep in mind that I have great respect for the fact that you were willing to report the anomaly in the first place.

I've said all along that the benefit of my approach would not be a different rank order in the results, but that the size of the differences would be much clearer. You can say, "The Aura did worse than the Camry and Fusion, that's awful." Or you can say, "But the difference is only a couple tenths of a repair per car." I wouldn't say the former. The latter is a major part of what I hope to get across.

I agree, but you aren't showing anything other than the same old story that the established consumerists are showing, and in the same format. If you really want to show that level of overlap, then you have to do it, and you have to stress it in your graphics If your survey responses are not random, you have to say so effectively so that we don't see TrueDelta as yet another consumerist echo-chamber.

I knew that you'd respond to this article as an attack, when it was largely a compliment in the face of a parallel industry that has managed to avoid any real measure of accountability by parlaying its own responsibility on us know-nothing frontliners. You haven't done that, but the previous consumerist industry has.

Until I see something that definitively contradicts that consumerist industry stance, and accepts accountability for its research and assertations, I'll continue to be just as skeptical. That's the consumerist industry, and not you, and the fact that you've opened the door with grounds for criticism has made TD a topic. If I'm skeptical, and use TD as an example, it's still directed more at JDP and CR, since they still don't see themselves as open to criticism.

Sincerely,

Ghrankenstein

[Ghrankenstein]

Originally Posted by asim

It looks like a couple people just picked up their statistic books. Ghrank, you use (p<0.10)?

I was specifically referring to p<0.10 for tests against the normality of the data. I never liked the fact that the scientists with whom I worked insisted on a 0.05 threshold when checking their assumptions because it was basically a licence to work with crappy data sets.