The NYT vs. Rider Training: Getting it All Wrong.

[ericalm]

I'm a fan and regular reader of the New York Times "Wheels" blog so was rather shocked when I saw the following headline this afternoon:
"Motorcycle Training Does Not Reduce Crash Risk, Study Says"

The lead sentence: "Courses designed to make new motorcyclists safer are not decreasing crashes, according to a new study by the Highway Loss Data Institute, an affiliate of the Insurance Institute for Highway Safety."

This post not only gets under my skin as an example of the decline of journalism (regurgitate the press release, get a couple quotes, post online) but is potentially dangerous. The possibility that someone could find that headline or read the post and come away thinking that rider training is useless sickens me.

In essence, neither the headline nor the lead are supported at all by the study and the data it presents. It's faulty logic and irresponsible journalism. Worse, these generalizations are contradicted directly in the study that's the subject of the post.

Here's what the study did find regarding training: In states that mandate training in order for riders under 21 to obtain a license, insurance collision claims by for riders under 21 is 10% higher than states that don't require training.

The IIHS press release notes that this number "isn't statistically significant" but then states, "it contradicts the notion that training courses reduce crashes."

Utter baloney. Unless the IHSS is sitting on some significant data that proves this point, they present no evidence that indicates that the statistically insignificant increases in claims in these states is, in fact, caused by the mandatory training. In fact, language in the study itself suggests other possible causes.

If the reporter had looked beyond the press release to the actual report issued by the IIHS, she would have found this:

The lack of statistical significance means it cannot be said with confidence that the collision claim frequencies of riders subject to a state education requirement actually are more likely to crash than riders of a similar age. However, if the increase is in fact real, one potential explanation might be that in some states, a participant is fully licensed upon completion of a course. This could, in practice, shorten the holding period for the permit and hasten riding.

It is important to emphasize that this analysis does not answer the question of whether riders who voluntarily take rider education courses have higher or lower crash risk.

(Why does their own press release suggest otherwise? I can't say for sure but I doubt that their press releases are written by their researchers.)

Additionally, mandatory training and the toughening of requirements for getting a motorcycle license created a disincentive leading to much higher numbers of unlicensed riders on the roads. Historically, unlicensed riders have had a higher risk of crashing. Neither this story nor the IHSS press release and study address whether the insurance claims from under-21 riders in these states were actually from licensed riders who completed the training.

[Skootz Kabootz]

... a new study by the Highway Loss Data Institute, an affiliate of the Insurance Institute for Highway Safety."

If the Insurance Institute for Highway Safety is behind the statement it wouldn't surprise me if they were skewing the information so they can raise rates...

[LuvMyScoot]

Studies like these make me sick too because they haven't truly looked at all the variables. For instance, I'd like to know the overall quality of automobile drivers in the states that require MSF training. If a study shows that those states have a high rate of accidents regardless of vehicle than it wouldn't surprise me that they would have a higher rate of claims involving motorcycles since a great deal of motorcycle accidents are caused by other vehicles. They also haven't bothered to investigate how bad an accident might have been if the motorcyclist hadn't had the training; a person with proper training can still be involved in an accident but without the training any injury might be much worse.

I read better investigative journalism in Vanity Fair than I do in most newspapers.

[illnoise]

The IIHS is full of gems like this one. Grrrr.

The Wheels blog is usually great, and they have some really scooter-friendly writers.

I realize training doesn't make you invincible, but it's GOTTA help. I can't wait for the MSF's rebuttal, which will be universally ignored because there is no justice in this world. (See also: Anti-Snell BS etc.)

[Hipnerd]

It's an excellent example of pointing out a corollary without discerning causation.

What if the states with mandatory training classes already had abnormally high accident rates, and so they decided to implement tougher licensing requirements?

Not enough information to draw conclusions. The study freely admits this. I wish the article was as clear.

[ericalm]

Even further: The numbers aren't based on number of crashes but numbers of insurance claims. It's quite possible that in states mandating training, a higher percentage of under-21 riders are actually insured.

And… How about correlating with states that have anti-cell phone use/texting laws?

Bah. Still riled by this, obviously. Need to go watch Lost.

[Cheshire]

Sounds to me like the insurance companies are trying to come up with an excuse to stop giving a rate deduction for MSF completion, even if they have to fabricate one.

Lost was pretty good.

[Skootz Kabootz]

I'll say one thing, my MSF rider training (and my continued study following it) has saved my life, or at least helped me avoid injury, dozens and dozens of times, if not every time I ride.

[sunshinen]

This is a good lesson in the huge communication gap between scientists and the PR dept/journalist/lay person. And why most scientists mistrust the media...

Scientists tend to be very precise in their language ... and the way they discuss their results is often with a very different emphasis and different language than lay people would use. For scientists it's all about what they can prove. For the rest of us, it's all about the "real world" implications. Often that real world meaning gets lost in that PR/media translation.

Scientist: "Our study did not show that A results in Z."

Media Interpratation: "Study shows A does not result in Z."

The whole story: The scientists set out to test whether A results in Z in a very limited, controlled setting. Unfortunately, the results of the study were not conclusive. It could still be very probably that A results in Z, there are still several other variables that should be tested and accounted for (such as C, D, E, the exclusion of confounding factor F, the combination of factors L and Q... — They are looking for funding to study these factors, but no one wants to pay for it...) but this particular, limited study could not prove that A results in Z. Unfortunately, "could not prove" is a pretty boring headline... "does not help!" is much more exciting.

[Vic]

I do agree that the article seems very flawed, at best, and the study seems to be quite limited so the results are not very helpful.

I have thought about the states that allow a MSF course to provide licensure, Ohio is one of these. It seems to me that we need to have some graduated licensure, much as we do with teen drivers, for the same reason that we have graduated licensing with teen drivers.

When I got my scooter a girl at work went out and bought a Triumph Triple somethingorother crotch rocket with way more power than is needed. She went out and took the MSF course and got her license, at which time she had ZERO-none-not even a single inch worth of experience piloting a two wheeled vehicle on a street. But, because she had taken the course she was licensed to ride anything, anywhere.

This seems insane to me.

I started out with a 150cc Chiscoot that topped out at about 45mph on a good day, transitioned to a Buddy 125 and took my test on that (got licensed with no problem), I have added a Sym 250cc scooter and ride that also. I took it easy with the roads that I ride on, to begin with I was limited by the power of my ride, I gained experience and gradually increased my cc which opened up different roads that I can take that scooter on.

My friend had her shiny new license, her overpowered bike and was struggling just to handle brake, clutch, friction zone, keeping upright, not getting rear-ended, watching for traffic, watching the light change, checking for oncoming traffic...

It seems to me that some sane graduated licensing would make a very big difference and I would be very interested in seeing a study that shows accident rates of new drivers that take the MSF course and are limited for the first x number of miles of experience (my friend also got her license but did not take delivery of her bike until almost a year afterward, so only spent a few months of being a "MC Novice*") that are riding smaller size vehicles and are limited in the roads that they are permitted to travel on as opposed to those who take the course and get maybe a mile or two of experience in MSF and then are set loose to ride whatever they want where-ever they want. I think this would be far more instructive! There are so many variables that this study does not seem to me to show much of anything of any use.

*in Ohio a MC Novice means you have a little notation on the bottom of your license for the first year after you are licensed and you have to wear a helmet. No other restrictions or anything.

-v

[sunshinen]

Vic also makes a good point.

All other things being equal, I have no doubt the MSF is a boost to your safety. BUT... all things are not equal, and the way the course leads to an easy Motorcycle license is probably a big problem.

I recommend the MSF for the knowledge it imparts, the emphasis on practicing, and getting some time in on a closed course to develop the basic skills — but utter beginners who take it and feel like they are ready to ride wherever whenever are probably fooling themselves!

I would say 80-90 percent of the people who passed the course I took should NOT have gotten on a motorcycle and felt like they knew what they were doing. But getting that M makes you feel like you have a license to ride (cuz you literally do...), and that false confidence could very easily lead to more crashes than for those who get their permit, spend more time over a longer period of time learning to ride with an experienced rider ... and then take the skills test at the DMV (where the person scoring you has no interest in passing you other than determining if it is actually SAFE for you to be out on the roads).

We use the MSF as an easy way to get the M on our license; however (while that is a huge motivating factor to take the class), giving people a license at the end of the basics course may not actually be in the best interest of many of the riders.