What can New Zealand’s Road Toll Teach us?
A perennial topic at this time of year in New Zealand is the holiday period road toll, probably because there is precious little in the way of domestic news at this time of year. The inure that gets discussed the most is the Christmas / New Year Holiday Period Road Toll, defied as the number of road deaths occurring between 4pm on Christmas Eve and 6am on the first work day of 2015 (that being January 5th this year).
Prior to events in France, our news was full of the high road toll – 17 deaths, up from 6 last year. This has caused no end of consternation in the media. People have blamed everything from drug use, to driver attitude to the Police’s low tolerance for speeders to causing the sharp jump in deaths.
There are, in truth any number of reasons one can come up with to explain the increase in deaths, but all of them have the same glaring weakness – they all start from the premise that the road toll has jumped sharply. That the case is made with just two data points irked at me until I decided to investigate a little.
First off, there are three problems I noticed in the media analysis:
- If we are worried about road safety we should arguably focus on fatal crashes rather than deaths. This would make the proper comparison 6 to 15 instead of 6 to 17.
- Furthermore, the holiday period was 11.6 days this year, but 9.6 last year. All things being equal, one would expect the road toll to be 21% higher for that reason alone. The best way to deal with this is to look at crashes per day, instead of just crashes.
- But more than either of these points, the biggest problem is trying to establish a trend with two data points. More data exists, why not look at it?
On the same Ministry of Transport page I linked to above is a table of historic Holiday Period road deaths (just scroll down a little). Plotting out fatal crashes per day, you get the following time series:
Looking at the data in its historical context, the recent number doesn’t look look all that scary. It’s a little on the high side, but still looks like part of the historical downward trend in crashes. This becomes more apparent if you apply a smoother to see how the 2014 figure compares to the historical trend:
The grey region represents a 95% confidence interval around the the trendline, which means that values within that range can be considered typical, given the downward trend. As you can see, the 2014 figure fits comfortably within the confidence interval range, suggesting that it is not anomalous. This makes attempts to explain the road toll figure largely redundant, there is nothing to explain.
There is a natural human tendency to jump to explanations when something happens, especially when that something is emotionally-charged as the deaths of 17 people. But the first rule of explaining a phenomenon is “confirm the phenomenon actually exists”, and that may be the real lesson to learn from this data.
Images created by the author using the ggplot2 and ggthemes packages in R.
The high…. Periodicity? That attribute of the measurement that makes the graph all spiky rather than smooth. Suggests that there is a lot of randomness in what’s being tracked.
But the downward trend over time — after smoothing. That is something. It’s safer cars, better drivers, public education about drinking over the holidays, maybe even less rainy summers because climate change. But something is causing average deaths to decline over time, even as year-to-year measurements remain highly volatile.Report
It’s weirder than spiky. There’s an almost perfect alternation of points above and below the trend line, That can’t possibly come from random data over 40 years– the odds would be about a trillion to one against. James, what the hell is going on?Report
@Mike-Schilling
It’s not quite as spiky as it looks, if you look closely you’ll see most of those spikes are made of multiple points. There are 44 points and 27 runs in the data, that’s a Z statistic of 1.22 on a runs test.Report
@james-k A scatter plot might make that a bit more clear. It’s a bit tough to make out individual points when they line up closely enough.Report
Off the top of my head, I’d guess that it correlates to the length of the Holiday Period. Sometimes it’ll be 12.6, sometimes it’ll be 8.6, right? We’d expect mad fluctuation.Report
I like Jaybird’s question regarding the length of the measured period.
As for the long-term trend, I’d ask how that compares to the long-term trend over the same period for deaths per million passenger miles. I suspect that the lesson is that seat belts, airbags, better-designed crumple zones, unibody construction with much stronger passenger compartment construction, anti-lock brakes and ubiquitous steel-belted radial tires all matter at the margin.Report
I suspect that the lesson is that seat belts, airbags, better-designed crumple zones, unibody construction with much stronger passenger compartment construction, anti-lock brakes and ubiquitous steel-belted radial tires all matter at the margin.
From what I understand, that’s not the case. People (at the margins) say “Oh, I have these additional safeguards? That means that I can drive with my butt!” (or similar) and they end up with the same safety numbers they had before the improvement went in.
I seem to recall reading that, at such, the best safety feature we could install in every car was a spearpoint in the middle of the steering wheel, pointing directly at the driver’s heart.Report
@jaybird The metric is deaths per day. There’s no particular reason to expect it to correlate strongly with the number of days. Maybe it does, but it’s not obvious why.
Also, the compensation for increased safety that you describe is called the Peltzman effect.Report
@brandon-berg There’s no particular reason to expect it to correlate strongly with the number of days. Maybe it does, but it’s not obvious why.
The longer it is until you have to go back to work and the more figurative “weekend” you have in front of you, the more drinking gets done. On a normal week, a Friday night does not equal a Sunday night, I wouldn’t think. A holiday period with more effective “Fridays” and fewer “Sundays” could be a factor, or do I misunderstand you?Report
Maybe it’s the result of social hysteria (or the lack thereof)? “HEY! BE CAREFUL! LAST YEAR WE HAD A KABILLION DEATHS!!!!” morphs into “Hey, thanks for coming over and drinking with us tonight!” and back and forth.Report
@jaybird , while there may be a Peltzman effect, it’s been offset by something. The long-term trend in the US (which is easier to find than detailed data about New Zealand) is that number of accidents, number of injury accidents, and number of fatal accidents are all in decline. Those are declining absolute numbers, independent of the increasing number of drivers and vehicle miles driven. US fatalities and injuries are down 25% over the last 25 years, total accidents are down 19%. We made the cars safer, the marginal driver becomes more careless/aggressive,… accident count goes down? Something goes in those dots.
Maybe it’s as simple as increasing congestion and the population shift from rural to urban/suburban: for more drivers, it’s simply not feasible to drive enough faster or more carelessly to offset the vehicle improvements. Fatality rates per million vehicle miles are much higher in rural states, where it’s easier to go fast, than in urban ones.
For what it’s worth, New Zealand’s annual road toll for the last 25 years shows a decline that looks almost exactly like the smoothed holiday decline in James’s chart.Report
Note also that I wasn’t using marginal in the sense of marginal driver, I was using it in the sense of marginal accident. Perhaps it would have been better to say that improved auto engineering has shifted the characteristics of the marginally-survivable accident.Report
@burt-likko
I agree that there’s a downward trend, but for some reason that doesn’t make the news.Report
@james-k
Good news doesn’t “sell”.Report
@damon
I don’t think it’s that, when the road toll is low, it makes the news. I think @brandon-berg is right about it being more about a “trend that is not a trend”, journalists paying too much attention to the last couple of points and not enough to the wider context.Report
@james-k
I agree, in part. But also, sometimes you have to take information presented one way and change it so it’s more “sexy” and draws attention….ie sell.Report
Nice post.
That was also my first reaction on seeing the news stories: low frequency count data is jumpy.
Then I remembered to adjust for expectations: they’d just cut the blood alcohol limit for adult drivers from .08 to .05 while claiming it would save a pile of lives. This is the first holiday season under that new regime. Second, they cut the speeding tolerance down from the 5% margin allowed during prior holidays to 0%. So where last year you’d be ticketed for doing 106kph in a 100, this year you’d be ticketed for 101.
So I started thinking about mechanisms for getting an increased (relative to last year) road toll despite the policy shifts. One plausible one seemed to be that maybe the zero speed tolerance limit led to more dangerous overtaking. Didn’t seem nuts, fit my intuitions, but when I emailed the MoT’s stats guy, he said only 1 of the accidents involved overtaking.
Noisy data is noisy still seems best explanation, but it is … odd. Had things dropped, the cops immediately would have been taking credit for their wonderfulness in pushing for a .05 limit and for the zero-tolerance speed policy, even had it been a noisy data thing. Maybe the weather was better this year and more people were on the roads.Report
That’s a very good point, actually – when implementing measures to control inherently noisy and low-frequency data, there seems to be a very pronounced human tendency to look for conclusions about the effectiveness of the measure, before any real trends could conceivably emerge from the noise.
Whoever’s favoured argument looks supported by what’s probably 95% noise at that point, will proclaim themselves right because science, while whoever’s argument looks undermined will say that the data are noisy and you can’t tell anything by such a small sample. The ones pointing out that you can’t draw any conclusions yet will be right, but most people will listen to the ones saying you can, because it makes for better editorial writing.
So the conclusion for policy makers is, the impaired driving and speeding changes were poorly timed. They should have done the analysis you did, found the trend line, and waited for a year that was significantly above it to make the changes (it wouldn’t have taken more than a few years)Report
If the number went down, the measure worked and should be done twice as much next year.
If the number went up, the measure wasn’t strong enough and should be done twice as much next year.Report
Now is not the time to raise taxes!Report
It’s always time to raise taxes!Report
I am of the opinion that road deaths is the wrong metric for measuring road safety.
Although it is easy to collect the data and makes for a better “juicy” headline.
There are so many variables that impact a death rather than an injury.
From the number of people in the crash thru to the rapid response(read helicopter) to get the injured to Hospital and a full medical response.
Surely a better safety performance metric would be a number of crashes over the period(reduced to a crashes per day measurement)
Ministry of transport extrapolate this from their crash database.
http://www.transport.govt.nz/research/roadtoll/christmasnewyearholidayperiod/
I have asked the Ministry for the last holiday numbers but they referred me to Police as they wont have them the numbers for up to 3 months.
As a separate exercise I wondered about the effect of “clamping down” on speed. My assumption of increased speeding ticket should somehow correlate with a reduction in crashes does not stand up to the data.
Luckily the number of speeding tickets is available from the NZ Yearbook(unfortunately only until 2012)
http://www.stats.govt.nz/browse_for_stats/snapshots-of-nz/yearbook.aspx
Trying to gather some meaning, I ran the annual number of speeding tickets issued, against the number of crashes. All I gleaned was that number of tickets per crash went up 60% from 2009 until 2012 which sort of leads me to the idea that issuing speeding tickets does not reduce the number of crashes….Report
@roy-edwards
That’s a fair point, but one of the problems with using all crashes is that the seriousness of crashes matters too, and can be influenced by transport policy.Report
Beautifully clear explanation, James. I’m going to have my methods students read this.Report
Thanks @james-hanleyReport
I second that. I had a discussion about absolute numbers v frequency on Facebook the other week and this is a great primer with real data.Report
Tangential New Zealand question: I was looking at GDP figures and noticed that New Zealand is fairly poor by Anglosphere standards. Italy poor. Australia, the other non-Chinese Anglosphere island nation in the region, is richer by about a third. Is there any consensus on why that is? I’m guessing that it’s probably some combination of small population limiting internal division of labor, lack of the natural resources that fed the Australian boom, remote location driving up prices, and a legacy of extremely high government spending prior to the mid ’90s, which is about when it started catching up. Does that sound about right?Report
From what I understand, attritional war with orcs has really stunted the economy.Report
No kidding. I saw the documentary about how productive those orcs were. They had that industrial revolution thing going full swing, and when they lost the war the whole place went back to agrarianism.Report
The orcs only have themselves to blame. Trying to ban pipe-weed (even for medicinal purposes!) really turned the native Halfling population against the occupation of the Shire.Report
@brandon-berg
I think the combination of our isolation and the large Australian mining industry is responsible for most of the difference – Australia has a lot of mineral wealth that, unlike a lot of ours, is sitting under land that no one cares about. Our valuable minerals tend to be located under lush bushland with conservation value.
Oh, just for the record, Australia isn’t an island nation – mainland Australia is a continent.Report
If it’s so continent, why is it leaking oil?Report
Really, guys? Nothing? Can I get a rimshot? Sad trombone?Report
Sometimes the responses flow freely, sometimes it’s already all wrapped up. It depends.Report
Your taxman is brutal over there. Our toll is only $2.50.Report
Warren Meyer over at Coyote Blog calls this media phenomenon a Trend That Is Not a Trend.Report
How does this compare to the rate during other, non-holiday periods?Report
As I noted in a comment above, New Zealand’s annual fatalities over the last 25 years shows the same kind of decline that James’s smoothed graph shows.Report