“Why Attack Football? Girls’ Soccer Has More Concussions!”: A Study in Bad Reasoning and Worse Stats

When I started in football injury epidemiology I focused on muscle, ligament, and bone issues. I shied away from brain injuries.

There were two reasons for this. First, I thought enough smart people were already working on brain injuries that my time was better spent elsewhere. Second, as a new entrant to the field I didn’t want to touch such a heated area. But since my article with Dr. Kathleen Bachynski estimating the prevalence of chronic traumatic encephalopathy (CTE) in NFL retirees appeared in Neurology last November I’m off the bench and into the game.

Seeing the backlash to those working to measure and quantify the short- and long-term impacts of brain trauma in football has been enlightening. Some of the points football’s “defenders” make have merit – we don’t know about the prevalence of CTE in high school or college players, for example, nor do we have a good handle on how many ex-players will show actual symptoms of CTE and other brain diseases rather than just the brain damage associated with them (although we do know that ex-NFL players die of neurodegenerative diseases such as Lou Gehrig’s disease at 3-4 times the rate of the general population).

But as the push to ban tackle football for kids 14 and younger gains steam, there’s one talking point from football’s “defenders” that is so callous in its logic and fallacious in its statistics that I feel compelled to call it out directly here.

 

Girls’ Soccer Has More Concussions, Why is No One Talking About Banning It!?

This argument is so bad we have to take it down in parts. First I’ll discuss its flawed logic and then its flawed statistics.

Logic

This is whataboutism at its finest. “Sure this thing is bad, but look at this other bad thing? What about that, huh? What do you have to say about that? HYPOCRITE!”

This is a weak, lazy, bad faith argument deployed strategically, often by powerful people, with the goal of stopping you from doing anything if you can’t fix everything. It is a common tool used by lobbyists and politicians to distract from their clients or donors who do bad things. You have to be on the lookout for it constantly as it can seem very persuasive, especially to good people who just want to help.

It turns out that in public health and many other fields we can walk and chew gum at the same time! Regulating full-contact football for children does not mean we can’t also recognize and address the fact that concussions are an issue in other sports, too. If anything, doing the former should help the latter, not hinder it.

Imagine another more obviously absurd example from environmental health: “Why is everybody so focused on lead, huh? What about arsenic? That’s poisonous, too.” The people making this argument are hoping that will lead you not to regulate either rather than both.

But maybe the people making the argument are really just invested in the brain health of young girls. How can we know they’re making this argument in bad faith? It’s simple: ask them whether they would support banning girls’ soccer for those younger than high school.

Statistics

Like all good lies, this one has a kernel of truth at its center that is then warped to the point of being unrecognizable.

The kernel: In sports played by both boys and girls, reported concussion rates are roughly 50-100% higher for girls. We’re not sure why – is it some sort of physical difference, or are girls just more culturally willing to report brain injuries? But it’s pretty clearly true. Football, however, is played (almost) exclusively by boys so this argument, even if true, is irrelevant.

Concussions also account for a larger percentage of injuries in girls’ soccer than in football. That does not, however, mean that there are more concussions in girls’ soccer than football – more on that below.

The warping: The above points are twisted in some motivated people’s minds to become that concussion rates are higher in some girls’ sports than football, and thus they are the real public health issue.

The truth: Below is a table of the annual number of concussions as well as the rate per 10,000 athlete-exposures (1 athlete participating in a game or practice is 1 athlete-exposure) for various men’s and women’s sports. The data come from two large nationwide sources for youth and college sports: High School Reporting Information Online (2017-18 report) and the NCAA’s Injury Surveillance System (two separate analyses from the 2009-10 to 2013-14 seasons and 2011-12 to 2014-15 seasons). I included data from two NCAA reports because the newer report did not include nationwide estimated concussion counts.

This is what happens when you actually dig into the data rather than parroting cheap talking points.

In high school, football has by far highest concussion rate at 9.70 per 10,000 AEs; second is girls’ soccer at 6.91 per 10,000 AEs, nearly 30% lower. Because of the number of children who play each sport, football is also responsible for nearly twice as many concussions each year as girls’ soccer (103,830 vs. 59,447).

In college, football (6.71-7.50 per 10,000 AEs) is essentially tied for the second-highest concussion rate. Wrestling leads the way (8.90-10.92 per 10,000 AEs). Football’s rates are in line with or a bit below those for men’s (7.40-7.91 per 10,000 AEs) and women’s ice hockey (7.50-7.80 per 10,000 AEs).

However, because more students play football than wrestle, football is responsible for by far the most collegiate sport concussions each year (3,417). Women’s soccer comes in second and is responsible for less than a third of football’s concussions (1,113).

Summary:

Football is either at (high school) or near (college) the top when it comes to concussion rates. Because of the number of people who play it, football causes far more concussions than any other sport among both high school and college athletes.

Stop using bad stats about girls’ soccer to distract from the concussion problem in football.

Article Review: Pitching a Complete Game and DL Risk

Despite the name of this blog being NFL injury analytics, since I’m now spending at least a third of my time working in baseball I’m going to start writing about that, too.

I wanted to start by discussing an article that came out a couple months ago in the Orthopedic Journal of Sports Medicine entitled “Relationship Between Pitching a Complete Game and Spending Time on the Disabled List for Major League Baseball Pitchers.

tl;dr: This article makes a claim that is completely unsupported by its analysis. Please don’t use it. Read on below for a deeper dive into (some of) the big reasons why.

Before I get into that, though, I just want to say that I’m only writing about this article in particular because it happened to slide across my Twitter timeline on a morning when I had some spare time and motivation to write about it. I do think the errors here are particularly egregious, but the same types of mistakes are extremely common in a lot of sports injury papers. I hate to feel like I’m singling out just one research group, so please understand that my criticisms will likely apply to a lot of what you read, not just these folks.

OK, onward!

Continue reading “Article Review: Pitching a Complete Game and DL Risk”

New Sports Performance Technologies: The 4 Key Questions Teams Need to Ask

I’ve had some thoughts on this topic for awhile, and then this article from a couple scientists with, I believe, the San Antonio Spurs came out in the Strength and Conditioning Journal. It does a great job at summarizing the key issues, but I wanted to try and distill it down to a 4-question framework that I use when evaluating a new sports performance technology for teams that I consult for.

So here are the four simple but broad questions that I ask with any new technology, in the order in which I ask them. I’ll use a single example of a hypothetical portable tool that measures forces acting on the shoulder joint of baseball pitchers to illustrate my four questions.

Continue reading “New Sports Performance Technologies: The 4 Key Questions Teams Need to Ask”

Just How Dangerous IS the NFL vs. Other Sports?

Turns out, yeah, pretty dangerous, actually.

As I’m working on my dissertation literature review I figured I’d put a small piece of it to good use for someone other than my committee. 4 people would normally read this, so I’m really hoping we can double that.

There is a wide-ranging belief that the NFL has the highest injury rates of the major North American sports, but by how much? Because game injury rates (rather than injury rates in practices) tend to be easier to calculate (and more available across studies) than those incorporating practices, we’ll focus exclusively on those.

We’ll need to introduce one concept to make the analyses below make sense: the idea of an “athlete-exposure” (AE). This is simply defined as 1 athlete participating in a practice or, in the case of this article, a game. Thus a single NFL game where every available player plays in the game would count for 92 athlete exposures – the 46 guys on the active game day roster on each of the two teams. All the injury rates below are presented per 1,000 AEs.

The studies compared below all have differences in how exactly injuries are defined (namely how much practice/competition time a player has to miss for something to count as an “injury”) and the years they covered, but in an effort to ensure high-quality data I’ve limited the studies to official league injury surveillance systems wherever possible.

Injury Rates Across Sports

Among the “Big 4” North American sports, the NFL has by far the highest game injury rate at 75.4 per 1,000 AEs. This means for every 1,000 players playing in a single game, 75.4 will suffer some sort of injury. So in a single game with 92 AEs, you might expect about 75.4 x 92/1,000 = ~7 injuries.

Compared to the other Big 4 North American sports, the NFL thus has roughly 4-5 times the in-game injury rate of the NBA, MLB, and NHL.

What about more international sports (MLS statistics weren’t readily available, so soccer goes here, stop complaining)? The NFL also has roughly 4 times the in-game injury rate of soccer and, interestingly, Australian rules football.

The only sport that comes close to touching the in-game injury rate of the NFL is rugby. The data are variable, but in general rugby seems to have either a similar or higher injury rate to the NFL. I wanted to look into concussions specifically in the NFL vs. rugby, but I’ve found some conflicting data and I want to do more research on this before saying anything.

I’ll spare you the gory details of the data sources unless you want to click below the jump, where I’ve included everything you should need to evaluate these on your own if you want!

UPDATE ~6 hours after original post:

I purposefully simplified or ignored quite a few issues in writing this post to try to communicate my main point, but some good questions from friends have come up and I want to briefly address a couple shortcomings of this analysis:

  1. This post only considers one possible way to measure injury rates: per player-game. The results would likely look substantially different if we used another, more precise denominator such as player-hours. The NHL numbers, in particular, would look higher relative to the NBA and MLB in such an analysis since hockey players on average play a lower proportion of each game than baseball or basketball players. So this is not the be-all end-all answer for which sport is the “most dangerous.”
  2. Along those same lines, although the risk of being hurt in any given NFL game is way higher athletes play way more games in the NBA, NHL, and MLB. So the differences across sports in an individual athlete’s risk of being injured in a season are going to be much less than the differences in injury rates I’ve shown here. Maybe I’ll take a stab at calculating risks in another post!

In a nutshell: this is just one way of measuring which sport is more “dangerous,” and it’s probably the way that casts the NFL in the most negative light. Injury rates per player-hour or 1-season risks would likely make football look less dramatically bad.

Continue reading “Just How Dangerous IS the NFL vs. Other Sports?”

Checking a Reported Drop in Starting QB Injuries in 2016: Definitions and Counts

I saw a remarkable stat reported in Pro Football Talk (PFT) this morning: that starting QBs missed only 35 games in 2016, vs. 76/77/59 in 2013-15, respectively. The claim is sourced to Peter King’s MMQB column this morning, but unfortunately I haven’t been able to find a source document for the actual numbers. Per King, the NFL’s Competition Committee views these data as evidence the new QB protection rules are having their intended effect.

Now that seemed like a huge drop to me. I did some quick calculations – Teddy Bridgewater (16 games) plus Tony Romo (10 games) plus Jay Cutler (11 games) alone gets us to 37 games missed by starting QBs in 2016. That’s already more than the number reported by King.

Epidemiologists spend a large chunk of our time just counting things. As it turns out, that’s not grade school math. It’s really, really hard, and a correct count relies on counting a.) the right things and b.) doing so in a consistent manner. So I wanted to go into my injury data, sourced from Pro-Football-Reference (pro-football-reference.com), and see if I could replicate the numbers reported by King (and, ostensibly, the NFL). Spoiler alert: I couldn’t really.

Continue reading “Checking a Reported Drop in Starting QB Injuries in 2016: Definitions and Counts”

The “Post-Probable” Injury Report Era: Full-Season Update

Hey, long time. Been awhile. How are the kids? Childish? That’s good.

I’ve been a bit distracted with side projects lately – buying a house, co-teaching a high-level statistics course, my dissertation…you know, little things – so sorry for not updating this blog that no one reads for a few months.

BUT! I’m back with a very exciting post: I’m updating my prior investigation into the effects of the NFL’s decision to remove “Probable” from its injury report this past season, now that we have a full season to see how teams adapted (the original analysis had only weeks 1-8). Let me tell you, it’s been miserable for NFL injury analysts and honestly…probably pretty much fine for everyone else.

Since my previous two posts lay out all the relevant background, methods, and data sources in detail, we’re gonna skip right to the results update!

Continue reading “The “Post-Probable” Injury Report Era: Full-Season Update”

Is it Really Too Soon to Judge Roberto Aguayo?

I threatened to do this occasionally in my introductory post, but steel yourselves for a football analytics post that has nothing to do with injuries. You have ESPN’s Bill Barnwell and my friend and colleague Daniel Adler to blame for this.

On Bill’s December 5th show the two were discussing Roberto Aguayo, the kicker for the Tampa Bay Buccaneers that the team drafted in the 2nd round this year. Due to his high draft position Aguayo’s struggles – he is currently just 15/22 on field goals in the NFL – have been highly publicized. I’m heavily paraphrasing, but they basically came to the conclusion that it’s obviously far too soon to judge whether Roberto Aguayo is, in fact, a good, bad, or mediocre kicker.

Now Bill and Daniel are super smart guys, but I wondered if the statistics would bear them out…

Continue reading “Is it Really Too Soon to Judge Roberto Aguayo?”