This week the British Journal of Sports Medicine (BJSM) published the results of a randomized trial of headgear to reduce concussions in soccer players.
Here’s an example of the type of headgear being investigated, in case you’re curious:
Before I jump in to my criticisms, I do want to commend the authors for running a randomized trial on an important question. Preventing concussions is a matter of physics. It’s all about dissipating collision forces so less of it transmits to your skull and shakes your brain inside of it. Helmets in sports like football were designed for a totally different engineering task: preventing skull fractures, which they’re great at. And as big of a concern as concussions have become, we only know of one surefire way to prevent them: stop collisions that shake people’s heads. All that is to say rigorous studies on new technologies that claim to protect the brains of sports participants from concussions are extremely important.
I also want to be upfront about what I expected this study to show, because scientists almost always approach a topic with some preconceived notions; we need to acknowledge and embrace that. I expected headgear to have virtually no effect on concussions.
And lo and behold, the randomized trial showed just that! So why am I writing this critical post?
Well, it’s dumb luck that I looked at this study more closely even though it confirmed what I expected to see. Someone tweeted out the main results table from the study and something didn’t look right – in fact, something was mathematically impossible (SEE Irregularity #2 below). That led me to read the whole paper, and there are some substantial irregularities in the analysis I want to call everyone’s attention to.
The below points aren’t my only concerns with the study, but they’re the biggest ones to which I want to call everyone’s attention.