Knowing How Science Works Will Keep Your Foot Out of Your Mouth

Look, you guys, I don’t understand what your problem is; we just talked about this a couple of weeks ago. There is no special, magical charm in football that you can’t find in any other sport or team activity, especially when you consider the inherent danger that is presented by banging your head into other dudes’ heads repeatedly for hours on end.

What’s that you say? That’s not really an issue after all?

For one thing, yes it is, and for another thing shame on Football Scoop for taking an absurdly myopic research study and misreporting on it just enough to give people the kind of misinformation and false hope that causes these debates to drag on unnecessarily.

The headline that Football Scoop chooses to go with is:

Study: There is no association between playing HS football and cognitive impairment or depression later in life

If you actually bother to read the details of the research study in question, a more accurate headline would be as follows:

Study: If you’re a male who graduated from a high school in Wisconsin in 1957, having played HS football is not an independent indicator of cognitive issues at age 65, but take it with a grain of salt since we gathered literally no other information

Obviously, one of those two headlines is much catchier than the other.

So Football Scoop reports that this study is a “big positive” for all those currently playing, aspiring to play, coaching, or aspiring to coach football, because it suggests a potential limit to the purported dangers that we’ve been hearing about ad nauseam for the past 5-10 years.

Here’s the thing; even if you disagree with all of the studies that essentially paint football as murderball, those folks aren’t pulling numbers out of thin air, so a study like this that flies so completely opposite of the prevailing wisdom should be cause for skepticism and further investigation.

It’s likely due to my graduate degree in physical therapy, which included spending weeks at a time elbows deep in research with the sole purpose of assessing the value of that research to the field of physical therapy.

Did they study enough people? How did they do it? What does the specific language of the study actually conclude? Based on all of that, what actual utility does this study have beyond “hey everyone, we did a thing”?

I appreciate that my perspective on this is perhaps unique compared to the average football fan, but that’s why I’m here to share the real truth. Football Scoop states the following:

The study looked at folks that played high school football in Wisconsin in the late 1950’s and after running tests, they concluded that there was no evidence to suggest that “playing football had a negative long-term association with cognitive functioning and mental health at 65 and 72 years of age.”

This is just factually incorrect. The study they are reporting on did not run a single test; as a matter of fact, the study itself was not really a study at all in the truest sense, in that there were no physical subjects present in a lab and being tested in any way. The researchers merely did a process that worked as such:

  1. Find a longitudinal study in the state of Wisconsin (that’s a study where they gather all sorts of data on you for essentially your entire life)
  2. Find all of the male subjects in that study who attended high school; pull their personal data out of the larger set
  3. Sort them into those that played football, those that played other sports, and those that played no sports in high school
  4. Gather all of the data from their charts that involved a cognitive test that was performed when they were roughly 65 years old
  5. Use that data to draw a conclusion

Process wise, this is no different than you or I spending some time cruising through college football historical data looking for patterns and talking about whatever we find. This study creates as many questions as it attempts to answer. Why Wisconsin? Why did you only report data on people who graduated high school in one specific year?

There are other gaps in the information that would take time but could be answered, such as whether the cognitive measures they gathered are actually good at assessing cognitive impairments, and whether it was a good idea (from a valuable data standpoint) to Frankenstein all those measures into a larger overall score.

Further still, you can’t actually draw any conclusion beyond saying that there does not appear to be any correlation between having played high school football in Wisconsin in the late 1950’s (1957 grads would have played from 1954-57) and cognitive impairment or depression, and even that statement is flimsy at best.

All these researchers did was grab a bunch of data on a bunch of people, determine what sport if any they played when they were in their teens, and then look at how they scored on a handful of cognitive assessments when they were 65. What about literally everything that did or didn’t happen to them in the 47 years in between? What about every assessment that they were given after the age of 65?

It’s equally likely that these high school football players in Wisconsin had some advantage that wasn’t present in Wisconsin in 1967, or in Arkansas in 1957, or some advantage between ages 18 and 65 that the study made no effort to capture. Maybe there’s no link at age 65, but there’s a massive one at age 67.

All of these very valid questions prove the point I’m trying to make, which is that this data mining adventure masquerading as a scientific study proves nothing other than that this exercise would probably be worthwhile to attempt on a much larger scale.

The study discussion at the end even says “Among men who graduated from Wisconsin high schools in 1957, playing high school football was not adversely associated with cognitive impairment or depression later in life…our findings are consistent with those of Savica et al,14 who found no differences in the incidence rates of neurodegenerative disease among football players and their non–football-playing classmates in a contemporaneous cohort.”

First of all, it actually shows that it was not adversely associated with cognitive impairment or depression specifically at the age of 65, as I’ve already discussed.

Then they throw in another study that found essentially the same result, except the study they reference is even poorer, because they did the exact same research in one town in the state of Minnesota, where they had to gather data over a decade just to get 578 subjects.

Showing that someone else did the exact same data mining and found the exact same pattern only shows that the process is easy for someone to replicate – it falls far short of serving as proof of your hypothesis.

Football Scoop concludes with the following:

For coaches out there looking for research to share with parents, players, and the community, this is a promising scientific study. The game has changed a ton since the 1950’s and coaches today are more aware of the dangers and concerns now than ever before, and are taking steps to make the game even safer with proper tackling, hydration, concussion protocols, and all sorts of other things.

Again, no. I’d love to meet these football coaches who are really geeking out about the opportunity to share research with the parents of their players. It is certainly true that coaches today are more aware of concerns, and are doing a lot more to make the game safer. But that fact, and the flimsy data presented here, do absolutely nothing to dispel the perception of football as a dangerous game anywhere other than 1950’s Wisconsin.

Good news, Wisconsin males in your late 70’s, that dementia you’ve got might not be from football! Aren’t you relieved?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s