In a recent New York Times article the increase in the use of analytics in the NFL, former Ravens head coach Brian Billick joined the "head in the sand" crowd by asking the rhetorical question "How do you quantify, statistically, Ray Lewis?” and his answer, of course, was "You can't." Think for a minute what that says about how Billick sees football relative to every other endeavor humans have taken on. Man on the moon? Done. Heart transplants? Check. Connect millions of people by streaming video via a device you can carry in your shirt pocket? No problem. Measure the impact a middle linebacker has on a football game? Not so fast Einstein!
Any statistician worth their data sets however will tell you that it is certainly possible to quantify the impact that Ray Lewis has on a game, it just has not been done yet. But in fairness to Billick, his statement probably has more to do with his point of view on what it means to quantify something as opposed to a disbelief in the ingenuity of humans.
The common perception is that anytime a number is put down, it is a fact and the purpose of that number is to define a precise quantity or level. This point of view severely limits our ability to measure anything and under that definition, Billick may be right, it may in fact be impossible to quantify the exact impact that Ray Lewis has on a game. A more useful view of quantifying and measuring is that any information that we use whether it consists of notes from watching game film, scouting reports, or analysis of quantitative data, is to help make better decisions and reduce the risk of making a wrong decision. In this context, quantitative information is not a precise measurement, but rather an additional piece of information that can help inform a decision maker.
With this framework for the purpose of quantitative analysis, lets rephrase Billick's original question in a slightly more productive manner: How can we use quantitative information to gain a better understanding of the impact that Ray Lewis has on a game?
To understand how this type of thinking can enrich our understanding and decision making, consider a situation in which we had no quantitative information about Ray Lewis. All we know is what we see - which is extraordinarily impressive, but does not give us a full appreciation for Lewis and his performance. So now we can add some quantitative information, like tackles, and we can see that in seasons in which Lewis played all 16 games, he averaged 114 tackles per season. That by itself is not all that meaningful, because there is no context telling us how good that number is. In fact, it is really impressive. Since 1995, only 18 linebackers have had a season in which they recorded 114 or more tackles, and Lewis is one of only three (Derrick Brooks and Donnie Edwards being the other two) to do it multiple times.
Does that information provide a decision maker or fan with a precise measure of Lewis' impact? No, but it does help give a pretty clear signal that Lewis is, at the very least, one of the best tacklers of his generation of linebackers, which helps us have a better appreciation and understanding of how Lewis impacts a game.
Tackling is easy though, because we know what it is and it can be easily counted. Billick's comments are more directed towards areas of a player's game that are not already counted and understood, the "intangibles" or "leadership" or the ever popular "swagger". I won't pretend that these areas have been measured in any meaningful way, but that does not mean it can't be done, it's not rocket science after all.
There are two basic reasons for the lack of quantitative information around previously unmeasured "skills". The first is that no clear definition exists of the unmeasured skill. We hear words like leadership and swagger all the time, but ask five people to define what that means, even in the very strict context of Ray Lewis, you probably get five different answers. Instead of throwing up our hands and saying that this lack of agreement means it can't be done, we can actually define it clearly and openly, so that any interested party (coach, GM, fan), understands exactly how "leadership" is being thought of, and can propose alterations to the definition if need be.
One recent research study attempted just this by defining a team's ability to cooperate. While the study may have flaws and we probably don't all agree with the definition of cooperation or the conclusions of the study, it is a first attempt and quantifying an important "intangible" and understanding its impact on the game. The study helps provide some additional information on an important question for teams.
Once a team agrees on a definition for an important area of study, then we run into the second reason: the necessary data are not already collected. Not having data though is a solvable problem. Technology is improving the efficiency of data collection on a regular basis, and once a definition is created, the data needed often become obvious so it is just a matter of resources to collect it, and gain a deeper insight into leadership or swagger or anything else that is not currently measured.
So finally, I'm generally a pretty optimistic person and believe that there are many great challenges in the world today that will be solved, whether they be technological, medical, or analytical in nature. Most of the challenges that engineers and scientists put their minds to are eventually solved. The great mystery of Ray Lewis' impact on a football game probably does not rank very high a lot of lists, but my guess is that it, or something like it, rates high on the list of enough people that we will continually get more and more insight into that question.