Data and Decisions
 -
RSS Follow

Recent Posts

So you want to work in sports analytics...
Preparing for a Career in Sports Analytics
Looking at Last Year's Rookie QB Class
Not My Ideas
Thinking and Reading about Analytics

Categories

49ers
analytics
analytics, intangibles, football, sports
business intelligence
career
decision making,
elite performance
Hot Hand
managing analytics
moneyball
NFL
perceptual cues
sports
students
Super Bowl
powered by

Analytics and Sports

Data and Decisions

The same scene plays itself out repeatedly in my classroom. I begin by asking my Sport Management students whether they believe that players can "hot" and that the phenomenon of the "hot hand" is real. Most if not all of the hands go up at this point. Then we spend time working through Gilovich, Vallone, and Tversky's seminal paper on the Hot Hand. The paper is just one of many that seeks to find evidence that the Hot Hand exists, and finds no support for the hypothesis. After working through the data and analysis in the paper I ask my students for a show of hands again on the belief in the existence in the Hot Hand. Most if not all of the hands that went up the first time, go up again.

At this point I congratulate my students for being skeptical and not just taking the results of one study as a fundamental truth. Then I confront them with the following question: "When was the last time that new data or information caused you to change your mind on a topic that was important to you?" Most students cannot remember that experience at all.

This speaks to what is always a recurring theme for analysts, which is that people love data that agrees with their point of view, but when the data challenges their belief, they dismiss it. This phenomenon can be rather demoralizing for those that spend their lives digging through data to find answers to difficult questions. It suggests, at first, that their only role is to provide more ammunition for a decision maker when their point of view is supported by the analysis - a rather trivial role when the initial goal was to help make better decisions.

A different perspective to take, however, is that this dismissal when the analysis does not agree with the decision makers gut, is actually an opportunity. It is an opportunity to get better at what you do.

Remember my students? Their story does not end with the inability to remember changing their minds. Instead we start to consider how much evidence they would need to change their mind on the subject of the Hot Hand. What does the study look like that would convince them that their belief is just that, a belief, and that there is a possibility that there is enough evidence to convince them that their belief is incorrect.

The process of finding out what is needed to change a decision makers mind is not simple. It is a significant challenge for the analyst, but asking questions that lead the analyst to that answer will make the analyst better. It will force the analyst to see dimensions to the question that they had not previously considered and push their own analysis to a more sophisticated level. One eye opening result of this process is that the analyst may end up changing their own mind and agreeing with the original position of the decision maker. Alternatively, they may put together such a solid case for their perspective, that the decision maker cannot ignore it any more.

Of course,the analyst may end up not being able to convince the decision maker, no matter what they do. That is of course incredibly frustrating, but I would challenge the analyst to not blame the decision maker in that case. Time sensitive decisions have to be made and analysts often do not have time to do all of the analysis they would like to or that they imagined would be necessary. When the analyst is pushed aside because they do not have such a clear and complete case that cannot be ignored, they have not adequately prepared for the situation that they found themselves in. This is a learning experience that every analyst will face. Instead of blaming the decision maker for not seeing the right answer, the analyst needs to go back through both their analysis and the presentation of their analysis to see how they could have been more complete and more persuasive.

In the end, a lot of my students are never convinced that the Hot Hand is just a trick of the human mind, but some are. Some see how humans can be fooled by random streaks and how coaches can take advantage of teams and players that get fooled by these streaks. With these students, I have succeeded and presented enough information to convince them. To others, some of whom are certain that they have experience the Hot Hand themselves, I have not presented enough information to overcome the evidence that they have lived. I keep trying though and someday, I hope to find the set of evidence that will convince them to change their minds.

1 Comment to Data and Decisions:

Comments RSS
Peter on Sunday, March 17, 2013 7:41 AM
Ben - I enjoyed the article and you've done a nice job describing a very real challenge, and as importantly, very real opportunities to improve as analysts. I hope other readers (and your fortunate students) will take this to heart. I know I have.
Reply to comment

Add a Comment

Your Name:
Email Address: (Required)
Website:
Comment:
Make your text bigger, bold, italic and more with HTML tags. We'll show you how.
Post Comment