Forecasting the Outcomes of Football Bowl Games Using the ap poll
Download 57.14 Kb.
|
- Navigate this page:
- Appendix MATLAB Program of Algorithm
| Forecasting the Outcomes of Football Bowl Games Using the AP Poll
Caroline E. Meyer, P. Trent Teague, Dr. Mitch Parry RET Program, The Department of Computer Science, Appalachian State University Abstract – This study analyzes the forecasting ability of the Associated Press (AP) poll that ranks college football teams in the Football Bowl Subdivision at two points during the season; the preseason and the start of the postseason. Data mining strategies were utilized to establish which poll serves as the better predictor of the outcomes of these bowl matchups by analyzing all bowl games from 1978 to 2012. An algorithm was developed that sought to provide a maximum rate of prediction success utilizing data from both polls. It was determined that this algorithm predicts better than the individual polls themselves. Key Words-Data mining, college football, sports history, modeling 1. INTRODUCTION A recent analysis by Ken Pomeroy, a noted college basketball statistician, focused on the ability of preseason polls to predict the eventual seeding of teams in the NCAA Men’s Basketball Tournament. Pomeroy’s analysis showed that the higher ranked a team is in the preseason, the higher their chances of postseason success. He notes that over the course of the 64-team tournament era, the top-ranked team in the postseason received a top seed in the tournament 68% of the time, a strong correlation [1]. Pomeroy also notes that “the preseason #1 team has made it to the title game a total of 10 times” [2]. A recent indicator of this situation occurred in the 2014 NCAA Basketball tournament; the University of Kentucky, the preseason top-ranked team, entered the tournament unranked in the Associated Press (AP) poll. However, Kentucky finished as the runner-up in the tournament, losing in the championship game. There have been similar studies into the AP poll’s ability to predict the outcomes of games in college football. In a study by Lebovic and Sigelman, there were similar correlations between placement in the AP rankings and on the field performance. For example, the authors note that for games in which both teams are ranked in the top ten, the higher ranked team wins more than two-thirds of the time, provided it is ranked more than two places above its opponent [3]. However, there has been very little research into the AP poll’s ability to predict the outcome of the postseason in college football--bowl games. This paper will seek to answer the question of whether the preseason AP poll or the postseason poll (referred to as the “prebowl poll” in this analysis) provides a better prediction for postseason success in college football bowl games by examining all bowl matchups from 1978-2012. This paper will work to develop a model that will combine the forecasting potentials of both the preseason and the prebowl polls. 2. METHODOLOGY Before data collection could begin, the nature of the data to be gathered had to be identified. It was clear that the focus would rest on the results of postseason bowl matchups. However, the question remained as to what range of years should be considered. A natural place to start seemed to be the 1977-1978 season, chosen as the beginning of the modern era in college football. This season that saw the split of Division I into two subdivisions: Division I-A, now more commonly known as the Football Bowl Subdivision (FBS) and Division I-AA, now known as the Football Championship Subdivision (FCS). Data were then collected for all bowl games played from 1978 to 2012 including teams matched up, the score of each game, which teams were home and visitor, and the betting line [4]. Ranking data was also collected for each school playing in a bowl game, specifically, preseason ranking and prebowl ranking [5]. This data was imported into MATLAB, which was used solely throughout this analysis. 3. RESULTS Out of 792 games analyzed, the home team won 416 games while the visiting team won 370, despite all games being held at a neutral site. Six games resulted in a tie; for the purposes of analysis, each tie was treated as a half-victory for both the home and visiting teams. It was determined through a chi-square analysis that there was no statistical advantage given to the home team (p-value = 0.2478) [6]. Because bowl games are played at a neutral site, this finding supports the idea that neither team will have a home field advantage. Figure 1. Distribution of victories for all bowl games between 1978-2012. The 212 instances in which neither team was ranked in either poll were excluded from the analysis, as this study focused only on games that included at least one ranked team. A series of parameters were established to identify games in which the higher-ranked team won its game, as well as those games in which the higher-ranked team was upset. The following formula was used to establish those parameters: A = (visitor score - home score) (visitor ranking - home ranking) If A< 0, the higher ranked team won, meaning that the poll correctly predicted the outcome of the game. If A > 0, the lower ranked team won, meaning that the poll did not correctly predict the outcome of the game. Ties were distributed as half-victories for each. All games that involved at least one team ranked in either the preseason or prebowl poll were included in this analysis (580 out of 792). Matchups with unranked teams in either poll were divided evenly between the “Correct” category and the “Incorrect” category. When inserting preseason poll data into this function and performing the above corrections, the following results were obtained: Table 1. Observed and Expected Outcomes in the Preseason AP Poll
It was hypothesized that the preseason poll would correctly predict the outcome of each game 50% of the time. A chi-square test was then performed, with this assumption serving as the null hypothesis. Comparing this expected outcome with the observed data generated a chi-square value of 7.948, and a p-value of 0.0048. In other words, these results could be explained by random chance in less than 0.5% of all instances. Thus, the null hypothesis was rejected, indicating that the preseason poll could predict bowl outcomes with a greater than 50% success rate. Overall, the preseason poll had a success rate of 58.3%. The process was repeated with the prebowl poll data to determine if it too would have a prediction success rate different than 50%. Table 2. Observed and Expected Outcomes in the Prebowl AP Poll
A chi-square analysis was performed with the data in Table 2, with a null hypothesis that the prebowl poll would predict each bowl game outcome with a 50% accuracy. With a chi-square value of 10.0552 and a p-value of 0.0015, the null hypothesis was rejected. Thus, the prebowl polls had a success rate that was greater than 50% (59.3%). With these results in mind, the analysis turned to establishing the possibility that one of the polls has a better rate of prediction. The 580 game dataset (in which at least one team was ranked in either poll) was subdivided into categories defined by the success or failure of both polls. Table 3. Observed Outcomes for all Games with at Least One Ranked Team
The resulting output included games in which one or both teams were unranked at any point in the analysis. In an effort to include these results in the final analysis, these games were evenly distributed as a half-win and a half-loss to each poll. The final results are reported in Table 4: Table 4. Modified Observed Outcomes
Expected values were computed based on the observed data that assumed each poll would behave independently of the other in its predictions [7]. Table 5. Expected Outcomes
A chi-square test was performed with a null hypothesis stating that the preseason poll and the prebowl poll predict the outcomes of bowl games equally well. For this analysis, the chi-square value was 2.138 (1 degree of freedom) and the p-value was 0.1437, indicating that this null hypothesis could not be refuted. It is notable in these results that despite being shown to be seemingly independent of each other’s results, both polls predict the outcome of bowl games with a similar degree of accuracy. What makes the two polls different? What kinds of measurements do they use? With this in mind, the research turned to answering the question of what scenarios worked best for each poll. During the data compilation, it was noted that there were numerous teams that started ranked in the preseason, only to enter the postseason unranked. If voters had such high expectations for these teams going into the season and these expectations were not met, what chances do those teams have at postseason success? An analysis was conducted to determine the rate of success for both polls in predicting the outcomes of those bowl games in which at least one team entered the game unranked after starting the season ranked in the preseason AP poll. 
Figure 2. Prediction success rate for both polls in matchups where at least one team began the season ranked and entered the postseason unranked. Figure 2 presents a striking trend. In almost every instance, the preseason poll serves as a better predictor of bowl results than its prebowl counterpart, provided that at least one team entered the bowl unranked despite starting the season ranked. The lone exceptions were for those teams that began the season ranked in the top two. However, this was the result of a very small sample set for both positions. As more ranking positions are incorporated into the sample set, the average rate of success for each poll begins to approach each other. When evaluated separately, the preseason poll had an overall prediction success rate of 63.42%, while the prebowl poll had a prediction success rate of 53.02%. When this chart was analyzed, it became evident that the preseason poll had the advantage under these given circumstances and was thus a better predictor of the outcomes of those games. With this understood, the focus now turned to examining scenarios in which the prebowl poll would be the more successful poll. A study conducted by Lebovic and Sigelman in 2001 suggests a trend in games between teams ranked 11th through 25th in the AP poll at any given week. While one would normally conclude the higher ranked team would win such a game, the authors’ research indicated that the better ranked team will tend to lose the game instead [3]. To test this idea, an analysis was performed on the historical data to ascertain each poll’s predictive success for all bowl games between 1978 and 2012 in which both teams were ranked between 10 and 25 at the time of the game. The analysis showed that the preseason poll would predict the outcome correctly 58.80% of the time while the prebowl poll would be correct 59.26% of the time. Various scenarios were tested to see what would maximize the predictive success of both polls. The scenarios included:
Table 6. Prediction Success Rate for the Preseason AP Poll and the Prebowl AP Poll Under Various Conditions
With the different optimal conditions for both polls, a predictor model for the historical data set was created that included each poll’s predictive success in bowl matchups. An algorithm was developed in MATLAB that used these optimal conditions to maximize the forecasting power of both the preseason and prebowl polls. This algorithm operates under these conditions (when “poll’s prediction” is used, it is assumed that the higher ranked team would win the bowl game):
The original MATLAB program that outlines this algorithm is presented in the appendix. With these conditions set, the model operated on a 59.5% success rate when predicting the outcomes of all bowl games between 1978 and 2012.
To better understand the model’s rate of success and its significance, a similar analysis was performed on the point spread of all bowl games in the time period analyzed. The point spread is the betting line that informs gamblers of the likelihood a team will win a matchup by a specified margin. For instance, in the 2012 BCS National Championship, the point spread was -9.5 in favor of the University of Alabama, meaning that the Crimson Tide was predicted to win the game by at least that margin. The point spread’s rate of success at predicting bowl matchups was 62.2%, a difference of 2.7% from the model. In comparison, the rate of predictive success for the preseason poll among all matchups eligible for analysis was 58.29% and was 59.31% for the postseason. The model created in this analysis had a slightly higher rate of success than both polls, but was slightly lower than the rate of success of the point spread. It can be assumed that a much greater deal of analysis goes into determining the point spread of a game, with many more variables taken into account. The model presented in this study utilized only two variables--the preseason poll and the prebowl poll. With this in mind, it becomes all the more remarkable that the difference in prediction accuracy between these two models amounts to only a few percentage points. However, it must be kept in mind that this model relies upon historical trends and thus may not hold true for future predictions. Before the completion of this study, it was assumed that the poll released immediately before the bowl games began would be the best predictor on the games’ outcomes, as it compiles data that takes into account the successes and events of the season as a whole. This poll acts as a “snapshot” of all teams at the end of the regular season and reflects a team’s state at that moment. However, the results of the study showed only the slightest of advantages for this prebowl poll overall. In fact, there were numerous instances in which the preseason poll was the more logical choice. There are possible explanations for this. This poll, released before the start of the season, ranks teams based on assumed talent and experience, and also takes into account such variables as personnel changes. The preseason poll, then, may provide a more detailed portrait of a team. And because bowl games are designed to create matchups of teams on an even level, the preseason poll gives the most realistic outlook for the potential of each team in these games. Further avenues of study include the application of this study to other sports in college athletics. The anecdote noted at the beginning of this study hints at the potential for similar conclusions to be drawn from analyzing college basketball polls and their relation to postseason success in the NCAA tournament. Other areas of potential research include college baseball, softball, women’s basketball, etc. Further study could also be applied to the FCS subdivision in Division I college football, comparing media poll rankings with success in the FCS tournament. 5. ACKNOWLEDGEMENTS This research opportunity was made possible by the National Science Foundation’s RET Program. The authors would like to thank Dr. Rahman Tashakkori and the Department of Computer Science at Appalachian State University for their support in this program. The authors would also like to thank Lincolnton High School in Lincolnton, NC for continued support of higher education and research opportunities. 6. REFERENCES [1] Pomeroy, K. (2012, October 21). More fun with the ap poll [Web log message]. Retrieved from http://kenpom.com/blog/index.php/weblog/entry/more_fun_with_the_preseason_ap_poll [2] Pomeroy, K. (2010, November 1). The preseason ap poll is great [Web log message]. Retrieved from http://kenpom.com/blog/index.php/weblog/entry/the_pre-season_ap_poll_is_great [3] Lebovic , James H., and Lee Sigelman. "The forecasting accuracy and determinants of football rankings." International Journal of Forecasting : 105-120. Web. 10 July 2014. [4] "Downloadable CSV and XML Files." Sunshine Forecast Downloadable Data Files. N.p., 30 Dec. 2013. Web. 8 July 2014. [5] "Football." - College Poll Archive. N.p., n.d. Web. 8 July 2014. [6] Light, C. (2008, August 1). Tutorial: Pearson's Chi-square Test for Independence. . Retrieved July 22, 2014, from http://www.ling.upenn.edu/~clight/chisquared.htm [7] Math Forum - Ask Dr. Math. (n.d.). Math Forum - Ask Dr. Math. Retrieved July 21, 2014, from http://mathforum.org/library/drmath/view/52756.html About the Authors: Caroline E. Meyer Caroline currently teaches Biology at Lincolnton High School. She is a North Carolina Teaching Fellow and graduate of North Carolina State University, c/o 2011 (BS Secondary Science Education). She is a member of the National Science Teachers Association and serves on several committees within the school community. She is very grateful for the opportunity to participate in the RET Program at Appalachian State University and is looking forward to implementing the strategies and skills she has learned in her own classroom. P. Trent Teague Trent currently teaches Advanced Placement Chemistry and Chemistry Honors at Lincolnton High School. He graduated from the University of North Carolina at Chapel Hill in 2012 with a B.S. degree in Biology and a minor in Chemistry. He is a member of the National Science Teachers Association (NSTA). He hopes that his work in the RET program this year will broaden both the applications and the appeal of computer science in all disciplines in the educational community of North Carolina. Mitch Parry Mitch joined the computer science faculty at Appalachian State University in 2012 after spending five years as a postdoc in the Biomedical Engineering Department at Emory University and Georgia Tech. His work focuses on separating underlying source signals from mixed sensor recordings. For example, audio signals contain a mixture of source signals such as voices or musical instruments; multispectral quantum dot images contain mixed emissions from multiple quantum dots; mass spectrometry tissue images contain the mixed contribution of multiple tissue types; and surface enhanced Raman spectroscopy contains the mixed contribution of multiple materials. His work helps to untangle the contributions of multiple effects to aid in visualization, quantification, and prediction. He is particularly interested in working on interdisciplinary projects and collaborating with researchers in the arts & sciences, and welcomes the opportunity to work with students on research projects. Mitch has worked with students from computer science and other disciplines to apply computational techniques to problems in music, chemistry, biology, and biomedical informatics. As a student, Mitch attended Georgia Institute of Technology as a member of the Computational Perception Laboratory, where he received a Ph.D. in computer science in 2007. He completed his undergraduate work at the University of Virginia, where he received a B.S. in computer science with a minor in electrical engineering in 2000. Appendix
Directory: ret ret -> Retro Game Programming Copyright 2011 by brainycode com Retro Game Programming ret -> 18 department of administrative and financial services ret -> Homebrewing computers in the 21 ret -> Curriculum vitae name: Nurettin Ak nationality: German employment history ret -> Fifth version of a growing list of Delta Pilot Authors with the books they have published, and a new section below on books about delta air lines. Version #4 was dated April 10, 2005. Those books published in (2005) ret -> Wynstones Press, Ruskin Glass Centre Download 57.14 Kb. Share with your friends:
|