Project_Design_Writeup.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Beat the Streak? - Project Design Writeup - Matthew McNew \n",
    "\n",
    "### Project Problem and Hypothesis\n",
    "MLB.com offers an online game titled, Beat the Streak. In the game players can attempt to beat Joe Dimaggio's hitting streak by picking a player that gets a hit in 56 consecutive games. Players are allowed to pick two players for each day and even pass on picking on a day. Since the game's inception in 2001, no online player has managed to beat the streak and win the grand prize of $5,600,000. \n",
    "\n",
    "This project will attempt to model beat the streak as a classification problem. The machine learning models will attempt to predict which players have a high likelihood of getting a hit for a given day. Due to the importance of picking correctly, the machine learning models will be optimized for precision. Ultimately, the model will suggest one or a few batters out of hundreds as suggested batters for a day.\n",
    "\n",
    "Even with a very accurate model the probability of actually beating the streak is very low. The realistic goal of the project will be to see if the machine learning models can outperform baseball experts. If the models are relatively useful or accurate an online website will be set up that provides daily beat the streak recommendations. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Datasets\n",
    "This project will be using retrosheet’s datasets from 2015, 2014, and 2013. The free and open source baseball analysis tool, [Chadwick](http://chadwick.sourceforge.net/doc/index.html), will be used to load the retrosheet data into easily parsable CSVs. \n",
    "\n",
    "The retrosheet event data includes the data for all at bats in every game of the entire season. A complete data dictionary for this dataset is provided on Chadwick's website: http://chadwick.sourceforge.net/doc/cwevent.html. \n",
    "\n",
    "The retrosheet data also includes the data for all games in the entire season. A complete data dictionary for this data is provided on Chadwick's website: http://chadwick.sourceforge.net/doc/cwgame.html\n",
    "\n",
    "The online [Lahman's database](http://www.seanlahman.com/baseball-archive/statistics/) includes statistics about each players performance in each season. A data dicitionary is available on the Lahman's website: http://seanlahman.com/files/database/readme2014.txt. \n",
    "\n",
    "To extract meaningful features for machine learning, data from each of these datasets needs to be combined to create possible Beat the Streak choices. This data will be combined from the retrosheet event data, retrosheet game data and Lahman's database. For example, an initial dataset for the project's machine learning algorithm might look like this:\n",
    "\n",
    "Variable |  Source\n",
    "---| ---\n",
    "Player Name | Retrosheet event data\n",
    "Player Batting Average in the last 5 at bats | Retrosheet event data\n",
    "Ball Park of Game | Retrosheet game data\n",
    "Temperature of Game | Retrosheet game data\n",
    "Player's Batting Average* | Lahman's Database \n",
    "Starting Pitcher's ERA** | Lahman's Database \n",
    "\n",
    "\\* Batting Average (BA) is defined by the number of hits divided by at bats.\n",
    "\n",
    "\n",
    "\\*\\* ERA (earned run average) is the mean of earned runs given up by a pitcher per nine innings pitched\n",
    "\n",
    "\n",
    "### Domain knowledge\n",
    "\n",
    "As a casual baseball fan, I have a basic understanding of baseball statistics. I have heard plenty of baseball folklore about what effects batters. A large part of this project will be investigating which parts of baseball folklore is useful when predicting which batters might get a hit.\n",
    "\n",
    "This project will definitely not be the first attempt to use machine learning to play beat the streak. Students taking an Artifical Intelligence course at Stanford published [a paper on their research](http://cs229.stanford.edu/proj2013/Goodman-BeatingTheStreak.pdf) and students at Northwestern published [their own paper]((http://peterhad313.github.io/beat_the_streak_site/) on an attempt to play beat the streak with machine learning.\n",
    "\n",
    "Both of these previous works produced moderately accurate predictions using very similar techniques. Each day and player combination were reduced to a single feature vector. This feature vector includes data about the game such as game location, starting pitcher, and temperature as well as data about the current performance of the batter in recent games. These feature vectors were inputs to a logistic regression or decision tree model. \n",
    "\n",
    "\n",
    "### Project Concerns\n",
    "\n",
    "As mentioned in the problem and hypothesis, even with a very accurate model the probability of actually beating the streak is astronomically low. It will be important to approach the problem with realistic expectations. \n",
    "\n",
    "Due to abundant amount of data available, performing dimensionality reduction will be a major challenge. Across all the different variables it will be difficult to isolate variables that affect the probability of getting a hit. \n",
    "\n",
    "Each batter is affected by input variables differently. At this point, it is unclear how to model these differences with batters. How should the model capture a player that is more likely to get a hit from left handed pitchers? Should each batter be a unique model? Should the batter's be clustered together with other similar batters and modeled together? Hopefully some of these questions can be explored in this project.  \n",
    "\n",
    "The model should be optimized for extreme precision. Out of all the hundreds of players who get a hit in any given day an accurate model will just need to choose one or two of these batters. This project will need to investigate which metrics will be useful in evaluating this desired performance. \n",
    "\n",
    "The biggest risk is that the project will be unable to wrangle the complexity of the dataset to generate accurate predictions.  \n",
    "\n",
    "Obviously, the cost of a inaccurate model is low. A poor model will just result in just another beat the streak failure. \n",
    "\n",
    "### Outcomes\n",
    "\n",
    "The ideal refined output for the model would be a short list of suggested batters with a high likelihood of getting a hit for a given day. A consumer of this output would be able to sanity check the suggested batters and then attempt to use them as picks in MLB's beat the streak. \n",
    "\n",
    "The typical variables analyzed when predicting hitters are: the hitter's batting average, starting pitcher's ERA, hitter's performance and the ballpark the game is in played in. It is quite likely that that this project finds these variables as strong predictors as well. \n",
    "\n",
    "The model developed for this project could be very complicated due to the all the possible features and the interactions between features. However, the most successful model does not necessarily need to be complicated. \n",
    "\n",
    "A successful model would beat batting average as a predictor for beat the streak choices. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}