Every year during the NFL draft, a lot of talk circulates about quarterback prospect being “consistent”, especially from week to week. Nobody wants a quarterback that they can’t rely on to put up good outings every week, so they look at how consistent that player was in college from week to week to determine if that player will be consistent in the NFL like Tom Brady and Alex Smith have been, as opposed to inconsistent in the NFL like Geno Smith and Andy Dalton have been.
Unfortunately, very few attempts have been made to quantify that consistency, and equally few have been made to see how consistency in college translates to consistency in the NFL. I set out to determine if consistency in college has any correlation to consistency in the NFL, and found out that it not only strongly correlates with NFL consistency, but to my surprise, almost equally strong with a high NFL passer rating.
Quantifying consistency
For my study, I looked at every FBS quarterback who was taken in the first or second round since the 2001 NFL draft, because that is how far back I could get college game logs from. The only requirements aside from draft slot, year, and college division were that the quarterback threw at least 200 passes in his final season of college and that they have started at least 3 NFL games, so that they provided me enough data to work with. This only eliminated a few guys, including Brian Brohm and Sam Bradford.
To measure consistency, I looked at the variance of the players’ passer ratings in their final year in college (ignoring any games against FCS opponents). The more inconsistent they were, the higher their variance. I did the same for their 2015 NFL season to determine pro consistency because I began this study during the 2016 season. I also gathered their passer ratings from their final years in college, and their passer ratings from their entire careers as pros. Variance values in college were higher because the college passer rating scale yields higher values, as well as a larger range,
The Dalton Dilemma
Of the 49 quarterbacks involved in my study, I had one outlier. As you can probably guess from the title of this section, that quarterback was Andy Dalton. While I will get to the findings in the next section, here is the data when you include Andy Dalton.
The Findings
As I said in the beginning, the original purpose of this study was determine the strength of the relationship (if any) between consistency in college and consistency in the NFL. My finding was that the short answer is “yes”, and the slightly long answer is “hell yes!”
Including Andy Dalton (I’ll say when he begins to be excluded), there is a correlation of .610 between variance in college and variance in the pros. This is so absurdly high, that for comparison, the correlation between passer rating in college and passer rating in the pros is a mere .161. The statistical significance of there being a positive relationship between college and pro variance (also known as the P value) is 1.07915 * 10-5, which again is a ridiculously strong value (most scientists only look for any P value below .05).
The Surprising Result
I had a pretty strong suspicion that consistency in college was going to correlate with consistency in the pros, and vice-versa. What I had no clue would happen was that consistency in college would so strongly correlate with higher passer ratings in the pros.
As I mentioned in “The Dalton Dilemma”, the correlation between variance in college and passer rating when you exclude Andy Dalton from the data is -.543. Remember that the correlation between college passer rating and pro passer rating is only .161, just a fraction of the strength of the variance to PR correlation. The P value for the positive relationship between variance and pro PR was 6.67852 *10-5, meaning that again, the results were very statistically significant.
The First Model
Once I had my conclusion, I decided to build a model for future NFL quarterbacks and the quarterbacks taken in this past draft. First I decided to look at the best trend between just college variance and NFL passer rating, which was an exponential trend line that looked like this:
Model #2
I decided to try one more model, this time using a multiple regression of college passer rating AND college variance. This was what I got:
Last Year’s Rookies
I decided to use the model on some of last year’s rookie quarterbacks and got mixed results
Other interesting results were the correct identification of Dak Prescott by both models, the differing results for Christian Hackenberg and the mediocre results for Jared Goff. Unfortunately, Carson Wentz did not play in the FBS, so he wasn’t run through. Keep in mind that while I tested this on later round prospects, it was made with and is therefore technically meant to be used on first and second round picks only.
This Year’s Prospects
This is probably the moment you’ve all been waiting for (at least those who are still reading) or skipped ahead to. Of course I had to run this year’s top QB prospects through the system. Here’s what I got:
Why The Trend?
The correlation between inconsistency in college and higher NFL passer seems counter-intuitive, especially given that it is stronger than the one between college and pro PR. A possible explanation is that consistency between games indicates work ethic and taking every game seriously not matter the strength of competition. The work ethic and psychological indication theory shows through in players like Johnny Manziel and Jamarcus Russell, who had high variances in college and then poor work ethics in the pros. Another possibility is that consistency in college indicates that a player doesn’t let success get to his head, which points back to work ethic. Finally, consistency decreases more than seasonal passer rating when a player has a very bad game, so bad games could just be weighted more heavily in a consistency-based metric. Regardless, the results speak for themselves.