The Bertasso-Koontz Model of NFL Quarterback Prospect Analysis

Every season, teams spend inordinate amounts of time and money scouting players and attempting to gain a competitive edge over their opponents. Scouts spend countless hours watching film, attending Pro Days, and examining Combine results in an attempt to gauge how college athletes will fare in the NFL.  Sometimes, scouts are accurate, and predictions match a player's NFL performance. At other times, scouts fail, sometimes spectacularly, and a supposedly slam-dunk pick becomes labeled as a bust. 

No other position is as visibly volatile as the quarterback position.  For every JaMarcus Russell and Akili Smith (players who were drafted highly but never panned out), there's a Tony Romo (who went undrafted) or Tom Brady (6th round pick).  Given their NFL success, EVERY NFL front office missed on Romo and Brady.  How does this happen, given the huge amount of time spent scouting players? The answer is simple. When it comes to the quarterback position, NFL scouts are doing it wrong.

What if I told you that I could give you an almost perfect picture of a college quarterback's tendencies, his clutch ability, whether he's conservative or takes risks under pressure, and whether he's more error prone in pressure situations? What if I continued to tell you that I can give you that information in an easy-to-read chart that can be easily compared with other QB prospects, a chart that even shows the player's development over  the player's college career (if you define your data sets to show such a development)? What if I told you that given the right data specifications, I could force a drastic decline in NFL QB bust rates coming out of the NFL Draft?

I know what you're thinking.  Why am I sitting here behind a laptop writing articles for a website and not working for an NFL team if I'm able to do this? The short answer is this. Since I'm not on the inside in an NFL scouting department, I'm not forced to look at scouting through the relatively rigid, socioculturally constructed system used by NFL front offices.   "The combine matters."  No it doesn't, that's a myth, at least as far as predicting a player's NFL success.  "His Pro Day was great, therefore he's going to be great."  Wrong again. JaMarcus Russell had a great Pro Day, and did terribly in the NFL.  Teddy Bridgewater had a terrible Pro Day (and almost dropped out of the first round in the draft because of it), and he looks like a decent franchise QB. In the scheme of things, unless you have questions about a player's personality or work ethic (at which point, interview opportunities are always important), the Combine and Pro Days are almost entirely useless.

If Pro Days and the Combine are useless, then what am I basing my model on? The only true measure of a player's onfield ability. How can I build such a predictably powerful model?  Film.  Miles and miles of film.  The process is simple, but tedious.  In order to create a nearly infallible picture of a player's onfield tendencies, analyze every single college snap, and define your parameters.  You can't do this for a single game and expect a complete picture; players get sick sometimes and perform poorly. Sometimes players are pushed to perform extra well because of family at a game or other psychological factors. Some games are just different than others.  To gain usable data, one must look at every snap from at least the player's most recent season, but preferably every snap from a player's collegiate career; when looking at data from a player's entire career, you can identify how a prospect has developed/changed over time, which is useful information to have in-hand. This process might be time-consuming, but it can easily bring about a significant decrease in QB busts in the NFL draft. And guess what?  A player's statistical numbers are not nearly as important as a player's situational trends. 

That's right. I just said that stats aren't the most important thing to look at when analyzing a quarterback.  Why?  Because some quarterbacks are more fortunate than others when it comes to supporting casts.  Can we really expect a college quarterback with no recognizable talent to perform on the same level as a guy with 3 potential first round prospects at WR?  No. That's just foolish. Yes, elite players make the players around them better, but by gauging scouting solely on a prospect's numbers, NFL teams miss out on a number of tenable NFL starters every single season (and whiff on guys who aren't equipped to play in an NFL scheme).  Can we expect a guy who gets sacked 4 or 5 times a game to perform on the same level as a guy with an elite O-line who gets sacked once a game at most? Nope. Again, situation is important. Define your parameters and collect your data.

What data am I looking for? It depends entirely on the questions you want to ask, but the finished product should read like a well-constructed flow-chart. Do you want to know if a QB prospect's natural tendency is to scramble or if he's scrambling because of lack of open receivers?  First, isolate the total number of plays a quarterback prospect ran the ball. Then decide whether it was a designed run play. If it was not a designed run play, did the quarterback have open WRs, or was he forced to scramble due to no open options? Was the pocket collapsing around him, or did he have time to let things play out?

Curious about how clutch a guy is?  Look at his performance with open WRs in scoring position. Does a prospect go for the first down instead of a TD even with open WRs in the endzone? Do his tendencies change when the game is on the line (i.e. during 2 minute drills or when playing from behind)? Does the player stay calm in pressure situations and make his reads, or does he rush to try to make things happen, missing reads that would result in a score? Again, a simple flow-chart is enough to give a picture of how a player performs in this situation. Such an approach can accurately and adequately answer any questions about a prospect's tendencies and NFL-readiness.

But how do I read the data? Once you decide exactly what questions you want to ask, break down situations into percentages.  Say prospect X chooses not to throw to open WRs in the endzone 12 % of the time.  He leaves more potential points on the table than prospect Y, who only misses those opportunities 10% of the time. If all other variables are similar, then Prospect Y gives you a better chance to win than Prospect X, IF Prospect Y is a fit for your system.

We are in the process of defining and outlining this model completely, but at the very least, a few variables are extremely important.  Points left on the table (unrealized points), first downs unrealized (does a guy try to force the ball downfield to a WR in coverage and throw a pick or incompletion when there's a WR in first down territory open), and a player's clutch performance.  This model, when complete and used properly, can and WILL be a powerful predictive mechanism for teams aiming to pick the best possible quarterback prospect for their roster.  Our philosophy differs from that of most NFL front offices.  Rather than look at prospective stats, we're looking at prospective points and prospective wins. The player that gives you the best chance at scoring the most points and most first downs gives you the best chance to win. This is not always the player who passes for the most yards. Throwing for 400 yards a game is great, but if you throw interceptions and incompletions in key situations, then you give your team a lesser chance to win than a guy that only throws for 300 yards a game but does not leave potential points on the table. It's counterintuitive, but when seeking a quarterback who gives you the best chance to win, statistical data is not always viable data.

What do I mean? If a quarterback is playing from behind every week, of course he's going to throw for lots of yards (if he's at least capable as a player).  Is he playing from behind because of his own imperfections as a passer? Did he leave points on the board, forcing himself to try harder to try to win the game?  If a player is playing with a lead, often his statistical numbers will be relatively lower in comparison. Why? Because he's more efficient. He strikes and scores when necessary, and doesn't leave points on the board, and then his coach salts the win away with the running game. Statistics are unimportant. SITUATIONAL statistics are key.  How a player performs in key moments and in key game-time situations in college can give a nearly perfect picture of how a player can perform in the NFL. Lots of guys throw for lots of yards in college, but many of these guys never pan out as NFL starters.  Lots of front offices miss out on lots of guys who are extremely solid NFL starters. Brady and Romo are two examples of this.  Our model can and will make it easier to identify NFL-ready talent at the QB position, and we suggest that NFL teams utilize our model when scouting players. Want help defining your parameters and collecting data? Contact us, we'll gladly help.   If you want a competitive edge when scouting quarterbacks, this is absolutely the way to go. 

This model will be expanded in future posts.

--Mike B. and Matt K.