Evaluating WVU’s Fourth Down Decisions Last Season

A college football coach’s main job is to put his team in the best position to win. He can accomplish this by recruiting well, instilling a great culture in the program, and preparing his team to play during the week. On gameday, a good coach will optimize his team’s chance to win when making tough strategical decisions. A common decision all coaches face is whether to kick or go for it on 4th down.

Lots of variables might go into a 4th-down decision: field position, score, time remaining, health and stamina of your team, weather, etc. The bottom line is that a coach should be making the decision that is going to give his team the best chance to win the game. But how do we judge if coaches made the correct decision?

One way is by comparing how the decision will affect the win probability. If you followed the NFL this year, you will recognize this example: The Packers were trailing by 8 points with 2:09 left in the 4th quarter of the NFC Championship. They faced 4th and goal from the 8-yard line. The Packers decided to kick the FG and ultimately lost the game. According to the data, the Packers had a 98% chance to make the FG, but even if they made it, they would still trail with little time left and would have an estimated win percentage of 9%. If the Packers had gone for it, they would have roughly a 33% chance of scoring the touchdown which would put them at a win pct of 31%. A missed FG or stop on 4th down would essentially doom the Packers. Accounting for the likelihood of successfully scoring the TD, the decision to go for it would give the Packers a 3.8% higher chance to win on average.

Ben Baldwin has created a bot that live tweets these decisions during the NFL season (@ben_bot_baldwin). A similar bot has been created for college football and can be found at @aisports_4th on Twitter. Using the data we have, we can go back and analyze how the Mountaineers performed with their 4th-down decision making this year.

The Mountaineers faced a 4th-down decision 80 times in the 2020 season. Most of these decisions weren’t very tough ones to make. For example, punting on 4th and 19 from your own 3-yard line in the 1st quarter versus Kansas State wasn’t exactly a genius tactical decision. To evaluate these decisions I’m using the win probabilities calculated by the aisports bot. You can look at how the bot would react to any situation at https://kazink.shinyapps.io/cfb_fourth_down/.

Let’s start with the good. These are no-brainers but the correct decision was made. The five best 4th-down decisions this year according to the model:

We see three punts on 4th and long which are easy calls to make. The other two decisions were going for it on 4th down versus Texas while trailing by more than a FG in the 4th quarter. These were the right decisions to make, but unfortunately neither were successful in practice.

Now let’s look at the bad: of the 80 total 4th downs, the bot recommended the Mountaineers make a different decision that would have increased their win percentage by at least 1% a total of 10 times. I’d like to make a disclaimer that obviously the bot cannot take all factors into play, but these plays at least give us a good idea of decisions that might be questionable. Here are the 10 decisions the bot disagreed with:

First, I’d like to mention that numbers 7-10 don’t have huge decision values so they don’t deserve as much weight. Frankly, #10 doesn’t even seem correct to me. Looking at just the top 6, all have decision values that indicate the model strongly prefers going for it as opposed to kicking. The main takeaway here is that the Mountaineers were likely too conservative in taking the points in these situations. All of these top 6 decisions yielded 3 points but were inside the opponent’s 25-yard line and were 4th and 5 or shorter. In fact, the field goal versus Kansas State was the worst 4th-down decision in terms of the decision value metric in the Big 12 this season!

The frustrating part of this is that looking at the 6 worst decisions, 5 of them came in close losses. Twice versus Texas (13-17 final), twice versus Oklahoma State (13-27 final), and once versus Texas Tech (27-34 final) the Mountaineers left incredibly valuable points out there by deciding to take 3 instead of trying for 7.

Win percentage decision bots do not give us the universal best decision for every 4th down, but huge decision values indicate places where the model strongly favors one decision. To win close games in a tough league, the Mountaineer staff has to figure out how to optimize their in-game decisions.