How to Settle the Manning Versus Brady Question Once and for All (Maybe)

The "Picasso of Choke Players" v. "One of the Great Clutch Players in Postseason History." The player who has "failed to live up to expectations in the postseason" v. "a truly transcendent performer who reserves his greatest games for the biggest moments." That's how Cold Hard Football Facts has been styling the Manning v. Brady question since 2004. That's the year CHFF first published its dossier supposedly proving that, when it comes to postseason heroics, Peyton Manning stinks and Tom Brady shines.

The dossier, titled The Complete & Unabridged Guide to Why Tom Brady is Better than Peyton Manning, has become the deep fountainhead for an endless stream of "Peyton Chokes" mantras.

Choke v. Clutch

Not surprisingly, with the Colts' 2008 first-round playoff loss to the San Diego Chargers---to a quarterback named Philip Rivers and a coach named Norv Turner---the Manning v. Brady, Choke v. Clutch debate raged again. The Chargers made 23-17 (OT) mincemeat out of the Colts, knocked them out of the playoffs, and squashed their run for the Super Bowl after a blistering nine game winning streak.

On January 1, 2008, Peyton Manning was named the Associated Press National Football League's Most Valuable Player.

On January 3, 2008, Peyton Manning and the Indianapolis Colts packed their bags and went home for the season.

All over the internet, the hue, cry, jeer and cheer went up: Peyton Manning choked.

Did he?

"A Complete and Utter Refutation"

Deshawn Zombie, writing at 18to88, had a few words to say about CHFF's Manning v. Brady analysis. Specifically, these words: A Complete and Utter Refutation of Cold Hard Football Facts.

In this intelligent and sharply analytical piece, Zombie systematically rips and shreds (what he calls) the "logical and methodological flaws in CHFF's article," which, as you might imagine, declared Tom Brady superior to Peyton Manning in approximately every way. While Zombie does start by calling CHFF's compendium of Brady superior-isms "ridiculous," ultimately he declares that it is up to each reader to reach a Manning v. Brady conclusion for himself.

I have a better idea.

Let's ask ZEUS.

ZEUS is a computer program developed by Frank Frigo and astrophysicist Chuck Bower, PhD., of Indiana University. Based on zillions of statistics faithfully kept by the National Football League, as well as on behavioral tendencies of coaches, ZEUS performs up to 1,000,000 game simulations every few seconds. The ability to sort through these game simulations allows ZEUS to make a robust and accurate assessment of critical play calls and their impact on a team's chance to win the game. According to Pigskinrevolution, there simply is "not a more accurate way to assess a critical play-calling decision in football."

What does play calling have to do with who is the best quarterback?

Potentially, a lot.

What I'm suggesting here is this: for all the jawboning about Manning v. Brady, and Choke v. Clutch, maybe the real issue is Dungy v. Belichick. Here you have two coaches who have very good quarterbacks. But what if one coach consistently was better at putting his quarterback in a position to win? Who would be Choke and who would be Clutch then?

Good is not great

Pigskinrevolution posts commentary on the results of ZEUS analysis of various aspects of NFL games by season. In a piece on the 2006 season, The Line Between Good and Great, Pigskinrevolution reported that a ZEUS play calling analysis, known as Critical Call Index rankings, showed that Tony Dungy had "one of the highest error rates in the league."

Let me repeat that: In the 2006 season, according to ZEUS, Tony Dungy had "one of the highest error rates in the league."

This means that ZEUS found that Dungy's calls on critical plays---defined as a 4^th down play with fewer than five yards needed to gain a first down---substantially decreased the Colts' chances of winning games.

Okay, so a computer thinks Dungy messed up. But, but . . . the Colts were wearing Super Bowl rings. Isn't it a little silly to let a computer argue with world championship success?

Pigskinrevolution says, basically, the computer does not lie. Their argument is this: "Dungy's shortcomings are masked by a very talented squad. . . . [P]oor coaching decisions may go unnoticed if you are a 90% favorite on player talent alone. However, when the game is a closely matched affair, . . . it can be a very different story."

Play calls matter

ZEUS found that Tony Dungy puts up bad numbers when it comes to critical play calling. So, what about Bill Belichick, Dungy's nemesis at the New England Patriots, and coach of the much-heralded Clutch Master, Tom Brady?

In the same 2006 commentary, Pigskinrevolution noted that Bill Belichick outperformed Tony Dungy in critical play calling by an average of 2-3% per game.

Trust me: that is not chopped liver. In fact, in commenting on the Colts v. Patriots game, the ZEUS guys went so far as to pin the loss on Dungy's play calling: "Dungy made two very conservative choices in the first half . . .. These two decisions alone may have been the deciding factor in what was not surprisingly a very tight contest."

In case that point was not sufficiently clear, the commentary closed by stating: "The post-game wrap-up in most media outlets focused on the heroism of Brady and the epic struggle of opposing defenses. Those dynamics may be important, but Dungy's conservative play-calling in the early phases of the game likely was the most significant factor separating these two great teams."

This does not mean Belichick is perfect. He calls some whoppers, too. In the November 2, 2008 (Week 9), game between the Patriots and the Colts, for example, the New York Times Fifth Down blog noted that ZEUS found Belichick's decision to take a time-out and then a field goal, rather than going for it on 4^th and short deep in the Colts' red zone, "squandered close to an 8 percent chance of winning the game. " Final score? Colts 18, Patriots 15.

Remember: this is commentary, but it is based on a 1,000,000-game-simulations-per seconds computerized statistical analysis of critical play calling. And it seems to me that you can't talk about how great a quarterback is in a vacuum. Play calls matter.

Does Manning choke or does Dungy blow it?

Basically, the Manning v. Brady debate has focused primarily on each player's statistics. Or, as Cold Hard Football Facts puts it, "Brady's superiority over Manning is built upon the stony, unyielding foundation of each player's postseason performance."

That sounds good (even a little poetic). But if one player's coach consistently puts him in a superior position to win, compared to the other player and his coach, how relevant is a strict comparison of player stats? How "stony" and "unyielding" is the foundation of a player who wins under the best of circumstances compared to a player who wins under the worst?

Dungy v. Belichick

ZEUS provides the analytical means to assess much more than how one player executed plays. It actually assesses the plays. So, bring it on! Rather than watch the rage about Manning v. Brady, Choke v. Clutch, maybe the ZEUS guys should feed Tony Dungy and Bill Belichick into the computer and figure out just what it is that each quarterback had going for him, and against him, in terms of critical play calling. In other words, let's see just how big a mountain each quarterback had to climb to reach success.

It also would be interesting to know Dungy's and Belichick's error ratings for the season and, separately, the post-season. If critical play calling got even worse in the post-season, that might be one for the coach's "choke" column.

Finally, if, as the ZEUS guys state in The Line Between Good and Great, the hugely talented and scrappy Colts largely were able to compensate for the negative impact of poor play calling, how often did this occur? In contrast, how often did Brady and the Patriots successfully compensate for poor play calling?

Sure, more statistics engender more debate. And none of this will settle the Manning v. Brady debate completely. But consulting ZEUS on how Dungy and Belichick performed in each game may be more enlightening than comparing how Manning and Brady performed. At least, ZEUS may move the ball a little farther down the field. And that is what football is all about.

Sources:

"Brady vs. Manning: Complete & Unabridged, Vol. II," Cold Hard Football Facts.

Deshawn Zombie, "A Complete and Utter Refutation of Cold Hard Football Facts, 18to88.

"FAQ," Pigskingrevolution.

"The Line Between Good and Great," Pigskinrevolution.

Toni Monkovic, "Zeus Finds Flaws With Belichick," NYT Fifth Down Blog.