Baseball umpire scorecards provide a unique perspective on the accuracy and consistency of balls and strikes calls made by home plate umpires during MLB games. Full game umpire scorecards have become increasingly accessible to both fans and analysts in recent years due to expanding data sources and new technologies that automate the scoring of ball-strike calls made by umpires.
MLB Advanced Media and Sportvision PBC have partnered to develop Pitch f/x technology, which utilizes highly accurate tracking cameras to record the location, speed, and movement of every pitch thrown in MLB games. By comparing the locations of pitches tracked by Pitch f/x to the ball-strike calls made by umpires, it is possible to determine the percentage of calls an umpire gets “right” or “wrong” based purely on location.
Sean Lahman gathered all publicly available Pitch f/x data and ball-strike calls from 2007-2014 and developed software to automatically generate complete MLB umpire scorecards after each game. These scorecards present the accuracy percentage for each umpire and also break down their performance on balls versus strikes, as well as for different pitch locations and types. Additional analysis of called ball-strike locations is used to determine if specific umpires exhibit zones that are systematically larger or smaller than the true 18-inch wide MLB strike zone.
The mass availability of umpire scorecards took transparency and scrutiny of umpiring performances to new heights. Fans could readily see not just the accuracy percentages but also the impact of missed calls on specific at-bats and games. Analysts were also empowered to study patterns in umpire zones and consistency over their careers.
It’s important to note some limitations of umpire scorecards generated solely from Pitch f/x data. First, the strike zone defined by Pitch f/x is an approximation based on average hitter heights and may not match the actual zone called on a given night. Pitch movement/location is also calculated to a certain margin of error.
Additionally, Pitch f/x only tracks what actually happens, not necessarily what was correctly called according to the rule book. For example, a borderline pitch near the edges that is correctly punched out as a strike by the umpire may still be scored as a miss by Pitch f/x data. Crucial framing skills of catchers can also influence called strikes on borderline pitches.
With these caveats in mind, umpire scorecards remain a very useful tool for assessing accuracy over large samples as long as appropriate context is provided. Studies have found consistent differences in called zones between individual MLB umpires, with accuracy rates ranging from the high 80s to low 90s percent based on Pitch f/x.
Perhaps the highest profile use of umpire scorecards came in August 2015, when longtime umpire Angel Hernandez had a widely publicized game with a suspiciously large strike zone and an accuracy rate below 80%. This single performance helped shine additional scrutiny on Hernandez, who ended up with one of the lowest career accuracy rates among active MLB umpires based on extensive scorecard samples.
Accurate umpiring is crucial in MLB where games often come down to one pitch and human eyes cannot possibly call every corner perfectly. Umpire scorecards help bring much-needed transparency and accountability to the job while also acknowledging the inherent human element at play. With continued improvements in tracking technology, the case for fully automated ball-strike calling may continue to grow. But for now, umpire scorecards represent an important step forward in publicly evaluating umpiring performances.
Publicly available MLB umpire scorecards assessed from Pitch f/x data have provided fans and analysts an unprecedented view into the accuracy and consistency of ball-strike calls. While not perfect, scorecards remain a valuable tool for transparency when viewed with appropriate context regarding their limitations. Continued studies using large scorecard samples will help ensure optimal umpiring standards moving forward.
