Can sporting artistry be measured?

In short, yes. But how accurately?

Well, the purpose of this blog is to delve deeper into the data drivers for sporting metrics. There are some big questions that call for closer examination:

• Do they measure what they claim to be measuring?
• Are the measures worth measuring?
• What are they not measuring?
• Can it even be measured?
• Is there any inherent distortion in the metrics?
• Are there better ways to measure sporting performance?
• Are there sporting skills that are not being measured that ought to be included?
• Can everything be measured?
• How is statistics changing in the age of data science?
• Can data science capture artistic creativity?

This blog seeks to explore these questions with two clear aims:

1. To take a critical look at sporting metrics. Determine if the actual measure is a valid proxy for what is intended to be measured. Unchallenged measurements could deliver distorted knowledge – knowledge that seems robust and reasonable at first sight but is actually deceptive.
2. To offer new ways to better measure sporting performance. Used properly, measurement can be a force for truth and justice. Thus, judicious measurement of the previously unmeasured can deliver real benefits.

Numbers: the good, the bad and the ugly!

In Trust in Numbers, Theodore M. Porter’s approach was to ‘regard numbers, graphs, and formulas first of all as strategies of communication’. And since the rules for collecting and manipulating numbers can be widely shared, they can readily be disseminated across cultures and continents. This allows highly, complex sporting movements/performances to be presented in a familiar, standardised form, thus being a tool for sound comprehension and understanding – taking it beyond the restricted realm of the sporting elite experts. In addition, as reprised in the book Moneyball, statistical analysis can help reveal the presence of clearly measurable but often neglected characteristics that have more significance and weight than those accepted through custom and practice based on intuitive understanding and accumulated experience.

However, because numbers carry the aura of objectivity, it is often inferred that the information is sound, hard and objective. Metrics are numbers, and thus readily appear to represent facts waiting to be revealed, consequently it’s just a matter of discovering them. The rub is that people gather statistics, they choose what to count, how to count, which of the resulting numbers are to be shared and which words are used to describe and interpret those numbers. Thus, metrics are not facts, they are interpretations. So, whilst numbers can push subjectivity to the side, it will never be completely removed.

Though standardised data allows a way to communicate a complex, multidimensional sporting skill by a single number (or two), it is likely to lead to distortion since the act of making things comparable often strips out context, history, and meaning. This is something that sporting metrics must consider.

• Does a goal count more or less when it is the winning goal? If so, by how much?
• Does it count more or less when the winning goal is scored later in the game? If so, by how much?

You may agree that some goals are ‘worth’ more than others, but how do metrics capture this? One way would be to ‘weight’ certain categories. But the questions then become: what is weighted and, more importantly, by how much? And, of course, who decides this?

A familiar dictum suggests, ‘not everything that can be counted counts, and not everything that counts can be counted’. The first part of this dictum can be categorised into two recurring flaws highlighted by Jerry Z. Muller in his book The Tyranny of Metrics. First, measuring the most easily measurable. Muller says, ‘There is a natural human tendency to try to simplify problems by focusing on the most easily measurable elements. But what is most easily measured is rarely what is most important, indeed sometimes not important at all.’ Second, measuring the simple when the desired outcome is complex. Most, if not all, sporting skills are multidimensional and reducing these skills to just one dimension often leads to deceptive results. The latter part of the dictum highlights the immeasurable.

Metrics, of course, can be gamed, and the accepted issue for sport is outcome reporting bias, in particular when individuals are compared.

For example:

• selective omission of outcomes where results are deliberately left out;
• selective choice of data for an outcome where limited data, usually that showing a favourable result, is reported;
• selective reporting of different analyses using the same data where results possess a greater impact when reported as different data types, i.e., changing a continuous variable to a category variable;
• selective reporting of subsets of the data where total numbers are not reported;
• selective under-reporting of data where actual data is avoided and reported in words such as, ‘greater’ or ‘fewer’.

Finally, in addition to vested interests being a determining factor, a critical stance must be taken towards measurement, in that it may provide distorted knowledge – knowledge that seems sound but is actually deceptive. What is worse, and very real, is that it’s done consciously. Now, can numbers really truly represent an athlete’s performance. For example:

• Can a player’s touch be measured?
• Can a player’s decision making be measured?
• Can a player’s influence be measured?

With all of these, if so, how accurately? In trying to understand how accurate a sporting metric can measure what is intended to be measured, a golfing example will be looked at.

Who is the best putter on the PGA Tour?

To help us answer Who is the best putter on the PGA Tour? we turn to putting metrics. Let’s have a look at the evolution of putting metrics over the last decade.

1. Number of Putts Per Round – a simple count of the number of putts a golfer takes in an 18-hole round.
   

   What is measured?

   What is not measured?

   What is not measured? In the example,both players have taken 4 putts for the first 3 holes. According to the number of putts per round metric,their putting performance is the same (after 3 holes). However,Player A misses two short putts. He misses a 5-foot putt on hole 1 and a 7-foot putt on hole 3. Player A chips in on hole 2. Player B holes an 8-foot putt on hole 1 and a 17-foot putt on hole 2. He then takes two putts from 55 foot on hole 3. Player B has putted far better than Player A but the metric does not show this. This metric does not take into account putt distances. A golfer who chips to tap-in range after missing the green takes fewer putts than a golfer who hits the green and two-putts from sixty feet. The fewer putts taken is due to a superior short-game shot (and an inferior long-game shot),not superior putting.

2. Putts Per Green in Regulation – average number of putts taken when a player finds a green in regulation (GIR).
   Note: A green is considered hit ‘in regulation’ if any part of the ball is touching the putting surface while the number of strokes taken is at least two fewer than par (i.e., by the first stroke on a par 3, the second stroke on a par 4, or the third stroke on a par 5).

   What is measured?

   What is not measured?

   What is not measured? In the example, according to the putts per green in regulation metric, Player B (1) is better at putting than Player A (1.5). However, Player A two-putts from 35 feet on hole 1. Holes a 14-foot putt on hole 2 and holes a 9 foot-putt on hole 3. Player A has putted better than Player B but the metric does not show this. Again, putt distances are not taken into account. A golfer who hits more GIR tends to be putting from longer distances. Also, if a player does not hit a GIR, then their putting performance is ignored from that hole. Noting that the PGA Tour average for GIR is just under 12 greens, that means on average 6 greens (approx. 33%) worth of putting data are discounted from the metric.

3. Length of Holed Putts – sum of the footage of putts holed in a round.
   

   What is measured?

   What is not measured?

   What is not measured? In this example, according to the length of holed putts metric, Player A (75ft) is better at putting than Player B (36ft). However, Player A missed a 5-foot putt on hole 1 and 3-putted from 30 feet on hole 2. They then holed a 72-foot putt on hole 3. Player B holes a 12-foot putt on hole 1, a 15-foot putt on hole 2 and a 9-foot putt on hole 3. Player B has putted better than Player A but the metric does not show this. This metric changes dramatically when a long putt is holed, but a single lucky putt may not be representative of putting skill over an entire round.

4. Putts Gained – measures the number of putts better or worse than the field from a given distance.
   

   Putts gained=PGA TOUR average putts to holeout-Actual putts to holeout

   What is measured?

   What is not measured?

   What is not measured? In this example, according to the putts gained metric, Player A (+0.4) is better at putting than Player B (-0.4). Player A has putted better than Player B and the metric does show this. Whilst this metric is a marked improvement on the others. We need to ask what it does not measure? First, the difficulty of the putt has been measured using only one dimension: distance. It does not measure other dimensions such as the amount of turn or the elevation change of the putt. An 8-foot straight, slightly uphill putt is far easier than an 8-foot downhill putt turning 6 inches. Second, the greens for the course/day may be much more difficult (bumpier and/or faster) or easier (flatter) than the average green.

5. Adjusted Putts Gained – in order to account for differences in putting difficulty from one course to another, and from one round to another, putts gained per round are adjusted by the average putts gained of the field for that round.
   

   Adjusted putts gained

   =

   PGA TOUR average putts to holeout – Actual putts to hole out

   –

   average putts gained of the field (for that round)

   What is measured?

   What is not measured?

   What is not measured? In this example, Player A has putted better (+ 0.6) than Player B (- 0.2) and the metric does show this. This metric has improved by factoring in the difficulty of the greens on that day. The average putts gained of the field was – 0.2, indicating that the entire field putted worse – more undulation perhaps – than the PGA TOUR average for the season. Player A has putted better than Player B but the metric does not show this. However, this metric is still only measuring one dimension of putting difficulty: distance. Other dimensions, such as the amount of turn and elevation change should be considered / measured. An 8-foot straight, slightly uphill putt is far easier than an 8-foot downhill putt turning 8 inches.

The evolution – improvement – of putting metrics over the last decade indicates that what is actually being measured is becoming a better proxy for what is intended to be measured, i.e., the true putting skill of a golfer. This example highlights a typical flaw of sporting metrics: measuring the simple when the desired outcome is complex. An example of using this flawed approach is the assist metric in football (see Assists must be replaced by expected assists). Sporting metrics will always have some distortion but the goal is to minimise this as much as possible. The chart below ‘Putting Metrics’ is just a visual representation to indicate, roughly, how much distortion, I believe, to be in each of these putting metrics.

Whilst the adjusted putts gained metric is a valid proxy for what is intended to be measured, it can still, and should, be improved. One way to achieve this would to account for the difficulty of the putt more, using more than just one dimension (distance). In other words, to measure the (total) amount the ball turns as well as the (total) amount the ball changes in elevation. I am under no illusion that this is simply, but that’s not the point. The point is we should be attempting to remove the distortion. Professional gymnasts and divers are marked / measured against the difficulty of their routine or dive. A golfer’s putting ability should be too.

The Stat Squabbler concludes:

1. More, reasoned-based measurement is needed to form more complex metrics that are a better proxy for what is intended to be measured.
2. That stats are not concrete facts (they all contain distortion – some significantly more than others) and we should always ask, what is not being measured?

Do you agree with the Stat Squabbler? Can sporting artistry be measured? If so, to what extent? (Can you put a number on it?) Comment below.