What cherry-picking means
Cherry-picking is choosing only the data points that support a story and leaving out the rest. The statistics quoted are usually true. What is misleading is the selection. A claim like “this team has won 8 of its last 10 at home” can be accurate and still give a very different impression from the full season.
It is rarely a deliberate trick. Commentators, journalists, and fans all look for something notable to say, and notable numbers tend to be the ones that stand out. The result is a steady supply of statistics that were found by searching, not by testing a prior question.
The time window
The simplest form is choosing where the clock starts. Consider this illustrative sequence of twenty results for a team, with W for a win and L for a loss:
L L W L L W W L L W | W W W L W W W W L W
Over all twenty games the team has 12 wins and 8 losses, a record of 60%. Over the final ten games the record is 8 wins and 2 losses, or 80%. Over the first ten it is 4 wins and 6 losses, or 40%. A writer who wants to say the team is on the rise can quote the last ten. A writer who wants to say the team started badly can quote the first ten. Both are using real numbers.
The start and end points matter more than people expect. “Since the manager arrived”, “since the patch”, “since the new roster” and “this calendar year” are all choices, and each can be placed to make a favourable stretch look like the whole story.
The split
The second form is slicing the data into categories until something striking appears. A team might be unbeaten on Tuesday evenings, or a player might have a great record against left-handed opponents, or on one map. If you cut the same data enough ways, some slices will look remarkable by chance alone, much as some short stretches of random coin tosses show a surprising run. Quoting only that slice hides the number of slices that were examined and found unremarkable.
A useful question is whether the split makes sense before looking at the data. A reason to compare home and away games is easy to state in advance. A reason to compare games on a particular weekday usually is not.
Splits can also mislead in the opposite direction, which is related to Simpsons paradox: a trend in the whole data can disappear or reverse inside the groups, so both the overall figure and the grouped figure need context.
The comparison
A third form is choosing a flattering comparison. “Best start in eight years” sounds impressive, but if the previous seven years began poorly, the bar is low. “More wins than any other team in the division” is a narrow comparison if the division is small. Ask who or what is being compared, and whether another baseline would give a different impression.
Measures, not just windows
The same can happen with the measure itself. A player may be described by whichever statistic is most flattering, such as kills but not deaths, shots but not accuracy, or an average but not the median. Choosing which measure to present is another form of selection, and it connects to the question of how the mean can hide outliers.
Why short windows are especially risky
Short windows contain few results, and few results are noisy. A small number of games can be dominated by luck, and a record that looks like a trend may just be what ordinary randomness produces. The reasons are explored in the article on small sample sizes and in the idea of random streaks, which shows that clusters of wins or losses appear naturally even when each game is independent. Believing that a streak must continue, or must end, leads towards the gambler’s fallacy or its opposite, the belief in a hot hand.
Questions that help you spot it
- Why this window? Would the claim hold if you started a month earlier or ended a week later?
- How many cuts were tried? Was this the only comparison, or one of many?
- What is the full picture? What does the whole season, the whole career or the whole patch look like?
- What is the baseline? What would you expect without any special cause?
- How big is the sample? Are there enough games to say anything?
- Is a cause being implied? Statements like “since the coach changed” suggest a link that may not be there, a point developed under correlation versus causation.
Fair use of selected statistics
Not every selected window is dishonest. Breaking data into sensible periods can reveal real changes, for example before and after a rule change. The aim is not to distrust every statistic, but to treat an unusually neat one as a prompt to look for the surrounding numbers. If the claim survives a different window and a different split, it is more credible. If it vanishes when the window moves a few games, it was probably a feature of the selection.
When you can, look for the complete table or season record and compare it yourself. A few minutes spent doing that is often enough to separate a genuine pattern from a convenient quotation.