Clicked Gallery

What is P-Hacking?

Highlighted from a real textbook passage. Explained by Clicked.

Used in a sentence

University Course Reader · STEM

The effect appeared only after two participants were dropped and one covariate was added, which reviewers flagged as p-hacking rather than analysis.

The reader highlighted one term — in an article or a paper PDF. Clicked made the statistics term “p-hacking” easy to understand:

Explained in three depths

Same facts, different vibe — Slang mode 😎

The Clicked way

●○○

Overview

P-hacking is analysing the same data over and over, in different defensible ways, until a result looks statistically significant, then reporting only the version that worked. Nothing is faked and every number is real. The cheat is choosing after seeing the answers. Researchers once used it to show, with a real experiment, that a Beatles song made listeners 1.5 years younger, which no song can do, and that was the point.
●○○

Overview

P-hacking is squeezing a dataset until it admits something publishable. No forged numbers, just working an angle: which measurements count, when to stop collecting, which control sneaks in, which comparisons make the cut. The setup is built so a fluke sneaks through once in 20 tries. One famous team simulated all four tricks together, and junk data passed as a discovery more than half the time. Not one step along the way looked like cheating. 😎

A quick take — often all you need.

●●○

Detail

P-hacking is running many versions of an analysis on one set of data and reporting the one that crossed the line for statistical significance, as if it were the only test. The versions come from ordinary choices. Drop the outliers or keep them. Add a control variable or not. Stop collecting data now, or keep going. Each choice is defensible on its own. That is what makes the practice invisible. The fault is the timing. Any analysis chosen before the data arrives is a test. The same analysis, picked afterwards from a menu of tries, is a search, because afterwards lets luck choose the method. In 2011 three researchers ran a real experiment on college students to prove how easy this is. One group heard a Beatles song, the other a neutral one. Reported honestly, the numbers said the Beatles group turned 1.5 years younger, p = .040, which cannot be true. They had also asked each student a dozen extra questions, from how old they felt to their father's age. The result cleared the line only when father's age was adjusted for. Without it, p = .33 and nothing was there. The fix is deciding the analysis before looking, and disclosing everything tried. P-hacking is different from publication bias, which decides which finished papers get read. P-hacking happens before the paper exists.
●●○

Detail

P-hacking is auditioning analyses until one passes, then pretending it was the sole audition. The scary part is that whoever is doing it rarely feels dishonest. Every tweak has a story. That outlier really did look weird, that extra variable really is standard, and surely five more participants can only help. Wanting the win does the rest, and the flattering option keeps feeling correct. Peeking is the classic form. Start with 10 per group, re-run the numbers with every new arrival, quit the moment the stars align, and you will catch an effect that is not there about 22% of the time. In one survey cited by the field's most famous takedown of all this, roughly 70% of behavioural scientists admitted to exactly that kind of peeking. The tell for a reader is one question. Would that call still have happened if the numbers were already significant anyway? If the true answer is no, the method got picked by the outcome. However it happened, the false alarms this produces are the findings that later refuse to replicate. 😎

Want more? One click digs deeper.

●●●

Analogy

You weigh yourself five times before deciding what to tell people. After the gym, before breakfast, shoes off, one hand resting on the towel rail. The lowest reading wins and becomes your official weight. Every number on that scale was real. The lie is in the choosing, because you saw all five, kept the flattering one, and never mentioned the rest. P-hacking is this exact move performed on research data. Many honest readings, one reported, and the search left out of the story.
Deepest level reached.
●●●

Analogy

A golfer tees up five balls, slices four into the trees, and writes down the one that found the fairway as a clean single stroke. Nothing on the scorecard is forged. That shot genuinely happened, and the card still lies, because four attempts are missing from it. Ask about his handicap and you get the fairway edition of his life. A p-hacked study is that scorecard. Real swings, missing retries, suspiciously tidy round, and the trees keeping the truth. 😎
Deepest level reached.

Unfamiliar concept? A real-world example makes it click — fresh analogies on tap.

AI explanations may contain errors · Not professional advice

Formal definition — The same term, explained the usual way

P-hacking is the practice of exploiting flexibility in data collection, analysis, and reporting so as to obtain a statistically significant p-value, typically below the conventional .05 threshold. Techniques include selective exclusion of observations, choice among dependent variables or covariates, post hoc selection of conditions, and data-dependent stopping rules. Because each reported statistic is computed legitimately, p-hacking is distinct from data fabrication; its effect is to inflate the false-positive rate above the nominal significance level, and it is mitigated by pre-specification of analyses and complete disclosure of analytic decisions.

Want Clicked to explain terms like “p-hacking” directly in your browser — including on PDFs?

Add to Chrome — Free

50 free Explanations · No credit card required