How to Be Sure You’ve Found a Higgs Boson

This article explains the meaning of five sigma analysis and their value.  It originally appeared in the Wall Street Journal.


July 6, 2012, 6:52 p.m. ET

How to Be Sure You’ve Found a Higgs Boson

            By CARL BIALIK

 The physicists who announced the likely discovery of the long-sought Higgs boson particle this week were operating according to an extremely high standard of certainty. As was widely reported, in order to achieve discovery status their experiment had to clear a threshold of “five sigmas” of statistical significance.

What precisely the five-sigma mark means, however, wasn’t always clearly explained in the coverage of a ground-breaking development that could explain how particles have mass and, by extension, why planets and all other objects exist at all.

That is partly because the five-sigma concept is somewhat counterintuitive. It has to do with a one-in-3.5-million probability. That is not the probability that the Higgs boson doesn’t exist. It is, rather, the inverse: If the particle doesn’t exist, one in 3.5 million is the chance an experiment just like the one announced this week would nevertheless come up with a result appearing to confirm it does exist.

In other words, one in 3.5 million is the likelihood of finding a false positive—a fluke produced by random statistical fluctuation—that seems as definitive as the findings released by two teams of researchers at the CERN laboratory in Geneva.

This is a very high burden of proof as far as science goes. For many medical experiments, researchers need merely to clear two sigmas. Sigmas don’t scale in a linear way: A two-sigma result can have as much as a 5% chance of occurring as a false positive. Three sigmas, needed to cite evidence—but not discovery—of a new particle in physics, correspond to a one-in-741 chance.

Other fields have different benchmarks because less is at stake if a result proves faulty, or, as with pharmaceutical studies, there is more upside to moving ahead quickly with a promising result, statisticians say. “Drugs can be withdrawn, psychological experiments can be refuted, but nobody wants to see the laws of physics proved wrong,” says David Spiegelhalter, a statistician at the University of Cambridge.

Sigma is the Greek letter used as a symbol for standard deviation—a measure of how far a finding departs from the expected one. The more sigmas attached to a result, the more likely it is significant and not due to chance. Say a series of experiments on a randomly chosen coin involves flipping it 1,000 times and then flipping it 1,000 times again and again. The average number of heads should be 500, but some experiments will yield more and some fewer. A five-sigma finding would be 590 heads.

The CERN finding, then, is the equivalent of getting a lot more heads than expected—which is unlikely to occur by fluke rather than for some systematic reason. The best explanation the CERN physicists have for this excess signal is the existence of the long-hunted Higgs boson.

They set such a high bar to rule out two other possible explanations: Either their equivalent of the coin is flawed in a way that tacks on extra positive signals; or they’ve run the study enough times and looked for anomalies in so many places in their data—the equivalent of running the coin experiment over and over with different coins—that they’ve stumbled upon a seemingly unlikely result just by looking too hard for it. Physicists call this the Look Elsewhere Effect, or LEE, and try to account for it. One CERN group said its finding’s significance falls to between 4.1 and 4.3 sigmas after accounting for LEE.

Another factor can pull in the opposite direction from LEE: when another experiment finds the same thing. That bolsters a finding’s significance. The Higgs boson was found at the five-sigma level by two CERN experiments.

The five-sigma requirement also helps guard against the equivalent of a faulty coin—some kind of measurement error. That appears to be the explanation for a finding last year that certain particles called neutrinos were traveling faster than the speed of light. It now appears to be the result of a flawed cable.

Particle physicists cite many examples of results that cleared three but not five sigmas and weren’t replicated by follow-up studies, which helped give rise to the five-sigma rule. University of Pisa physicist Giovanni Punzi, who has collaborated on a parallel hunt for the Higgs boson at the Fermi National Accelerator Laboratory in Chicago, says physicists have debated whether the rule is “exaggeratedly high.” It was devised decades ago to provide a margin of safety in case calculations of statistics for an experiment, done without today’s computing power, were flawed.

But counterarguments prevailed: Modern computing power makes it easier to find false positives; and with so many particles discovered, new ones better clear a high bar. Maybe the rule is “not a bad idea, after all,” Dr. Punzi says.


Tags: , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: