Dark energy explained, three decades on

Dark energy is a name attached to an observation, and keeping that clear avoids most of the confusion around it. The observation is that the expansion of the universe is accelerating. The name covers whatever is responsible. Three decades after the measurement, the responsible thing has not been identified.

What was actually measured

Two independent teams in the late 1990s used Type Ia supernovae to measure the expansion rate at different times in cosmic history. Because the explosions have a nearly fixed intrinsic brightness, their observed brightness gives a distance, and their redshift gives the amount the universe has expanded since the light left.

Both teams found distant supernovae fainter than a decelerating universe predicts — further away than expected. The expansion is not slowing under gravity, as everyone assumed; it has been speeding up for roughly the last five billion years. The result has since been corroborated by the cosmic microwave background and by the large-scale distribution of galaxies, which are independent methods and agree.

The simplest reading

Einstein’s field equations admit a constant term, the cosmological constant, which acts as an energy density filling space uniformly and does not dilute as space expands. Because it does not dilute while matter does, it inevitably comes to dominate — which produces late-time acceleration of exactly the observed kind.

That fit is good, and it needs one number. The constant accounts for about 68 per cent of the total energy density of the universe, with dark matter around 27 per cent and everything made of atoms under 5 per cent.

Whether the term is genuinely constant is the open question. If the energy density varies with time, that points to a dynamical field rather than a constant, and the two possibilities are distinguishable by measuring the expansion history precisely enough. Current surveys exist largely to make that measurement.

Why the natural explanation fails

The apparently obvious identification is with quantum vacuum energy. Quantum field theory says empty space is not empty, and the fluctuations carry energy — which sounds exactly like a uniform energy density that does not dilute.

Calculating it gives a number about 120 orders of magnitude larger than the observed value. This is not a factor-of-ten disagreement or a factor of a thousand; it is the largest discrepancy between prediction and measurement anywhere in physics. Something must very nearly cancel the vacuum contribution, leaving a small residue, and no mechanism that does so has been found. The near-perfect cancellation is harder to explain than a total cancellation would be.

What is being measured now

Three independent lines are running. Large galaxy surveys map the distribution of matter across billions of light years and read the expansion history from the scale of features imprinted in the early universe. Weak gravitational lensing measures how much the shapes of distant galaxies are distorted by the mass between them and us, which constrains how structure grew. And Type Ia supernova samples continue to expand, with better calibration than the 1990s discovery had.

What all three are chasing is one number: the equation-of-state parameter, which says how the dark energy density changes as space expands. A cosmological constant has a value of exactly minus one. Anything measurably different means a dynamical field instead, and the two possibilities lead to different accounts of the far future.

What the situation actually is

The observation is secure and confirmed three ways. The description — a cosmological constant at around 68 per cent of the energy budget — fits well and requires one free parameter. The explanation is missing.

That is a defensible place for a field to be, and it is worth stating plainly rather than dressing up. Dark energy is not a discovery about what the universe is made of. It is a well-measured effect with a placeholder name, and the placeholder has held for thirty years because nothing better has earned the position. The measurements that put it there are collected in the supernova and cosmology section of the publication index.