My Personal History With the Quantum Theory of de Sitter Space

Abstract

The purpose of this brief article is to clarify certain distortions of the history of ideas about the quantum theory of de Sitter space that have appeared in recent literature.

1. Introduction

I first heard about the cosmological constant problem from K. Johnson in a lecture at the Scottish Summer School in St. Andrews in 1976. L. Susskind, who had emphasized the importance of the gauge hierarchy problem to me since we first worked together in 1975, often referred to it as an even bigger mystery. In 1999 I was sitting on the patio of my new home in Big Sur, staring at the ocean, when a number of thoughts came together in my head. We had learned from AdS/CFT that when it was negative, the cosmological constant should be thought of as a parameter characterizing the high energy behavior of particular models, and was not renormalized in low energy effective field theory. My friends Fischler and Susskind had proposed generalizing the Bekenstein-Hawking bound for black holes to cosmological space-times, and this had led Bousso to a general conjecture on entropy bounds for space-time regions. The universe we inhabited seemed to be approaching a de Sitter asymptotic future, and Gibbons and Hawking had predicted that such a universe had finite entropy. Moreover, conventional string theory seemed unable to cope with such a universe.

Suddenly it all made sense. The Gibbons-Hawking entropy should be thought of as the log of the dimension of the Hilbert space, which implied that empty de Sitter space was a maximal entropy state, and indeed from the point of view of any geodesic, anything not bound to the geodesic appears to fade into its cosmological horizon in a time of order the de Sitter radius. Furthermore, the black hole entropy formula showed that objects bound to the geodesic lowered the entropy.

Note by the author: It wasn't until somewhat later that I realized that this gave a derivation of the precise value of the Gibbons-Hawking temperature independent of quantum field theory. It was much later that Fischler and I realized that this was the key clue to how local physics arises from a quantum theory of horizon degrees of freedom.

I spoke about these ideas at the 60th birthday celebration for L. Susskind at Stanford in the Spring of 2000. Fischler was there and told me he'd had the same idea independently. He spoke about his version at Geoff West's 60th birthday celebration in Taos, New Mexico, also in 2000. I've always tried to give him equal credit, but the community has resisted this. I finally published a version of this in connection with the talk I gave at the strings conference in Michigan. Some people greeted the idea that de Sitter space had a finite dimensional Hilbert space with a certain amount of enthusiasm, but for the most part it was met with skepticism and mockery. I spent hours trying to convince E. Witten that the idea was correct, despite the fact that in global coordinates the past and future asymptotic regions of de Sitter space seemed to have arbitrarily large space-like slices.

I gave two arguments, one mathematical, one physical. The mathematical argument was that because there were no asymptotic space-like or null regions, de Sitter isometries were all gauge transformations and all of de Sitter space consisted of gauge copies of a single static patch. The physical argument was that generic asymptotic data from the past or future extrapolated to a singularity that did not allow one to continue to the future or past of de Sitter space, so that specifying both past and future de Sitter asymptotics restricted one to a finite dimensional Hilbert space. Gary Horowitz told me this sounded plausible, and Bousso and Freivogel eventually proved a theorem along these lines. Witten seemed to partially concede in his own paper on quantum gravity in de Sitter space.

The conjecture that de Sitter space has a finite number of states gets reinforced by the computation of Coleman DeLuccia tunneling amplitudes between two de Sitter spaces, which obey a rule of detailed balance appropriate to tunneling in finite systems. This leads to a controversial reinterpretation of the theory of eternal inflation, with by far the most sensible conjecture about the meaning of transitions to Big Crunch cosmologies. It turns the search for a measure on an ill defined infinite probability space into an exercise in the non-equilibrium statistical mechanics of finite quantum systems.

In the meantime, Fischler and I continued to develop these ideas into a general formalism for dealing with cosmological space-times quantum mechanically. Eventually we gave it the name Holographic Space-time. The central idea was to generalize the conjecture about de Sitter space to a conjecture about the density matrix of every finite area causal diamond in "empty space". Bousso pointed out to us that he had made a similar conjecture in his review papers. This led us to a model of the maximally entropic universe, which is a flat Friedmann-Robertson-Walker universe with equation of state in which the pressure equals the energy density. If we had been more clear thinking, we would have realized that this model in fact contradicted our initial ansatz for the density matrix. In fact, to fit the data of that universe, we had to assume that the modular Hamiltonian at any time was that of a one-plus-one dimensional conformal field theory, with central charge that grew like the area of the apparent horizon. We did notice the connection to earlier work of Carlip on black hole horizons, but did not pay enough attention.

And it was only much later that we realized that simply stopping the expansion of the Hilbert space of our model with a fixed conformal field theory gave a description of the flat Friedmann-Robertson-Walker model whose scale factor is the cube root of the hyperbolic sine of three times the time divided by the de Sitter radius, which asymptotes to de Sitter space. This implies that the density matrix of de Sitter space is not proportional to the unit matrix. It does not affect the ability to interpret the de Sitter temperature in terms of a connection between energy measured along a geodesic, and the number of constraints on boundary degrees of freedom.

Fischler and I have always been loath to use the term observer because of all of the nonsensical literature about the connection between quantum measurement theory and consciousness. Instead, the Holographic Space-time formalism has always been based on the Hilbert spaces associated with intervals along time-like trajectories. One thinks of these as the trajectories of detectors that have no back-reaction on space-time geometry. I'll discuss the problem of back-reaction below. The covariant entropy bound implies that for short enough proper time the maximal entropy causally accessible to a detector on such a trajectory is always finite, which means expectation values of operators are computable in a Type II-one von Neumann algebra. The hyperfinite Type II-one algebra is just the limit of a finite number of fermionic creation and annihilation operators. For a black hole of mass one gram in four dimensions, the entropy is approximately ten to the eleventh, so one must be able to do measurements fine enough to distinguish between the infinite dimensional algebra and that of ten to the eleventh q-bits in order to falsify the claim of Holographic Space-time that the Hilbert space is finite dimensional. For the universe as a whole the number of q-bits is about ten to the one hundred and twenty-third and it's clearly impossible to imagine making the distinction. As we'll see, realistic measurements inside a causal diamond cannot actually probe even a finite fraction (parametrically) of the quantum information in the diamond. Thus, we've always assumed finite dimensional Hilbert spaces, which also implies a discreteness of time evolution. We take the discrete interval to be the Planck time.

A key feature of the formalism has always been the Quantum Principle of Relativity (which should have been a clue to the tone deaf that we were talking about "observers"). Any given space-time is described by multiple quantum systems, living in a Hilbert bundle over a manifold of time-like trajectories. The simplest collection of time-like trajectories that form a manifold is the space of time-like geodesics. In examples like anti-de Sitter space one might want to add null-geodesics or accelerated trajectories that reach the boundary. For any two choices of interval along a pair of trajectories in the base space of the bundle there is an overlap region and a causal diamond, unique up to symmetries, of maximal area in that region. We insist that there be a Hilbert space associated with this diamond and unitary embedding of this space into the Hilbert spaces of each individual trajectory as a tensor factor. The density matrices on those two tensor factors must have identical entanglement spectra. This constrains both the time dependent Hamiltonians and the choices of initial state on all fibers of the Hilbert bundle.

This formulation of Holographic Space-time depends on the assumption of a background space-time. The proper way to think about this is using Jacobson's demonstration that Einstein's equations, doubly projected onto an arbitrary null vector so that the cosmological constant doesn't contribute, are the hydrodynamic equations of the area law for causal diamonds. Thus, we view a space-time as a hydrodynamic background, into which we must fit a quantum theory whose hydrodynamics matches that background. The refinement of Jacobson's argument due to Carlip and Solodukhin, generalized from black holes to arbitrary diamonds by Banks and Zurek, shows us how to do this. These authors show that the fluctuations of the conformal factor of the background geometry of the holographic screen of each diamond are the hydrodynamic equations of a one-plus-one dimensional conformal field theory with central charge proportional to the area of the diamond's holographic screen. They derive the area law for the diamond entropy, assuming the modular Hamiltonian is the L-zero generator of that conformal field theory. The Holographic Space-time formalism provides us with a guess for what the conformal field theory is.

2. Locality

Perhaps the most important insight into the relation between the quantum theory of gravity and quantum field theory comes from Fiol's explicit modeling of the entropy reduction of localized objects in de Sitter space. The variables are rectangular matrices, each matrix element being a one-plus-one dimensional massless fermion field, with an ultraviolet cutoff described elsewhere. Localized states are states on which matrices bilinear in the fermions are block diagonal. This prescription reproduces the thermal properties of de Sitter space, but also leads to a model of scattering of particles in Minkowski space in the limit of infinite de Sitter radius. This is an alternative view of the emergence of local physics from a holographic theory of quantum gravity, which shows how locality breaks down at the threshold of black hole formation. It has puzzled me that so little attention has been paid to it.

By using tensor network technology, it can be made compatible with the standard tensor network picture of locality in anti-de Sitter space, on scales larger than the anti-de Sitter radius. Although the original implementation of Fiol's idea was based on the assumption that the density matrix of empty de Sitter space was proportional to the unit matrix, it works just as well, with one proviso, for the Carlip-Solodukhin ansatz. The proviso is that we must give up the requirement that the probability distribution for states that have energy, as measured along a geodesic in the diamond, of order the inverse diamond radius is just given by the ratio of the dimension of the constrained subspace in which they live, to the total. This remains true for those low energy states that can actually be detected on the geodesic, and are well described by quantum field theory, but is not true for most of the states on the boundary.

2.1 The limitations of quantum field theory

There is a widespread folklore in the string theory community that in various "weak string coupling" or "large anti-de Sitter radius" limits, quantum field theory is a good approximation to everything in a model of quantum gravity. This is simply incorrect and has been known to be incorrect since at least the 1970s. If one considers a causal diamond in Minkowski space of proper time T, the field theory Hilbert space one associates with it has a dimension which is the exponential of T times the ultraviolet cut-off mass, raised to the power one less than the space-time dimension. Most of these states have semi-classical gravitational back-reaction that produces black holes with size larger than T. Eliminating those, one is left, generically, with an entropy bounded by something of order the diamond area in Newton-constant units raised to the power of one minus one over the space-time dimension. In particular, one has to omit the area's worth of states that give rise to the ultraviolet divergent state independent quantum-field-theory entanglement entropy of the diamond.

Note by the author: In a pre-existing black hole background, the near horizon states have smaller gravitational back reaction due to the gravitational redshift. This is the reason they've been viewed as candidates for the explanation of black hole entropy. Making this assumption leads to the firewall paradox.

In an important paper, Cohen, Kaplan and Nelson showed that omitting these states did not affect the agreement between quantum field theory and any known experiment. The intuitive reason for this is that experiments are done on near geodesic trajectories for finite proper times, and are relatively unaffected by virtual excitations, in quantum field theory language, far from those trajectories. Cohen, Kaplan and Nelson used a crude correlation between ultraviolet and infrared cutoffs and found that their bounds were close to current experimental limits. More sophisticated cutoff procedures allow more high momentum modes near the experimental trajectory and put experimental probes of the bound further out of reach.

The upshot of these arguments is that we have no phenomenological reason to believe the quantum field theory picture of most of the states on the boundaries of a causal diamond. Since that picture leads to the firewall problem, and does not account microscopically for the large increase in entropy when a low entropy object is dropped onto a large black hole — whereas the Holographic Space-time models referred to above avoid the firewall and do explain the entropy increase — one should simply accept the fact that quantum field theory breaks down, not just at large energy and short distance, but in any situation with large entropy. There are indications from perturbative string theory that this is the case.

3. The back reaction of detectors

According to the modern rules of quantum measurement theory, a detector that has to measure and store the quantum information about a quantum system with M dimensions has to have M independent "collective variables" with the following properties:

• There is a large integer N such that the uncertainties of all of the collective variables scale like one over N. The equations of motion of the detector system relate collective variables to each other, so we can follow histories of these variables with no ambiguity, for times of order N.
• The interference terms in the probability sum rule for different histories of the collective variables are exponentially small, of order the exponential of minus N raised to a power of order one or larger.
• The detector's collective variables must retain their "semi-classical" status over time scales N long enough to do multiple measurements on different bases of the M dimensional system, in order to be able to extract all of the probability amplitudes for different time dependent transitions.

It follows from this that the detector must be a quantum system with a Hilbert space whose dimension is much greater than M. Just the requirement that the detector be a subsystem of a closed finite entropy universe makes it impossible to imagine a detector inside a closed universe that can verify every detail of the quantum model of the universe. However, the features of the quantum theory of gravity discussed above make the problem much more severe than this.

First, all experiments to date have taken place on near geodesic trajectories in a causal diamond. The more complex a detector we imagine on such a trajectory, the more constrained the state of the boundary quantum system it is measuring. Thus, not only can't we imagine a detector with enough collective variables to determine the quantum behavior of the universe, but the more precise and efficient our detector, the less of the Hilbert space of the universe the detector can explore.

Furthermore, the kinds of systems that we know to exist on near geodesic trajectories are those described by quantum field theory, and black holes. We've argued that, if our universe is a de Sitter space with the observed value of the cosmological constant then the log of the dimension of the Hilbert space of the maximal quantum-field-theory detector allowed by theory is about ten to the ninety-second, while that of the universe is ten to the one hundred and twenty-third.

Note by the author: The actual entropy of localized matter in our universe is about ten to the ninetieth and is dominated by the entropy of supermassive black holes in the centers of galaxies.

Black holes have more quantum states for a given entropy deficit on the horizon, but they have many fewer collective coordinates with which to record and store data. So the quantum gravity constraints on detectors inside a de Sitter universe imply that much of the quantum information in the universe is undetectable.

The combination of quantum measurement theory and the constraints of quantum gravity thus imply that no observations in a de Sitter universe can ever test most of the details of a quantum model of the entire universe. Note that we have not yet taken into account the constraint that the detector's collective coordinates survive long enough to make all the required measurements. If we imagine a detector moving along a geodesic, these constraints are very severe. Most time-like trajectories approach the boundary of the causal diamond of any given geodesic in a time of order the de Sitter radius. Quantum fluctuations of the detector's position, and disturbances of its position by gravitational scattering from other localized matter, Gibbons-Hawking radiation and so on will make it deviate from geodesic motion on time scales short compared to those necessary to the collection of the maximal amount of quantum information it can process.

Fortunately, for detectors that belong to local groups of galaxies like our own, geodesic motion is irrelevant. The huge set of quantum states of the local group decohere the trajectory of its center of mass motion through the universe. In principle then, the useful lifetime of such a massive complex detector is limited by the time it takes it to collapse into a black hole. After that point, the information it has collected has "unhappened", it is lost in the detailed quantum microstate of the black hole, and no macroscopic record remains.

4. Conclusions

I list the main take-aways from this article.

• The original claim that the density matrix of empty de Sitter space is maximally uncertain was made independently by myself and W. Fischler in 2000. The most important part of that claim was the identification of the energy of localized states as a count of the number of constrained q-bits in the subspace of fixed localized energy. Fiol invented a fermionic matrix model of this effect, and it leads to an understanding of how local particle physics emerges from a holographic model of gravity in small causal diamonds. This is quite different from the way that locality in the anti-de Sitter part of space-time emerges in the tensor network approach to the AdS/CFT correspondence.
• Operator algebras and Hilbert spaces of finite area diamonds are finite dimensional, and time evolution is consequently discrete, consisting of entangling of larger and larger tensor factors of a large Hilbert space with the rest of the degrees of freedom. The quantum of time is taken to be the Planck scale.
• The maximal uncertainty ansatz is probably wrong and should be replaced by the Carlip-Solodukhin ansatz that the modular Hamiltonian of a diamond is the appropriately cut-off Virasoro generator L-zero of a one-plus-one dimensional conformal field theory living on the stretched horizon of the diamond. This is valid for all causal diamonds in space-times with non-negative cosmological constant. For negative cosmological constant the Carlip-Solodukhin ansatz is valid for boundary anchored Ryu-Takayanagi diamonds, and for diamonds smaller than the anti-de Sitter radius. Diamonds larger than the anti-de Sitter radius must be treated by the tensor network construction and have the modular Hamiltonian of a thermal conformal field theory in higher dimensions. For small causal diamonds or diamonds in space-times with non-negative cosmological constant, the connection between local physics and boundary constraints, and the derivation of the de Sitter temperature from this relation, are compatible with the Carlip-Solodukhin ansatz as long as we restrict our assumptions about the range of validity of quantum field theory in the bulk, in the manner suggested by Cohen, Kaplan and Nelson. A particularly welcome consequence of abandoning the maximal uncertainty hypothesis is that it implies fluctuations of the modular Hamiltonian in inflationary horizon volumes. These give rise to the observed temperature fluctuations in the Cosmic Microwave Background. A model without them would be ruled out by observation.
• Quantum field theory is valid in a causal diamond at most for states of entropy bounded by the diamond area in Newton-constant units raised to the power of one minus one over the space-time dimension.
• The "vacuum" in Minkowski space is a high entropy state, with the entropy coming from zero energy states. This follows by viewing Minkowski space as a limit of de Sitter space, but also as a limit from negative cosmological constant. In the anti-de Sitter case, the Minkowski diamond is a tiny subsystem entangled with the much larger anti-de Sitter background, as seen in the tensor network formulation. In correlation functions the entanglement comes from Witten diagrams in which arbitrary numbers of massless lines are exchanged between operators focussed on the Minkowski diamond, which converge to Minkowski scattering-matrix elements with finite numbers of particles, and operators that create excitations that propagate far from the diamond. The nature of the asymptotic Hilbert space and the existence of a non-perturbative scattering matrix remain open, particularly in four dimensions.
• The joint constraints of quantum measurement theory and quantum gravity imply that no detector in a closed universe can probe most of the quantum information that the covariant entropy bound allows in that universe. Consequently, no model of such a universe with infinite dimensional algebras of observables has any observational meaning, and even finite dimensional models will have a lot of freedom that cannot be verified or falsified by experiment.

For those with endless patience, a fairly complete list of my thoughts on the entropy of states in de Sitter space and their wider implications, full of mistakes and false starts, can be found in the collected references.

Acknowledgments

The work of T.B. was supported by the Department of Energy under grant DE-SC0010008, Rutgers Project 833012. I would like to thank W. Fischler for collaboration over many years. He deserves equal credit for most of the insights about entropy, space-time and locality that we've explored in the past 25 years. The opinions expressed in this paper are my own.

(The numbered citations and the reference list are omitted for length; the complete text is at arXiv:2312.10729.)