Lifshitz theory of the cosmological constant

Lifshitz theory of the cosmological constant

Ulf Leonhardt, Department of Physics of Complex Systems, Weizmann Institute of Science, Rehovot, Israel.

Annals of Physics 411 (2019) 167973.

Abstract

Astrophysics has given empirical evidence for the cosmological constant that accelerates the expansion of the universe. Atomic, Molecular, and Optical Physics has proven experimentally that the quantum vacuum exerts forces — the van der Waals and Casimir forces — on neutral matter. It has long been conjectured (Ya. B. Zel'dovich, 1968) that the two empirical facts, the cosmological constant and the Casimir force, have a common theoretical explanation, but all attempts of deriving both from a unified theory in quantitative detail have not been successful so far. In AMO Physics, Lifshitz theory has been the standard theoretical tool for describing the measured forces of the quantum vacuum. This paper develops a version of Lifshitz theory that also accounts for the electromagnetic contribution to the cosmological constant. Assuming that the other fields of the Standard Model behave similarly, gives a possible quantum-optical explanation for what has been called dark energy.

Keywords: Casimir forces, dielectrics, vacuum fluctuations, dark energy.

1. Argumentation

1.1 Introduction

Einstein introduced the cosmological constant for having the possibility of a static, eternal universe as solution of his field equations of gravity. The cosmological term he wrote there acts as a repulsive force that may counter-balance the gravitational attraction of ordinary matter in equilibrium. Hubble's astronomical observations however, of galaxies receding from each other on average, revealed a different picture: the universe is not static, cosmic distances are expanding with a universal, time-dependent factor. The first derivative of the expansion factor differs from zero and is positive. More recent measurements with supernova explosions and of the Cosmic Microwave Background (CMB) have refined Hubble's results with sufficient precision to determine the second derivative of the expansion factor, that turned out to be positive, too: the expansion of the universe is accelerating. This is only possible if, on cosmological scales, the net force of gravity is repulsive, which gives strong, empirical evidence for the cosmological constant. The analysis of CMB fluctuations has established the currently best quantitative value of the cosmological constant as 1.106 plus or minus 0.023, times 10⁻¹⁸ per square metre, valid for the time the CMB was formed. However, predictions of standard quantum field theory exceed the empirical value by about 120 orders of magnitude. The problem arises from the nature of the quantum vacuum.

The universe, with an average density of approximately 10⁻²⁹ grams per cubic centimetre, is mostly made of empty space, but this cosmic vacuum is thought to be filled with quantum fields in their ground state — the quantum vacuum that may cause the measured cosmological force. (To a good approximation, the universe is spatially flat, such that the first Friedmann equation holds; relating the mass density to the energy density gives the quoted approximate value for a Hubble constant of 2.2 × 10⁻¹⁸ hertz from CMB measurements.) Empirical evidence for forces of the quantum vacuum comes from Atomic, Molecular, and Optical (AMO) Physics. Here they appear as the van der Waals and Casimir forces, and here they agree with theory up to an accuracy on the percent level that is only limited by the experimental precision of the material parameters involved, in contrast to cosmology. Zel'dovich suggested that the quantum vacuum appears on cosmological scales as the cosmological constant. His theory predicts the correct structure of the cosmological term, but a vastly incorrect quantitative value, and so did other theories, or they could not account for the empirically observed forces of the quantum vacuum. Perhaps for want of a more illuminating explanation, the cosmological constant has been called dark energy.

Given the success of the theory of the quantum vacuum in AMO Physics, it seems natural to take a similar approach for calculating the cosmological constant, which is what this paper strives to achieve. The starting point is the observation that a space-time geometry appears as an effective medium to the electromagnetic field. (Gordon's metric was rediscovered several times.) To make this point as simple as possible, assume that three-dimensional space is flat, without curvature, and expands in time (figure 1a) such that distances grow in time by some factor. Now imagine (figure 1b) another space filled with a dielectric medium of refractive index evolving in the same way in time. To electromagnetic waves — light — both spaces appear exactly the same. So, if attention is restricted to the quantum fluctuations of the electromagnetic field, expanding flat spaces are indistinguishable from uniform media with time-dependent refractive indices. Calculations of the zero-point energy and pressure in such media will give the electromagnetic contribution to the cosmological term. Much less is known about the quantum forces of the other fields of the Standard Model — experimentally, nothing at all — but it seems reasonable to assume that their net effect follows suit.

(For most dielectric materials one needs to consider not only the refractive index, but the electric permittivity and the magnetic permeability. Both give the index through their product, but also the impedance of a medium through their ratio; in impedance-matched media and in curved space-time the permittivity and permeability are each equal to the refractive index, which we assume throughout this paper.)

Note that the model of expanding flat space is a realistic approximation for the universe on cosmological scales, above 100 megaparsecs. There space is empirically known to be isotropic and homogeneous. Moreover, the relative contribution of spatial curvature has been reduced by cosmic expansion to a small value, below 0.005 in magnitude, already at the time the CMB was released. Space has become nearly flat, and is expanding with a uniform refractive index.

Figure 1. Cosmic expansion. a: Visualization of the expansion of the universe in a space-time diagram; three-dimensional space is illustrated as a plane in coordinates affixed to the origin, time appears as a third dimension. At the black circle the shown patch of space reaches a cosmological horizon where the expansion velocity equals the speed of light. No communication from beyond that sphere is possible. The waves show the last signal from the outside reaching an observer in the center. b: Representation of the expanding universe as a medium with time-dependent refractive index, illustrated for one-dimensional space in co-moving coordinates. The shades of gray and the bending of the space-time sheet visualize the variation of the index. The white lines mark, in co-moving coordinates, the boundary of the same patch shown in fixed coordinates in a, reaching the horizon.

Mathematically, the geometry of space and time is characterized by the metric that measures the increment of proper time along a space-time trajectory; for light the increment is zero. In particular, the space-time geometry of the expanding, spatially flat universe is described by equation (1): the squared line element equals the squared speed of light times the squared time increment, minus the squared refractive index times the squared spatial increment. Here the speed of light is the speed in the absence of gravity, or, equivalently, for zero electromagnetic susceptibility. Note that the expanding universe distinguishes a global frame, and the metric of equation (1) is written in the corresponding space-time coordinates. In the fictitious electromagnetic analogue of the expanding universe (figure 1b) these are the laboratory coordinates. In cosmology, these coordinates are called co-moving coordinates, because they are constant for observers staying put on geodesics parallel to each other, thus co-moving with the cosmic expansion.

1.2 Lifshitz theory

For calculating the cosmological constant I develop in section 2 a version of Lifshitz theory. The theory, due to Lifshitz, Dzyaloshinskii, and Pitaevskii, has become the well-tested, well-established theoretical tool for predicting and describing experiments with quantum forces. Lifshitz theory is applicable to realistic materials with dispersion and loss. It has predicted, for example, the regime of a repulsive Casimir force that was experimentally verified in quantitative detail. Lifshitz theory has also conceptual advantages over rivalling theories: it starts from a fundamental theorem, the fluctuation-dissipation theorem, and it involves a natural and intuitive renormalization procedure.

Renormalization is necessary, because the energy density and pressure of quantum fluctuations seem infinite in most cases. This infinity must be removed to lay bare the part that does physical work, usually by comparing an arrangement of dielectric bodies at finite distances with the same bodies infinitely apart. The difference in stress on each body gives the physically meaningful force. Obviously, such a procedure applies only to calculations of the forces between bodies, but not inside them. If the entire space is filled with a medium varying in space, or time, as considered here, the medium forms a single dielectric body one cannot take apart. But here also Lifshitz theory offers a natural renormalization procedure. The energy density and pressure is calculated twice for each point: first assuming the actual refractive-index profile and then assuming a uniform medium equal to the local value of the index. As uniform media have no reason to exert any force inside them, the difference, if finite, gives the physically relevant energy density and pressure. Note that it is essential to remove the infinite contribution locally; a hypothetical overall infinite baseline is ruled out by experiments.

Although Lifshitz theory gives a general prescription for calculating the energy density and pressure of quantum fluctuations in media, and excellent agreement with experiment for the van der Waals and Casimir forces between bodies, the quantum force inside bodies was poorly understood. It turned out that the unphysical, infinite contribution depends not only on the local value of the refractive index, but also on its derivatives. (Representing an inhomogeneous medium by infinitesimal, piece-wise homogeneous media does not give a converging Casimir force either.) Otherwise, the difference in Casimir stress is not finite. This problem does not become apparent in regions of constant index — in piece-wise homogeneous media, that is, between dielectric bodies immersed in a uniform background. For calculating quantum forces inside bodies — in inhomogeneous media — progress has been made only recently. This paper builds upon our work on the Casimir force inside dielectrics, and upon the work of others on the quantum theory of light in media that, hopefully, may shed some light on dark energy.

One may pause here and wonder whether the renormalization does not remove the most significant contribution of the quantum vacuum to gravity. The argument goes as follows. While it is acceptable that AMO quantum forces originate from only part of the total energy density and pressure — the renormalized part — gravity perceives everything. On the right-hand side of Einstein's equations stands the total energy-momentum tensor. In the conventional picture of Casimir forces, the total vacuum energy density is the infinite sum of all the zero-point energy densities of the modes involved. While only part of the sum may do mechanical work, all of it should gravitate as mass density.

Yet Lifshitz theory offers also an alternative picture due to Schwinger. The fluctuations of the electromagnetic field originate from the sources of the field. The sources are the quantum-fluctuating charges and currents the fluctuation-dissipation theorem requires to exist in the medium. The source fluctuations propagate as field fluctuations with the classical electromagnetic Green function as propagator. In this picture, renormalization is not the mere extraction of the mechanical energy from the infinite zero-point energy, but the removal of an artefact in the theory: the interaction of each source with itself. Here one needs to take into account the local environment, as the spurious self-interaction depends on it. There is another caveat. Getting a finite result after renormalization is necessary, but not sufficient for obtaining the physically relevant energy density and pressure, as the renormalization may introduce artificial, finite contributions. But here also Lifshitz theory suggests physically motivated, heuristic arguments for the correct renormalization.

So, in Schwinger's picture, the energy and pressure of the cosmological constant is exactly proportional to the AMO vacuum energy and pressure in spatially uniform media with time-dependent refractive index.

1.3 Objections

One may immediately raise three objections against the chances of AMO Casimir theory in cosmology.

First, transforming the time coordinate to conformal time, equation (2) — the integral of the time increment divided by the refractive index — transforms equation (1) into equation (3): the squared line element becomes the squared refractive index multiplying the difference of the squared light-time increment and the squared spatial increment. The metric has become conformally flat, meaning it differs from the metric of flat space-time only by an overall prefactor that may depend on space and time. Light rays, for which the line element vanishes, do not depend on the prefactor of the metric, and so light propagates in conformal coordinates like in empty, flat Minkowski space-time. As Maxwell's equations are conformally invariant this remains true for electromagnetic fields and their fluctuations. (The conformal invariance of Maxwell's equations is easily seen with the help of Plebanski's interpretation of geometries as media: any prefactor in the metric drops out of the constitutive equations.) Since the renormalized vacuum energy and pressure vanishes in uniform, static media, the cosmological constant should be identically zero.

Second, even if the AMO Casimir energy is not zero, it can only depend on derivatives of the refractive index. On cosmological length scales where space is uniform, the index varies on time scales of 10¹⁰ years, but the resulting vacuum force should dominate the dynamics of the universe. How can such a slow variation exert such a significant force?

Third, the cosmological term in Einstein's equations appears like a fluid with positive energy density and negative pressure, equation (4) setting the pressure equal to minus the energy density, whereas the vacuum pressure of the electromagnetic field is related to its energy density by equation (5), the pressure being one third of the energy density, because the trace of the electromagnetic energy-momentum tensor vanishes. (Here is a simple physical argument for equation (5). Pressure is the momentum transfer over an infinitesimal surface in infinitesimal time. For propagation with the speed of light, the momentum is the energy divided by the speed of light. From this follows that the pressure is equal to the part of the energy density transported over the surface, in one specific direction. Assuming all three directions to be equal, we arrive at equation (5).) How can, for positive vacuum energy density, the pressure become negative and equal in magnitude to the energy density? This hypothetical, ultra-strong negative pressure should drive the expansion of the universe. So related to this problem is the question: how can the AMO Casimir force in spatially uniform, time-dependent media become repulsive?

1.4 Cosmological horizons

Let me remove the objections of section 1.3 one by one. Although one can transform the expanding universe to flat Minkowski space-time for electromagnetic fields, I will argue that cosmological horizons remain essential (figure 2). Since Bekenstein's and Hawking's theory of black holes, horizons are known to emit thermal radiation. The transformed space-time is therefore not in a vacuum state, but in a thermal state.

The easiest way of seeing this mathematically is by transforming the co-moving coordinates to another set of spatial coordinates, equation (6), obtained by multiplying the co-moving coordinates by the expansion factor. The new coordinates absorb the expansion factor and thus appear fixed. They are affixed to the origin, the only point where they agree with the co-moving coordinates. To see how the so-affixed coordinates experience expanding space, one transforms the metric, equation (1), with the result, equation (7): the squared line element equals the squared speed of light times the squared time increment, minus the square of the quantity spatial increment minus the Hubble constant times position times time increment. Here the Hubble constant is defined in equation (8) as the time derivative of the refractive index divided by the index itself. Throughout this paper dots denote time derivatives. The Hubble constant is only constant for exponentially varying index, but the term constant is commonly used.

The transformed metric, equation (7), has an interesting physical interpretation: it describes a moving fluid with radially symmetric flow speed, equation (9), the Hubble constant times the radius. The expanding universe thus appears as an outward-moving fluid in the coordinates affixed to the co-moving observer at the origin. As one can shift the origin to any other point, this is also true for all other observers co-moving with the cosmic background. For each and every one of them, the universe flows away with that radial velocity. Equation (9) describes one of Hubble's laws: the expansion velocity grows linearly with growing distance. At some radius, the flow velocity reaches the speed of light. No classical communication from a sphere of greater radius is possible; the radius where the flow speed equals the speed of light defines a cosmological horizon (figure 2).

Figure 2. Cosmological horizon. The expanding universe appears like an outward moving medium around any arbitrary point in space in a coordinate system affixed to that point. The flow velocity follows Hubble's law, equation (9), reaching the speed of light at the horizon. The figure shows typical wave fronts in conformal time and co-moving coordinates. a: Phase fronts of the last waves incoming, against the Hubble flow, from the horizon, reaching the observer in the center. The waves are reflected at the center and move out with phase fronts shown in b. As outgoing waves, they propagate with the Hubble flow and so are free to cross the horizon.

Each co-moving point in space is surrounded by a cosmological horizon, separating the world into an inside and an outside. The inside sphere depends on the co-moving point and encloses different spatial regions for different points. The quantum vacuum, however, is universal; the vacuum cannot possibly adjust itself to all the conflicting horizons, and so it must remain indivisible. The vacuum bridges the dividing horizons, entangling the field in the inside with the field outside of each horizon in Einstein-Podolsky-Rosen states. To an inside observer, the pure entangled state appears as a statistical mixture with maximal entropy: a thermal state. In moving fluids, the temperature is given by the velocity gradient at the horizon. For cosmological horizons, I obtain the Gibbons-Hawking temperature, generalized here beyond de Sitter space, equation (10): Boltzmann's constant times the temperature equals the reduced Planck constant times the Hubble constant, divided by two pi. I deduced the Gibbons-Hawking temperature with the help of the coordinates affixed to a co-moving observer, equation (6), but the quantum radiation of cosmological horizons does not disappear in other coordinate systems, only the radiation temperature changes when the measure of time is changed. In conformal time one gets equation (11): Boltzmann's constant times the conformal temperature equals the reduced Planck constant times the time derivative of the refractive index, divided by two pi — as the ratio of frequency and temperature must remain invariant, with the frequency changing by a factor of the index according to equation (2).

1.5 Equivalence principle

In conformal coordinates space-time appears flat to electromagnetic waves, but it is not empty: it is filled with thermal radiation. Yet for a Hubble constant in the order of one per 10¹⁰ years, that is 3 × 10⁻¹⁸ hertz, the Gibbons-Hawking temperature lies around 2 × 10⁻²⁹ kelvin. How can such a tiny temperature compete with the 2.7 kelvin of the CMB? From Lifshitz theory in ordinary media one will certainly not expect a significant figure for the Casimir force in media varying on the time scale of 10¹⁰ years. What can be different between ordinary media and space-time? Did I not argue that they are the same?

The equivalence principle makes all the difference. According to the equivalence principle, the space-time geometry applies to everything equally, not only to the electromagnetic field, and for electromagnetic waves not only to a small range of frequencies — in contrast to ordinary media. The response of ordinary media varies with frequency and vanishes for frequencies beyond the atomic scale, whereas space-time should act equally on the entire spectrum — up to the Planck scale. Close to the Planck scale space-time is expected to become dispersive, violating the equivalence principle. We found in our previous work that the renormalization of the Casimir force relies critically on the fact that the response of physical media drops sufficiently fast with frequency. Without this, the Casimir stress in planar, inhomogeneous media would become infinite in general. As this attempted infinity appears in renormalization, it is not influenced by an ordinary thermal background like the CMB. But in renormalization, the vast spectrum seen by gravity may turn the tiny Gibbons-Hawking temperature of the expanding universe into a quantity deciding its fate.

Let me estimate what it takes for the vacuum energy to have a significant influence on the cosmic dynamics. Einstein's equations in a homogeneous and isotropic space are reduced to the two Friedmann equations. In flat space, the first Friedmann equation relates the total energy density to the Hubble constant, equation (12): the squared Hubble constant equals eight pi times Newton's gravitational constant times the energy density, divided by three times the squared speed of light. Suppose that the energy density is essentially given by the vacuum energy density; the general case is considered later in section 4. Being ultimately a quantum energy, whether directly or indirectly via the Gibbons-Hawking temperature, the vacuum energy density must be proportional to the reduced Planck constant. In order to influence the dynamics following equation (12), it should scale as the squared Hubble constant. Having the physical dimensions of an energy density implies that it should be equal to the reduced Planck constant times the squared Hubble constant divided by the speed of light, times a dimensionless constant divided by the square of a length. If I set this length to the order of the Planck length, equation (13) — the square root of the reduced Planck constant times the gravitational constant divided by the cubed speed of light — the first Friedmann equation is satisfied. So, if the renormalized energy density of the quantum vacuum diverges with an inverse length squared in an ideal space-time honoring the equivalence principle indefinitely, the vacuum energy becomes cosmologically relevant for a realistic space-time respecting the equivalence principle only up to the Planck scale.

In planar media, where the index varies only in one direction in space and is otherwise constant, the Casimir stress diverges logarithmically with the frequency cut-off. Media with index constant in space but varying in time are different though, due to the existence of horizons and, as will be seen in section 2, causality.

1.6 Trace anomaly

The second Friedmann equation follows from the conservation of energy and momentum. In a spatially isotropic and homogeneous universe, all energy and matter must move on average with the expanding cosmic background. For a fluid of given energy density and pressure following adiabatically the expansion of the universe, entropy must be conserved. The conservation of entropy is part of relativistic fluid mechanics; in the absence of any net transport relative to the cosmic background it becomes the only non-trivial aspect of energy-momentum conservation. One obtains from thermodynamics the second Friedmann equation, equation (14): the time derivative of the energy density equals minus three times the sum of energy density and pressure, times the Hubble constant.

The pressure of the electromagnetic vacuum is given in terms of its energy density by the equation of state, here equation (5). The energy density must be a function of derivatives of the refractive index, equation (15), as the renormalized vacuum energy of a uniform medium vanishes. The dependence on derivatives implies, however, that the quantum vacuum is not adiabatic. The second Friedmann equation, obtained from adiabaticity, combined with equations (5), (8) and (15), would give equation (16). This equation needs to be satisfied for all possible refractive-index histories, for otherwise it would define a differential equation for the index in conflict with the first Friedmann equation. Equation (16) is satisfied for all such histories only when all the terms in front of higher time derivatives of the index vanish, from the highest to the first derivative, and when the derivative of the energy function with respect to the index equals minus four times the energy density over the index — so the energy density varies as the inverse fourth power of the index, which gives the standard equation of state for radiation in the expanding universe. There is no room for maneuver to include derivatives.

The quantum vacuum violates adiabaticity and hence energy-momentum conservation. Wald discovered the root of the problem: the lack of reciprocity in the renormalization procedure. In Schwinger's picture of Lifshitz theory, the problem takes the following form. Each source is split into an emitter and a receiver infinitesimally apart. The self-interaction of the source depends on the local environment of the emitter, but not on the environment of the receiver. Emitter and receiver are not reciprocal, which — using non-technical language — causes an imbalance in recoil that appears as an additional energy and pressure. In technical terms, the lack of reciprocity in renormalization causes a trace anomaly.

Suppose, for simplicity, that the total energy density and pressure are solely given by the quantum vacuum, including the recoil imbalance (trace anomaly). If I write equation (17) — the total energy density as the vacuum energy density plus a further energy density, and the total pressure as the vacuum pressure minus that same quantity — the right-hand side of the second Friedmann equation is not affected, but the left-hand side gets an additional term taking care of energy-momentum conservation: that quantity is the missing recoil energy density. The notation is suggestive. In equation (17) the additional energy density is accompanied by a pressure of equal magnitude but opposite sign, exactly like in equation (4) the pressure of the cosmological constant.

Let me represent the vacuum energy density by equation (18), which writes it as a dimensionless constant times a quantity that is a function of the refractive index and its derivatives with the dimension of a frequency squared, carrying a prefactor built from Newton's gravitational constant and the speed of light. Differentiating the first Friedmann equation, equation (12), with respect to time, and applying the second Friedmann equation, equation (14), I obtain an equation of motion for the cosmic expansion driven by the quantum vacuum, equation (19): the time derivative of the Hubble constant equals four times the dimensionless constant times that quantity. This establishes a differential equation for the expansion factor, as both the Hubble constant and the quantity depend on the index and its derivatives. Furthermore, I get directly from the first Friedmann equation the cosmological energy density and hence the cosmological constant, equation (20): three over the squared speed of light, times the squared Hubble constant plus two thirds of the dimensionless constant times the same quantity.

Given a solution of the dynamics, equation (19), the cosmological constant is determined. As argued in section 1.5, the quantum vacuum dominates the dynamics if the vacuum energy density diverges with the inverse square of a length set to the order of the Planck length. In this case, the dimensionless constant lies in the order of unity.

1.7 Results

In section 2 I calculate the renormalized electromagnetic vacuum energy density in spatially uniform, time dependent media. I find equation (21): the quantity in question is the third time derivative of the reciprocal Hubble constant, plus the Hubble constant times the second time derivative of the reciprocal Hubble constant. In section 2 I also express the dimensionless constant in terms of the cut-off for the electromagnetic contribution to the vacuum energy.

For the other fields of the Standard Model, the quantum noise will be linear, even for non-Abelian fields with non-linear field equations. Massive fields are known to modify the standard Casimir force, because their amplitudes decay in propagation, reducing the reflection amplitude the force relies on, but the renormalization of section 2 is local and hence should not be affected. Therefore it seems reasonable that the principal structure of the result, equations (18) and (21), extends beyond quantum electromagnetism, but with a different dimensionless constant taking into account the sum of the contributions of the other fields of the Standard Model.

As expected, the quantity and hence the vacuum energy density depends on derivatives of the refractive index, up to fourth order, according to the definition of the Hubble constant. Equation (19) shows that, in the absence of any other energy and matter, one can multiply the index by any constant scale factor and have the same dynamics. So the evolution of the spatially flat, empty universe is independent of its size, which is natural, as there are no other length scales involved. (The inverse Planck length squared in the cosmological energy density compensates for the natural constants in the Friedmann equations such that the equation of motion in flat space does not depend on a length scale.) The situation is different in curved space, but equations (18) and (21) for the vacuum energy remain the same, as I show in section 3.

Equation (21) implies that the cosmological constant of equation (20) is no longer constant, except in de Sitter space where the Hubble constant is truly constant. Here the universe is expanding exponentially. In general, the cosmic expansion creates the vacuum energy that, in turn, corrects the expansion. In exponentially expanding de Sitter space, no correction is required; de Sitter space is a consistent solution of Einstein's equations of gravity and quantum field theory.

How does the interplay between the cosmic expansion and the quantum vacuum react to small perturbations of the exponential expansion in flat space? Consider equation (22), writing the reciprocal Hubble constant as a constant reciprocal rate plus a small perturbation. In equation (19) write the time derivative of the Hubble constant as minus the squared Hubble constant times the time derivative of its reciprocal, linearize in the perturbation using equation (21), and integrate the result in time, absorbing the integration constant in the choice of the constant rate. One arrives at the equation of a damped harmonic oscillator, equation (23): the second time derivative of the perturbation, plus the constant rate times its first time derivative, plus the squared constant rate divided by four times the dimensionless constant, times the perturbation, equals zero — with damping rate one half the constant rate for the amplitude of the perturbation. Small perturbations of de Sitter expansion are damped and so corrected for by the quantum vacuum, assuming of course that the vacuum dominates the cosmic expansion.

Lifshitz theory thus predicts that the cosmological constant is not constant, but a dynamical quantity similar to quintessence. Perturbations of it should last in the order of the Hubble constant. There are indeed some indications from astronomical observations that it has varied. The directly observed Hubble constant from galactic cepheids differs from the calculated Hubble constant using the cosmological constant from CMB measurements by 6 per cent, which suggests that the cosmological constant at the time of the formation of the cosmic background radiation was different than the present one.

1.8 Methodology

Figure 3 illustrates the physical assumptions behind the mathematical method applied in the calculations of sections 2 and 3.

First, in order to identify the self-interaction of each source to be subtracted in renormalization, one imagines each point in space and time as being split into two: an emitter and a receiver. This is known as the point-splitting method.

Second, in the self-interaction the emission depends on the local environment of the emitter. In our previous work we found that, in planar media, one should take into account the local value and the derivatives of the refractive index up to second order. Here I assume the same for time-dependent media: in the self-interaction the index is set to a quadratic function around the time of emission.

Third, there is an important difference between point-splitting in space and event-splitting in time: causality. While we can go back and forth in space, we cannot do so in time; the time of emission must precede the time of reception. Section 2 shows that causality, combined with the second-order expansion of the local environment, causes a subtle discontinuity in the renormalization that produces a divergence of the energy density with an inverse length squared. As argued in section 1.5, this singularity allows the quantum vacuum to influence the dynamics of the universe. It naturally comes from causality.

Fourth, another important difference to the planar case is the existence and radiation of cosmological horizons. I argued in section 1.4 that horizons are necessary for the renormalized vacuum energy of uniform space to be different from zero, and this also follows naturally from the theory of section 2 without making additional assumptions.

Fifth, my starting point is the same as in Lifshitz' original paper: the fluctuation-dissipation theorem. In the context of quantum forces the theorem relates the fluctuations of the electromagnetic field to the dissipation during propagation, depending on temperature. In order to define a temperature, one needs a Hamiltonian. In media with time-dependent refractive index, a Hamiltonian exists only for the free propagation in conformal time. The horizon temperature however depends on time. In the fluctuation-dissipation theorem one needs to identify a definite temperature and hence a definite time, even in the limit of the infinitesimally close emission and reception time taken in the point-splitting method. It appears natural to assume that the temperature should be taken at the conformal time exactly between emission and reception (figure 3).

Figure 3. Methods. Points in space and time in conformal coordinates are split into an emitter (white dot) and a receiver (black dot) infinitesimally close to each other. Causality requires that emission precedes reception. The cosmological horizon (black lines) generates thermal radiation influencing the propagation of field fluctuations between emitter and receiver. The radiation consists of superpositions of incoming and outgoing waves. The intensity pattern (level of brightness) of one of such waves is shown.

This paper does make assumptions and extrapolates Lifshitz theory vastly beyond the experimentally tested validity range of AMO Casimir physics, but the assumptions are grounded in proven physical principles, are not specific to the cosmological constant, and some if not all are experimentally testable, if not directly then in laboratory analogues. For example, the concepts from the Casimir theory of planar media have physical consequences in the aggregation in liquids where they can be tested. The theory of this paper should and can be confronted with empirical data, as it does make quantitative predictions. The paper is conservative in the physics — no new fields are introduced to explain the cosmological constant, but rather new concepts in fields as old as quantum electromagnetism.

(Section 2, Calculation — quantum electromagnetism in media, point splitting, the Green tensor and the renormalization that yields equation (21) — and section 3, which extends the result to spaces of non-zero spatial curvature, together with appendix A on light in de Sitter space and appendix B on the Green tensor, are omitted for length; the complete text is at the source.)

4. Conclusion

Lifshitz theory in homogeneous and isotropic space with time-dependent refractive index predicts the cosmological energy density of the quantum vacuum in equations (18) and (21). The trace anomaly of the vacuum energy gives the cosmological constant. Here the cosmological energy density appears as a contribution to the total energy density and pressure in addition to those of matter and radiation, and those of the quantum vacuum itself, as set out in equation (97): the total energy density is the matter energy density plus the vacuum energy density plus the cosmological energy density, and the total pressure is the matter pressure plus one third of the vacuum energy density minus the cosmological energy density.

From the Friedmann equation in spaces of negative, zero and positive curvature with a given radius, equation (98), and the conservation of energy and momentum expressed in equation (14), follows the equation of motion for the universe on cosmological scales, equation (99): the time derivative of the Hubble constant, minus the curvature term, equals four times the dimensionless constant times the quantity of equation (21), minus eight pi times Newton's constant over the squared speed of light times the sum of the matter energy density and matter pressure. The dimensionless constant depends on the cut-off length, in relation to the Planck scale, and the effective number of fields involved.

According to the Lifshitz theory developed in this paper, the vacuum energy and the associated cosmological constant are dynamical quantities. (The theory of quintessence considers the cosmological constant as a dynamical field as well, but it does not include the physics of the quantum vacuum.) The vacuum energy density responds to the evolving universe as described in equations (18) and (21). From the Friedmann equation (98) with equation (97) as equation of state follows equation (100) for the cosmological constant, three over the squared speed of light times the squared Hubble constant, plus the curvature term, plus two thirds of the dimensionless constant times the quantity of equation (21), minus the matter contribution.

In turn, the energy density of the quantum vacuum acts on the evolution of the universe as described in the equation of motion, equation (99). The vacuum energy appears as a correcting force to deviations from exponential expansion according to equations (22) and (23). Pure exponential expansion in flat space does not require any quantum correction.

The theory does not predict a specific cosmological constant, as it depends on dynamics, which implies that it may have had different values. There are in fact two phases of cosmic expansion: one is measured — the recent phase — and one is conjectured, the inflation of the early universe where the Hubble constant and hence the cosmological constant was much larger. The theory of this paper accounts for both phases without requiring additional inflaton fields, although it is not yet clear how the inflationary phase ended and settled to the more sedentary pace of the recent era.

This paper unifies for the first time the proven AMO physics of van der Waals and Casimir forces with the cosmological constant, following Zel'dovich's vision with insights and tools from transformation optics and modern quantum optics. The theory still depends on one parameter — one may say one constant is traded for another — but it also includes variations of the cosmological constant and inflation. Moreover, the parameter has a physical meaning: it is given by the characteristic length near the Planck scale where the equivalence principle ceases to hold, and the effective number of fields involved. Its precise value cannot be predicted at present, but astronomical observations may infer it by fitting the measured expansion of the universe to the equation of motion, equation (99). Observations of the universe on the largest scale may thus probe the smallest scale of Nature.