"There are more things in heaven and earth Horatio, than are dreamt of in your philosophy."
Hamlet, Act 1, Scene V

There are undoubtedly times when astronomers, physicists and cosmologists are gripped by the feelings of awe and amazement that were expressed by Shakespeare's Hamlet on seeing his father's ghost. Over the last century, the frontiers of the known universe have been pushed outwards, our scientific understanding of the underlying physical laws has been revolutionised, and the constant development of telescopes and instrumentation has produced a wealth of new observations and fresh theoretical challenges.

A major revolution in our comprehension of the universe took place in the 1920s. The American astronomer Edwin Hubble demonstrated in 1924 that the vast collection of stars closest to us--the Milky Way--was just one of a large number of such galaxies separated by immense expanses of space. We now know that our galaxy is one of about one hundred billion galaxies visible through our telescopes each with several hundred billion stars.

Even more startling was the discovery announced by Hubble in 1929. A systemic examination of the composition of the light from galaxies revealed a similar structure to that of nearby stars with one significant difference--in every case the telltale black lines in the visible spectrum, indicating the presence of different chemical elements, were shifted towards the red end of the spectrum. The only explanation for the "red shift" was that the galaxies--all of them--were moving away from us at high velocity. Furthermore, the more distant the galaxy, the larger the "red shift" and therefore the greater the speed it was receding.

Hubble reached the inescapable conclusion that the universe was expanding. So entrenched had been the view that the universe was essentially static that Albert Einstein in formulating this theory of general relativity in 1915 arbitrarily introduced a so-called cosmological constant as a sort of "anti-gravity" to prevent the universe from collapsing under the influence of gravity and maintain its steady state.

One possible implication for the expanding universe was that it had originated in a colossal big bang, which sent matter and energy in all directions at high speeds. The "Big Bang" theory, first elaborated through the solution of Einstein's equations, has since been further refined and confirmed by observation. In 1965, two American physicists, Arno Penzias and Robert Wilson, detected a low level background microwave radiation from all parts of the sky--the "afterglow" of the Big Bang predicted by theory.

Now it appears that cosmology and physics may be about to undergo a new upheaval. Observations published last year by two groups of astronomers present a radically different view of the universe--one, which is not only expanding but expanding at an accelerating rate. The new data challenges one of the main predictions of the "inflation" theory--the most widely accepted variant of the "big bang"--that the universe should be "flat," neither expanding forever, nor contracting into the so-called "big crunch".

The findings are the result of more than a decade of painstaking work to develop a method of accurately measuring the distance of far-off galaxies using a certain type of exploding star or supernova. By more exactly determining the distance of galaxies billions of light years from us, the scientists had expected to find the rate at which the universe was slowing down. Instead, to their amazement, the data revealed the distant galaxies were hurtling away at a greater and greater rate.

One possible explanation lies in resurrecting the cosmological constant, which Einstein had famously abandoned as his "biggest blunder" when it became clear in the 1920s that the universe was not static but expanding. Now theoretical physicists are considering reintroduction of Einstein's "anti-gravity" term as a means of explaining the more rapid than expected expansion.

So astonishing was the discovery that a special conference of leading cosmologists was convened at the Fermilab in the US last May to discuss the findings and to thrash out the theoretical implications. Not all the participants at the conference entitled "The Missing Energy in the Universe" accepted the results. In a highly unusual move, a vote of those scientists present was taken--40 out of the 60 recognised the validity of the new findings.

In December, the US-based Science magazine conferred their "Breakthrough of the Year for 1998" on the two teams of astronomers--the Supernova Cosmology Project headed by Saul Permutter at the Lawrence Berkeley Lab in the US, and the High-Z Supernova Search Team led by Brian Schmidt of the Mount Stromlo and Siding Spring Observatories in Australia. Both groups involve the international collaboration of researchers from many countries, including the US, Australia, England, France, Germany and Sweden, and the coordination of observational facilities around the world.

The January issue of Scientific American published a special report entitled "Revolution in Cosmology" devoted to explaining the significance of the findings and discussing alternate theoretical explanations. In introducing the feature, the magazine stated: "Cosmologists thought inflation theory could explain all the basic processes that shaped the universe--until new observations violated a central prediction. For the past year, theorists have scrambled to make sense of the latest data. Either the universe is dominated by a bizarre form of energy... or our universe is just one strangely curved bubble of space-time in an infinite continuum."

Ingenious observational techniques

To determine whether the expansion of the universe is speeding up or slowing down requires an accurate measurement both of the velocity and distance of far-off galaxies. The red shift provides an accurate determination of velocity but the measurement of distances in the order of billions of light years has proven elusive. In the past, astronomers have simply assumed that there was a direct relationship between the red shift of a galaxy and its distance from us--the ratio of the two is known as Hubble's constant. The relationship has been verified for relatively close galaxies but not for more distant ones.

Since the early 1970s, astronomers have been searching for a method of more accurately determining large distances using a so-called "standard candle". The method appears relatively simple--the further away a light-emitting object, the dimmer it appears. If one can measure its brightness accurately then it is possible to calculate the distance--providing one knows its intrinsic brightness, that is the brightness measured at its origin. The great difficulty lies firstly, in finding an astronomical object that can be seen at such huge distances and secondly, in determining its intrinsic brightness, which can vary greatly.

Looking for a "standard candle," therefore, becomes the hunt for an extremely bright astronomical object with virtually identical light emission behaviour wherever it is found in space. If one finds such a "standard candle" then its inherent brightness can be determined for nearby samples and thus used to measure the distance to its more far-off kin.

Astronomers initially considered quasars as possible candidates. These strange immensely energetic sources, first discovered in the 1960s, are now thought to be huge black holes sucking in vast amounts of gas, dust and stars. Quasars are certainly visible at great distances but were found to be far too variable to serve as distance measuring markers.

Attention turned to supernovae--relatively rare stellar explosions brighter than the light from a billion suns. A supernova explosion occurs after a star has used up all its nuclear fuel and is compressed by the force of gravity to densities a million times greater than ordinary matter. Many simply fade away. A few begin to swallow up matter from nearby space, growing denser and denser until they erupt in cataclysmic thermonuclear explosions.

Although exploding stars are also highly variable, astronomers have found that one type--the Ia supernova--is relatively constant in its behaviour. It is readily identified from its spectrum, particularly the absence of hydrogen lines. Moreover, the brighter a Ia supernova the longer it lasts. So by carefully studying the duration of the explosion, scientists can deduce its inherent brightness within an accuracy of 12 percent.

The problem is to find Ia supernovae, and early enough in their cycle so that their duration and other characteristics can be accurately measured. On average, two or three such supernovae explode roughly every 1,000 years in a galaxy. Their brilliance reaches a peak in about three weeks then fades away over a number of months. Scientists have had to develop highly sophisticated techniques to monitor thousands of distant galaxies at once and rapidly identify Ia supernovae so that their brightness can be tracked by major telescopes around the world.

Furthermore, astronomers have had special difficulties in gaining access to the world's largest telescopes and the Hubble telescope. As Saul Permutter, head of the Supernova Cosmology Project, put it: "It was a chicken and egg problem. To get telescope time, you had to guarantee you were going to find a supernova. But without time on a major telescope, it was impossible to show that they were there, and that we could find them."

Both groups developed highly ingenious techniques that would virtually guarantee that exploding stars could be found on demand. By using specially designed electronic light detectors on large telescopes, a broad swathe of the night sky could be quickly and accurately imaged. A single exposure with the Big Throughput Camera used on the four-metre Blanco Telescope at Cerro Tololo Inter-American Observatory in Chile creates a picture of about 5,000 galaxies in 10 minutes.

By taking and comparing exposures of the same area of sky a few weeks apart, sudden changes in the brightness and thus possible Ia supernovae can be identified. The whole process has been automated through the use of digital cameras, which count the number of photons in each faint object. Computers are then used to "subtract" the first image from the second, removing all but the images of objects, which have varied markedly in brightness during the period. Computers also automatically correct for different atmospheric conditions and different image size. A final examination with the human eye is necessary to distinguish possible Ia supernovae from other objects such as variable stars, quasars and asteroids.

The technique has been successfully used to identify a number of Ia supernovae with high red shifts. The best images have come from the Hubble Space Telescope, which has been able to distinguish the exploding star from its host galaxy. The Supernova Cosmology Project has fully analysed 42 of the more than 80 Ia supernovae it has discovered. The red shifts or Z-values vary from 0.18 to 0.86--a Z-value of 0.5 indicates that the object is being viewed as it was about one third of the way back in time to the initial Big Bang. The High-Z team submitted a paper last March based on the discovery of 16 Ia supernovae.

A challenge to present cosmological theories

Once the data was assembled an unexpected picture of the universe emerged. As astronomers Craig Hogan, Robert Kirshner and Nicholas Suntzeff commented in their Scientific American article: "The big surprise is that the supernovae we see are fainter than predicted even for a nearly empty universe... Taken at face value, our observations appear to require that expansion is actually accelerating with time. A universe composed only of normal matter cannot grow in this fashion, because its gravity is always attractive. Yet according to Einstein's theory, the expansion can speed up if an exotic form of energy fills empty space everywhere... Once we admit this extraordinary possibility, we can explain our observations perfectly, even assuming the flat geometry beloved of theorists."

So challenging are the implications of these observations that astronomers have had to exhaust other, more prosaic explanations for the fact that the supernovae are 25 percent dimmer than expected. A number of possibilities have been considered and ruled out: that the view of the supernovae is obscured by cosmic dust; that distant supernovae behave differently from nearby ones; and that "gravitational lensing," caused by the bending of light rays around massive objects like galaxies, could cause the dimming.

 

Robert Knop, a member of the Supernova Cosmology Project, explained: "We are now searching for more supernovae with high red shifts in order to get more information about the early universe. But we are also looking for supernovae with low redshifts--nearby supernovae--to make sure that young and old type Ia supernovae are essentially the same, and make for dependable standard candles. We want to be sure we aren't being fooled by interstellar dust dimming the supernovae, or that stellar explosions weren't somehow weaker in the distant past. So far we haven't found anything to shake our confidence, but this is such an unexpected discovery that we'll keep looking for loopholes."

The findings are reinforced by the fact that two teams using different techniques have independently come to the same conclusion. But there are also deeper reasons for accepting that the universe is expanding faster than predicted by the present models. A number of discrepancies have been accumulating in recent years between the theoretical predictions of cosmology and astronomical observation.

The first concerns the age of the universe. If the known universe is the product of the Big Bang then it possible to use present data to calculate backwards and estimate the time that has elapsed since the initial explosion. The result can be checked against the calculated age of astronomical objects--all of which should be younger than the universe itself. Hubble's first figure of two billion years was completely at odds with the age of our planet Earth, known to be about four billion years old.

The theoretical calculations have since been developed and refined, putting the age of the universe at around 10 billion years. But certain globular clusters of stars appear to be older and even according to recent reassessments may have an age of at least 10 billion years--a very tight fit between theory and observation. But if the universe were expanding at an accelerating rate, then the age of the universe would be older than previously thought and tally more precisely with information about the globular clusters.

The second relates to the total mass of the universe. If as the standard inflationary theory predicts, the universe is flat then a calculation can be made of its mass. The difficulty is that a tally of the visible matter in the universe amounts to at most 10 percent of what is theoretically required. In recent years, astronomers have been engaged in a search for the missing 90 percent--so-called "dark matter" either in the form of burnt-out stars and black holes, or an abundance of exotic, previously unobserved, nuclear particles such as WIMPS or weakly interacting massive particles.

None of the postulated "dark matter" has accounted for the missing mass. More recent investigations involving huge clusters of galaxies also suggest that the density of matter in the universe is too low to meet theoretical requirements. According to Lawrence Krauss in his Scientific American article "Cosmological Antigravity": "These many findings that the universe has too little matter to make it flat have become convincing enough to overcome the strong theoretical prejudice against this possibility. Two interpretations are viable: either the universe is open, or it is made flat by some additional form of energy that is not associated with ordinary matter".

Either alternative raises fundamental new challenges to theoretical physics and cosmology. If there is a cosmological constant or new form of energy then physicists need to explain its existence. In the past, attempts to combine Einstein's general theory of relativity with quantum mechanics have raised the possibility of a non-zero cosmological constant. But its size was so immensely large that the consequent warping of space would make it impossible to see objects in front of one's nose let alone distant galaxies. Various possibilities are being examined to explain a far tinier value.

In their Scientific American article "Inflation in a Low-Density Universe," two theoreticians Martin Bucher and David Spergel examine the other option--that the universe is "open," a term referring to its particular curved geometry. They attempt to modify the standard inflation theory to provide for the possibility that the universe is not flat but open. Their conjectures lead to the conclusion that the known universe is just one self-contained "bubble universe" of many in an infinity of time and space. Perhaps even more remarkably their calculations suggest a possible means for testing which of the two theoretical options is more likely by examining detailed structure of the cosmic background microwave radiation. Two satellites with microwave equipment sensitive enough to make the necessary observations are due for launching--the Microwave Anisotropy Probe by NASA next year, and its European counterpart, Planck, in 2007.

Whatever the ultimate consequences of the latest observational findings of distant galaxies, the results have certainly opened up a new phase in the development of cosmology and given a further spur to theoretical physicists to develop and unify existing theories.