Brian Schmidt: What is dark energy?

In this edition of Titans of Science, Nobel laureate Brian Schmidt discusses how his work led to the discovery of our universe expanding at an accelerated rate, and how dark energy was theorised to be responsible...

Chris - We understood gravity pulls things together. But if the universe is growing and the faster it goes, the faster it grows, which is what you effectively saw, you had to come up with a way to explain that. So was there some pushback or were people saying, well how, Brian, do you explain this possibly that out of empty space comes some way for the universe to fuel its own growth faster?

Brian - So like most good things in cosmology, Einstein had actually already come up with the answer. So he had realised in 1917 that his equations of gravity allowed something he called the cosmological constant to be there. He thought of that as a possibility because in 1917 when he was looking at his equations for the first time, he realised they indicated that the universe would be in motion and he thought the universe should be static. And this was a way he was going to balance out the effects of gravity because this term, it turns out, naturally balances gravity. It actually makes gravity and the things in the universe that are full of this cosmological constant repel as opposed to attract. Now in 1998 when we made this discovery that the universe seemed to be accelerating, we knew immediately we could use a cosmological constant. And it turns out the amount of cosmological constant we needed was an interesting amount because it brought the total amount of stuff in the universe to that magical number where the universe is neither curved away from itself or onto itself where it's flat. And that's a magical number because the theorists working through something called inflation were convinced that that was the only sensible shape for the universe, was for the universe to be flat. And this fixed that issue. So rather than being highly sceptical, many of the community were actually kind of ebullient in saying, this must be the answer. It fixes everything. It makes the age of the universe right again, it causes the universe to be flat and all these things. It had many, many good things at solving the issues of cosmology at the time.

Chris - You still had to explain the mechanism, what is causing that expansion. And that presumably is where dark energy comes in.

Brian - Yeah. So dark energy is the word we use to describe the cosmological constant or anything else like the cosmological constant that might be there. Now the cosmological constant you can think of as being energy that is in every part of space always. It never changes. It's always there. And that characteristic is what makes gravity push rather than pull on it. So dark energy, the cosmological constant, was our solution. As I said, we call it dark energy now just in case it's changing a little bit over time and allowed. But whatever it is, it looks an awful lot like what Einstein developed all those years ago. But we have no idea why it's there. It just seems to be there at this point.

Chris - Researchers have been looking at the density of the dark energy. There's a project which is studying enormous swathes of space and distant space to work out the density of the dark energy in those different places. If it was just fixed, it would be relatively easy to explain. They're saying that they look in different places and at different stages of the universe and dark energy isn't just pushing with the same amount, it's density is changing over time. So is that a spanner in the works then?

Brian - So this is very recent work that's gone on up to this point. You know, from 1998 until 2023, every measurement had sort of shown that it hadn't changed its density at all. It seemed to be absolutely constant back in time. Over the last nine months there's been a few experiments that are getting better and better data that seem to see hints that it is changing back in time. That would be exciting at some level because it would be something different than Einstein's cosmological constant. It's not what he predicts, it's got to be something else. And one of the problems with Einstein's cosmological constant is that it kind of generically emerges from a whole bunch of different types of theories and it's almost impossible to test therefore. But if we see something that's a little different from it, that is much more peculiar, probably, to the correct theory. And so I think we'll make it easier to understand what's going on. But I need to caution everyone, these are very preliminary findings at this point. These are very hard measurements and they're great teams that have done it. But I'd like to see over the next five years we're going to get a lot more data and I want to see if this answer of it changing over time persists.

Chris - Because the implications of it changing over time are that its effect is being tweaked somehow, something is changing the potency of dark energy and therefore the outcome of its presence. So that would change the behaviour of the universe. So it isn't just going to be a case of the universe grows and grows forever. It could alter its its evolution as it evolves

Brian - Indeed. If we want to understand the future of the universe, so up to this point we've assumed dark energy has no reason to go away. And so we would expect it to push the universe apart faster and faster over time. But it also helps us hopefully understand why dark energy is there to begin with. We have these mysteries of how quantum mechanics and gravity work together. Maybe this will be a clue of how to explain dark energy emerging from those two forces. Or maybe it has nothing to do with it. There's one other issue that's out there, which is that my colleague Adam Riess has, as I said, made those measurements very accurately over the last 20 years of the Hubble constant with the James Webb Space telescope now and Hubble before. And the number he gets doesn't quite agree with what we expect from the other really precise measurement we made of the universe - the cosmic microwave background. They're about 8% off. So maybe we just don't understand what's going on well at all. That's a distinct possibility. And that would be really exciting for an astronomer. But, as I said, I feel like we need to get some better data now because we're all a little confused again.