How Can AI Researchers Save Energy? By Going Backward.
Nash Weerasekera for Quanta Magazine
Introduction
For Michael Frank, efficiency has always been a major preoccupation. As a student in the 1990s, he was originally interested in artificial intelligence. But once he realized how much energy the technology would use, he took his research in another direction. “I started getting interested in the physical limits of computation,” he said. “What’s the most efficient computer you can possibly build?”
He soon found a candidate that took advantage of a quirk of thermodynamics: a device whose computations could run backward as well as forward. By never deleting data, such a “reversible” computer would avoid wasted energy.
Now, as progress in traditional computing is slowing — new chips are running into fundamental physical limitations that prevent them from getting smaller — reversible computing could keep computational progress going.
“There aren’t that many other ways to improve power,” said Christof Teuscher, who researches unconventional approaches to computing at Portland State University. “Reversible computing is this really beneficial, really exciting way of saving potentially orders of magnitude.”
Feeling the Heat
The first person to study reversible computing was Rolf Landauer, a celebrated physicist at IBM who in the 1960s helped establish the field of information processing. A major theme in his work involved the relationship between entropy, which is a measure of disorder, and information.
Roughly speaking, the more we know about a system the more orderly its arrangement can be, and the less we know, the less orderly. Atoms going from predictable behavior to unpredictable motion is what makes some physical processes irreversible. For example, when you stir milk into coffee, disorder increases. You can, in theory, separate them again to make the arrangement of milk molecules and coffee molecules more orderly. But to do so, you’d need knowledge of the path of each drop of milk, an implausibly vast amount of information.
In 1961, Landauer showed that these principles govern computers as well. When a computer system deletes information, electrons in the computer’s chips go from known paths to unknown ones. Those discarded electrons are effectively lost, and their energy bleeds off as heat. Landauer proved that this is an inescapable fact: No matter how a computer is built, it must give off a certain minimum amount of heat for each bit of information it deletes.
Michael Frank has long worked to make practical reversible computers, which would save considerable energy by not deleting data.
Johnny Carroll/Albuquerque Headshot & Portrait
This loss is a fundamental aspect of how computers operate. For example, when a computer adds two numbers together, it returns a single number for the total: 2 + 2 = 4. There’s a loss of information as you go from two numbers to one. You could have added 2 and 2, or you could have combined 1 and 3. The missing information makes the calculation irreversible. Computers that process information this way — and almost all of them do — are always going to lose some information as heat, no matter what.
Landauer wondered if a machine could get around this limitation by simply never deleting data. Such a device would need to keep a record of every operation, every pair of numbers added at every step. These records would rapidly fill its memory, making such a computer unusable in practice, despite the energy savings. Landauer soon moved on, convinced that reversible computing was a dead end.
A decade later, he learned that he had been mistaken.
Hitting Reverse
Charles Bennett, a younger colleague of Landauer’s at IBM, argued in 1973 that there was another option. Instead of saving every single scrap of information, you could run each calculation forward, store the result you care about, then run the calculation backward. Bennett’s idea, which he called uncomputation, is a little like if Hansel and Gretel picked up their trail of breadcrumbs on the way back home: The pair are guaranteed not to get lost, and they don’t waste any breadcrumbs. Uncomputation means you are left with only the data you want, and you never lose track of it. Because none of the initial information is deleted, you never lose energy to heat.
Unfortunately, uncomputation also takes twice as long as an ordinary computation, which makes it impractical.
Still, Bennett continued to improve on his idea. In 1989, he showed that you can uncompute in much less time by using slightly more memory. Researchers began to tinker with the details, finding ways to shave off further memory and time.
But computers don’t lose energy just from deleting data. The way their transistors are linked is inherently inefficient, so for a computer to save a meaningful amount of energy from reversible computing, it must be designed with low heat loss in mind from the beginning.
In the 1990s, a group of engineers at the Massachusetts Institute of Technology set out to do exactly that. The team built prototype chips that improved the inefficient circuits. Frank joined the group as a doctoral student in 1995 and soon became one of reversible computing’s chief proponents.
Then, at the start of the new millennium, interest slowed. The chips didn’t yet have what it took to save energy in the real world. Support was scarce.
“Program reviewers would say, ‘This stuff sounds really useful, industry should be funding it,’ and yet you go to industry and they have no idea what you’re talking about,” Frank said. “It sounds crazy to them,” because it dealt with a problem that seemed so remote. Regular computer chips were improving exponentially. Why worry about theoretical alternatives?
Frank abandoned the work, and for a while he even left the field to open an internet café. But before long, the concerns that industry thought were distant got closer. Computer circuits were getting so small that they were hitting fundamental physical limits that would stop them from shrinking any more.
“You aren’t going to be able to scale conventional technology any further,” he said.
Frank built a research group at Sandia National Labs and began trying to rally attention toward energy efficiency.
Then, in 2022, Hannah Earley, at the time a researcher at the University of Cambridge, delivered a rigorous account of how efficient these computers could be. A reversible computer emits much less heat than a conventional one, but she found it must still emit some heat. When voltage turns on in a wire, for example, the metal heats up, with more heat the faster the voltage changes. The more slowly a reversible computer runs, the less heat it emits, a relationship Earley calculated precisely.
That relationship between heat and speed is crucial for reversible computing’s most promising application: AI. Computations in AI are often run in parallel, meaning different processors each run one part of a computation. This creates an opportunity for reversible computing to shine. If you run reversible chips more slowly, but use more of them to compensate, you end up saving energy: The advantage of running each chip more slowly wins out against the disadvantage of running more chips. And if you run them slowly enough, you might get away with not needing as much cooling, which will let you pile chips closer together to save on space, materials, and time spent shuttling data between them.
Investors have taken notice. Earley has co-founded Vaire Computing, where she and Frank are working to create a commercial version of a reversible chip.
After decades of theory, said Torben Ægidius Mogensen, who works on reversible computing at the University of Copenhagen, we may finally see this approach in action. “The most exciting thing would be actually seeing reversible processors made in practice so we can use them.”
