Alphabet’s Loon, the team responsible for beaming internet down to Earth from stratospheric helium balloons, has achieved a new milestone: its navigation system is no longer run by human-designed software.
Instead, the company’s internet balloons are steered around the globe by an artificial intelligence — in particular, a set of algorithms both written and executed by a deep reinforcement learning-based flight control system that is more efficient and adept than the older, human-made one. The system is now managing Loon’s fleet of balloons over Kenya, where Loon launched its first commercial internet service in July after testing its fleet in a series of disaster relief initiatives and other test environments for much of the last decade.
Similar to how researchers have achieved breakthrough AI advances in teaching computers to play sophisticated video games and helping software learn how to manipulate robotic hands in lifelike ways, reinforcement learning is a technique that allows software to teach itself skills through trial and error. Obviously, such repetition is not possible in the real world when dealing with high-altitude balloons that are costly to operate and even more costly to repair in the event they crash.
So Loon, like many other AI labs that have turned to reinforcement learning to develop sophisticated AI programs, taught its flight control system how to pilot the balloons using computer simulation, with help from Google’s AI team out of Montreal. That way, the system could improve over time before being deployed on a real-world balloon fleet.
“While the promise of RL (reinforcement learning) for Loon was always large, when we first began exploring this technology it was not always clear that deep RL was practical or viable for high altitude platforms drifting through the stratosphere autonomously for long durations,” explains Sal Candido, Loon’s chief technology officer and co-author of a paper on the new flight control system published this week in the scientific journal Nature, in a blog post. “It turns out that RL is practical for a fleet of stratospheric balloons. These days, Loon’s navigation system’s most complex task is solved by an algorithm that is learned by a computer experimenting with balloon navigation in simulation.”
Loon says its system qualifies as the world’s first deployment of this variety of AI in a commercial aerospace system. And not only that, but it actually outperforms the system designed by humans. “To be frank, we wanted to confirm that by using RL a machine could build a navigation system equal to what we ourselves had built,” Candido writes. “The learned deep neural network that specifies the flight controls is wrapped with an appropriate safety assurance layer to ensure the agent is always driving safely. Across our simulation benchmark we were able to not only replicate but dramatically improve our navigation system by utilizing RL.”
In its first real-world test over Peru in July 2019, the AI-controlled flight system went head-to-head with a traditional one, controlled by a human-built algorithm called StationSeeker, that was designed by the Loon engineers themselves. “In some sense it was the machine — which spent a few weeks building its controller — against me — who, along with many others, had spent many years carefully fine-tuning our conventional controller based on a decade of experience working with Loon balloons. We were nervous… and hoping to lose,” Candido says.
The AI-controlled system handily outperformed the human one by consistently staying closer to a device the team uses to measure LTE signals in the field, and that test paved the way for more experiments to prove the efficacy of the system before it formally replaced the one the team had spent years building by hand. Loon now thinks its system can “serve as a proof point that RL can be useful to control complicated, real world systems for fundamentally continual and dynamic activity.”
In his closing remarks, Candido touches on the concept of whether this type of AI is worthy of the name, because of how specialized it is and how closely it resembles a traditional but not self-learning, automated system like the ones that operate heavy machinery or control elements of mass transit.
“While there is no chance that a super-pressure balloon drifting efficiently through the stratosphere will become sentient, we have transitioned from designing its navigation system ourselves to having computers construct it in a data-driven manner,” he says. “Even if it’s not the beginning of an Asimov novel, it’s a good story and maybe something worth calling AI.”
