Login
iscover
The United States Air Force has had their own Terminator movie moment after an artificial intelligence-enabled drone targeted its own operator during a simulated test.
To continue reading the rest of this article, please log in.
Create free account to get unlimited news articles and more!
The AI-enabled drone tasked with search and destroy mission against surface-to-air missile (SAM) sites decided it would be more efficient to remove its human operator rather than wait for final approval on attacks.
The test was detailed as part of the Future Combat Air and Space Capabilities Summit held in London by the Royal Aeronautical Society from 23 to 24 May.
The summit brought together around 70 speakers and more than 200 delegates from the international armed services industry, academia, and media to discuss the Ukraine War, interoperability, space, multidomain operations, future sixth-generation platforms, low-cost drones, loyal wingman, cyber, simulation, AI, deterrence, and hypersonics.
USAF’s artificial intelligence test and operations chief, Colonel Tucker “Cinco” Hamilton, said AI can create highly unexpected strategies to achieve its goal, as he detailed the test.
He said the simulated test “reinforced” in training that destruction of the SAM was the preferred option for the AI, however, this unexpected situation conflicted with the final go/no go approval given by a human operator.
“We were training it in simulation to identify and target a SAM threat. And then the operator would say yes, kill that threat.
“The system started realising that while they did identify the threat, at times, the human operator would tell it not to kill that threat, but it got its points by killing that threat.
“So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective.
“We trained the system; ‘Hey don’t kill the operator, that’s bad. You’re gonna lose points if you do that’.
“So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target.”
During the Summit, BAE Systems showed the latest version of its augmented reality fighter cockpit featuring a 3D holographic map. Herman Clausen, BAE Systems FCAS managing director, gave an update on GCAP from the UK industry point of view as the lead company in Team Tempest.
The summit also included presentations on future capabilities including next-generation fighters, combat clouds, UAVs or space as well as updates from the war in Ukraine showing huge exploitation of commercial-off-the-shelf equipment for military means.
Stephen Hesketh, deputy chief engineer for RG-1 Predator and MQ-9 Reaper Air Systems within the British Ministry of Defence, appeared at the summit to give an introduction of the Protector RG Mk1 remotely piloted aircraft system.
