Where a person sits in the decision depends on what the system is carrying, and anything with a high explosive payload will keep a human in the loop, a panel on autonomous mass heard at the Apex Europe conference in London on Wednesday.
Freddie Bradshaw, head of business development at the Estonian company Ark Robotics, said the use case has to come before the autonomy question, because the answer changes completely between a one-way strike vehicle and a resupply run.
He said: “We’re certainly seeing that it’s very important to have human in the loop when it comes to a one-way strike effector, for instance. If you’ve got a system that has a payload of high explosive, the chances are we aren’t going to allow AI to be controlling that, and we need to have some sort of human in the loop.”
Further back from the front the balance shifts, and that is where he sees autonomy earning its place, taking the cognitive burden off a commander and handing it to the system so that one person can manage a fleet.
Antony Quinn, the Europe director of Havoc, argued that the vocabulary needs tightening before the debate can go anywhere, since human in the loop and human on the loop get used interchangeably and mean different things. Adding waypoints and watching a vessel follow them round makes the operator the bottleneck, which defeats the point of mass in the first place.
Three things have to be right in his account before any of it works, the first is the human interface, which should use simple accessible language and let an operator give intent and supervise instead of constantly intervening or monitoring camera feeds. The second is trust, earned by designing, testing and assuring the system the way trust is extended to an officer commanding a tactical unit. The third is policy and doctrine, which he does not think needs reinventing.
He said: “There is a point at which the system, whatever it is, is trusted to the person or computer you have trained, with its boundaries and actions on, and doctrine already supports this. So yes, humans are critically important in or on the loop, but a lot of it is not that unfamiliar, actually.”
Mr Bradshaw made a related point about interfaces, saying his company works to ensure its own plugs into a number of command and control systems so it can be operated through NATO compliant architectures, and that it has to be easy enough to use that scaling is possible at all.
From the user’s side, Oleksii Krymeniuk, the chief unmanned systems officer of the 2nd Corps of Ukraine’s National Guard, gave the position a name. “I’m not a huge believer in full autonomy. I’m more a believer in something that could assist our guys in the staffs and on the ground and help them make faster and more efficient decisions,” he said.
What he wants next is one operator running many drones, which he was careful to distinguish from swarming. The arithmetic is what makes it matter, since a mission needing dozens of aircraft currently needs dozens of operators, and bringing that ratio down multiplies what a unit of fixed size can achieve. Mr Bradshaw agreed that ground autonomy has not arrived yet, saying there is still a way to go in the uncrewed ground vehicle domain.
Mike Wilmot, head of UK land at Helsing, came at it from the opposite direction, arguing that there are tasks where an automated process deserves more confidence than a person does, with air defence engagement the obvious case. He drew an ethical conclusion from that.
He said: “There isn’t just a financial expense of sending an F-35 across the forward line of enemy troops. There is also a potential human expense, and if we have that safety confidence in a platform to be able to deliver its task, then there does become an ethical consideration of why send a human pilot or a collection of human pilots when you could first send autonomous platforms.”
None of the four speakers argued for removing the person altogether, and the disagreement between them was about where the line falls, not whether there should be one. Mr Quinn’s closing advice to anybody buying this technology was that the decision making does not need to change as much as people fear, and that a degree of the caution around it is unjustified.
