Why we trust machines too much

In other words, we are not only bad at evaluating the reliability of a machine, we are also easily won over by mechanical objects when they start to behave a bit like a social partner with our best interests at heart.

So, unlike the person who has been stung by unresponsive computers and therefore mistrusts all of them, the person who has learned to trust certain systems – such as airplane autopilots – may have hard to understand how they could be wrong, even when they are.

De Petrillo notes that she experienced a sense of confidence in computers when interacting with voice-activated assistants like Apple’s Siri or Amazon’s Alexa.

“I guess they’re acting in my best interest, so I don’t need to question them,” she says. As long as they appear competent and reasonably warm, Kulms’ study suggests that this will continue to be the case for de Petrillo and many others. Kulm points out that this is why it is so important for technology designers to ensure that their systems are ethical and that their functionality is transparent.

The great irony of all of this is that behind a seemingly trustworthy machine may lie a malicious human with nefarious intentions. Undue reliance on a faulty machine is dangerous enough, let alone a machine designed to deceive.

“If we were to take the same type of risk with a human, we would be much more attuned to what the potential negative outcomes might be,” says Proctor. She and Haux agree that more work is needed to determine how much chimpanzees in their studies genuinely trust machines, and to what extent that reveals truths about human behavior.

But there is a hint here that our occasional, sometimes disastrous, over-reliance on technology has been influenced by a simple fact: We have evolved into social animals in a world where machines did not exist. Now they are doing it and we are putting our money, our personal data and even our lives under their supervision all the time. It’s not necessarily the wrong thing to do – it’s just that we’re often bad enough at judging when it’s right.

–

Join a million Future fans by liking us on Facebook, or follow us on Twitter Where Instagram.

If you liked this story, subscribe to the weekly newsletter on bbc.com features, called “The Essential List” – a selection of handpicked stories from the BBC Future, Culture, Working life, To travel and Reel delivered to your inbox every Friday.