Semi-autonomous driving systems like Tesla's Autopilot and Cadillac's Super Cruise are designed to reduce driving stress by automating tasks such as steering, acceleration and braking under certain circumstances, but the driver must be ready to take over at any time. It's a problem, Mary Cummings, a professor at Duke who studies human interaction and autonomous driving systems, Business Insider told.
"Any confidence in man to intervene in any form of safety critical capacity is a red herring. They will never be able to make it reliable enough to do these systems, I believe, safe for the road," she said.
Read more: We tested Teslas, Cadillacs and Mercedes' semi-autonomous driving systems ̵
Systems like Autopilot, Super Cruise, Nissan's ProPilot Assist and Volvo's Pilot Assist relying on unrealistic expectations of how well humans are able to pay attention to the road and respond quickly if the semi-autonomous system needs them to take control, Cummings said.
"We can't maintain attention, especially in boring environments like highway driving," she said. "Expecting that man can only enter when we know they have not been aware is a major problem."
A Tesla representative directed Business Insider towards a page on the website describing his approach to vehicle safety and includes statistics on accident rates for Tesla vehicles with and without autopilot engaged.
"We believe that the unique combination of passive safety, active safety and automated driver assistance is crucial to keeping not only Tesla drivers and passengers safe, but all drivers on the road," says the site.
In the fourth quarter of 2018, Tesla said it was an accident per 2.91 million miles driven by Tesla cars with Autopilot engaged in an accident per 1.58 million miles driven by Tesla vehicles without autopilot engaged and a car crash each 436,000 miles driven for all cars in the United States. But Tesla's data does not take into account factors such as those environments where Autopilot is most often used, which can also account for some of the differences in accident rates.
Nissan and Volvo did not respond to a request for comment.
There are semi-autonomous features, such as those that alert the driver if the car is running out of track or some advanced cruise control systems – which can slow down or accelerate out of surrounding traffic – which can be useful, especially at low speeds, so Cummings. The danger comes when cars take control of driving tasks at higher speeds.
"I'm a big fan of low-speed autonomy," she said. "It's when we see cars at increasing speed under unstructured environments with much more uncertainty that car management systems are really falling apart."