Safety Drivers, Remote Diving And Assist - The Long Tail Of Robotaxis

Remote driving room at Vay, where operators have video game consoles and multiple screens to control ... More cars on Las Vegas streets.
With Tesla now planning to launch its pilot robotaxi service in Austin TX this week using safety drivers (Tesla employees in the passenger seat able to supervise and intervene) it's a good time to review the history of the safety driver and all the other technologies being used to help self-driving cars deal with the 'long tail' of problems they must solve to work on our roads.
Making a car that can handle every possible road situation with perfect safety is a science fictional goal--nobody is close to knowing how to do it. As such, all robocars call on humans in one way or another, from tasks as simple as cleaning and recharging them to intervening when they make an unsafe move. I'll look at all approaches.
Removing the safety driver is the 'big hard step' that changes a vehicle from a testing prototype to a real robotaxi. There are many other steps, but they are baby steps compared to the first time the vehicle goes out without a human overseeing it and able to take control.
Safety Driver
The very first robocars, which were pretty primitive and failed often, were set up so a human driver could sit in the driver's seat and grab the wheel or press the pedals at any time. That immediately disengaged the self-drive system and the car became manually driven. Many cars also had the 'big red button,' an emergency stop button to be used if grabbing the controls failed. It usually did a hard disconnect of all systems, but in practice most are never used.
Outside of closed courses like the DARPA Grand Challenge, all robocar testing from day one has worked this way. All teams hope for the day they can remove that safety driver, as that is the whole goal. It generally works well. With properly behaving safety drivers, test robocars have very good safety records. Except for one giant black mark, when Uber ATG did not manage safety drivers well, and hired one who watched a TV show instead of doing her job, allowing the vehicle, when it failed (as prototypes are expected to do) to strike and kill a pedestrian.
In the early years, cars typically had two staff in them, one behind the wheel, and the other, sometimes called the software operator, who monitored the driving soft
ware to make sure it was doing the right things. Safety drivers can take the wheel at any time, and are told to do it if they feel anything odd is going on, or sometimes if a risky situation is likely. Especially in early years, if there were children on the street, you always took over. In addition, if the software detected any problems, it would alert the safety driver to take over.
Teams (and governments) track interventions. The best teams take every significant intervention and create a simulation scenario to duplicate it, then test what would have happened if the human had not intervened. If the car would have done something bad, like hit something, that becomes a priority problem to fix. Indeed, interventions where it turns out the car would have done fine are often not even counted.
Tesla took things to a new level when it released Autopilot and FSD. These had ordinary untrained customers act as supervisor for the vehicle. Google/Waymo had only used trained employees who took a driving safety course. When Tesla did this, there was great skepticism that relying on ordinary customers would be unsafe, but in reality, it worked out. Probably not as safe as ordinary driving, but fairly similar. (Tesla misleadingly claims it is much safer, but this is false.)
Safety Driver not in Driver's Seat
The normal place is behind the wheel, but some vehicles put a safety operator in another location, such as the passenger's seat. In vehicles designed with no controls (like some shuttles) the employee may have access to just an emergency stop button that commands the vehicle to stop and pull over, or slightly more involved controls. This may also include a video game controller, wheel or gamepad that can be plugged in for manual driving.
This is probably not as safe as a person behind the wheel. In the passenger's seat, there is a ong history of human driving instructors training teen drivers by having their own brake pedal and the ability to grab the wheel. I remember my own driving instructor doing that. It works, though it's not clear if it has any purpose other than saying, 'nobody at the wheel.' It's more for PR than safety in vehicles that still have a wheel.
Even so, some companies have done it. Russian robotaxi company Yandex used it in Austin and other cities. (Yandex is now non-Russian and called AVRide.) Cruise did their first 'driverless' test with an employee in the passenger seat. Most shuttle companies keep a worker in the shuttle who can hit the emergency stop, and pull out a game controller to drive.
Remote Driving
It may surprise some to learn there are remotely driven cars on the road today. German company Vay uses this to deliver cars to customers in Las Vegas. Several other companies have built different tools for remote driving. I worked (with compensation) with one such company, DriveU.Auto to produce a video about some of these approaches.
Remote driving is done over public data networks, which of course face interruptions and packet loss and sometimes long latency. As such, it is typically done with a system capable of doing safe basic operations without remote input so that it will, at worst case, just come to a stop if comms get too bad. They are also usually designed to use multiple communications channels to survive problems with any one of them.
Remote driving still requires paying a human, so you lose a lot of the cost advantage of a robocar over say, an Uber. However, you don't have to pay a human while the vehicle is sitting waiting. (Uber doesn't pay its drivers for that either, but it effectively builds into the fares for actual driving enough to make drivers tolerate the wait. They can also do other work or read or watch videos between rides.) As such, it can be cheaper to operate a remote driven fleet than a human driven one, and you can do WhistleCar service (car delivery) like Vay.
Some companies, like Waymo, make use of low speed remote driving when they are in a situation where they want to move a stuck car or solve a problem the software can't. At these low speeds, you can't do much damage and you can stop on a dime if the connection has an issue.
Some delivery robots, such as the early Kiwibot and Coco, were entirely remote driven, because at the speed of sidewalk delivery robots, that's fairy doable without safety concerns. (Indeed, many of these robots are so light that they don't hurt people even if they did hit them.)
Tesla has advertised for programmers for some time to work on their remote driving and control systems for both robotaxis and the Optimus robot.
Remote Supervision with Driving
Remote supervision is effectively taking the safety driver and making them remote. The car mostly drives itself, but the remote supervisor is always watching (usually with an array of screens or possibly a VR headset) and 'grabs the wheel' virtually if they see the need to take over. You need to not need 'instant' takeovers that depend on sub-second reaction times (humans need about 0.7 seconds even when in the car) but is good for most problems which are apparent further in advance when you have the power of a human mind. This approach is not used by any team, at least publicly, though it has been speculated that Tesla is considering it.
Remote Monitoring with Stop
Most companies have the ability to connect to a car and watch what it's doing, even in full autonomous mode. Companies decline to comment on this, but all of them probably did this when they first dared to send the cars out with no safety driver. It seems like it would be foolish not to. Full time 1:1 remote monitoring doesn't scale very well, but it makes perfect sense in a pilot. In addition, these remote monitors probably have some ability to send a 'kill' command to the vehicle, to ask it to immediately stop and pull over, known as a 'minimum risk condition.' The difference between this and remote supervision is that these remote monitors can't do live steering, just hard stopping. Once stopped, however, they can usually switch into remote assist mode.
Remote Assist
All companies tend to have a remote assist operations room. There, operators are present who can help vehicles solve problems when they get confused. They usually cannot drive the vehicles directly, only give them strategic advice, like 'Turn around and take this new route' or 'Make the 2nd left' or 'Follow this set of waypoints to get around that obstacle' and most often 'Continue with your current plan, it's OK.'
For remote assist to scale, you need to have many more vehicles on the road than remote assist operators, so that each vehicle on average needs active assist just a small fraction of the time. At Starship technologies, a delivery robot company, we set a goal of having 99% autonomy, meaning 100 robots for each operator. This makes the human labor cost effective. It's easier to do for delivery robots which can just stop and wait at any time. A leak from Cruise revealed robots were asking for help about every 5 minutes, which Cruise CEO Kyle Vogt felt was according to plan during their pilot stage. Over time the numbers would get better, but are never expected to get to zero. In most cases, according to Cruise, all the remote operator does is say, 'yes, continue with your plan A.' (In a typical remote assist, the vehicle sees a situation it is not fully sure of and offers multiple plans to the human remote operator, who picks one, or just allows plan A, or rarely crafts a new plan.)
Sometimes remote operators will make human-like mistakes, which has been responsible for some strange incidents at Waymo, including one crash when the remote operator approved the vehicle going when it should not have.
Some companies do have remote operators watching multiple cars at a time, which humans can do. While this may not scale long term, it's reasonably affordable today and wise during the pilot and growth stages of a robotaxi fleet.
Waymo recently added the ability of remote operators to do low-speed remote drive to do things like move cars off the road, or out of trouble situations like blocked streets and emergency vehicles.
Rescue Driver
When all else fails, most teams can send humans in a car to rescue a vehicle by manually driving it, or in the worst case, towing it. For cars without controls, these teams will have a plug-in video game style controller. There are reports that some cars also have such a controller locked in a compartment that law enforcement can open so they can move cars without controls.

Try Our AI Features

Explore what Daily8 AI can do for you:

Learn with AIFact CheckingText to SpeechAI Summary

Related Articles

Yahoo

21 minutes ago

• Yahoo

Texas will require permits for self-driving cars starting in September

Starting September 1, fully autonomous cars will require a permit to operate in Texas. This new restriction comes after the state's governor, Greg Abbott, signed into law the SB 2807 bill that requires authorization from the state's Department of Motor Vehicles for self-driving cars on public streets without human interaction. The signing of the new law coincides with Tesla's launch of its robotaxi service in Austin today where invited guests got to ride in fully autonomous Model Ys. While the law doesn't go into effect until a few months later, Tesla's robotaxi launch could have skirted this new regulation since all rides are accompanied by a human "safety monitor" in the front passenger seat. In the coming months, Texas' new law may prove to be more than just a headache for Tesla. Besides the permits, the upcoming regulation allows state authorities the ability to revoke permits and requires companies to provide methods of dealing with self-driving cars in emergency situations to police and first responders. While this could become a serious hurdle for Tesla, it's also likely to affect Waymo, since it currently operates an autonomous ride-hailing service in Austin.

Gizmodo

24 minutes ago

• Gizmodo

Elon Musk's Trillion-Dollar Robotaxi Gamble Is Here

The wait is finally over. After years of promises from its eccentric CEO, Tesla debuted its highly anticipated robotaxi service on June 22 in Austin, Texas, a launch that is central to the company's entire future. This isn't just about a new feature; it's the cornerstone of Elon Musk's narrative that Tesla is not merely a car company but a world-changing AI and robotics powerhouse. As the automaker faces fierce competition from Chinese rivals like BYD, the success or failure of its autonomous vision could define its next chapter. 'The @Tesla_AI robotaxi launch begins in Austin this afternoon with customers paying a $4.20 flat fee!' Musk announced on X, followed by posts congratulating his teams. The @Tesla_AI robotaxi launch begins in Austin this afternoon with customers paying a $4.20 flat fee! — Elon Musk (@elonmusk) June 22, 2025The service kicked off with a small fleet of 10 to 20 Model Y SUVs navigating public roads. In a demo posted by Tesla, users within a specific 'geofenced' area in south Austin can hail a ride through a dedicated app. The concept is simple: a taxi with no human driver. However, the reality of this 'limited launch' is more cautious. The first rides were given to a select group of influencers and fans, and videos posted by the company show a 'safety monitor' sitting in the passenger seat, a detail at odds with the fully autonomous dream. — Tesla (@Tesla) June 22, 2025Musk himself admitted the company is being 'super paranoid about safety,' a sentiment that seems justified given a new Texas law requiring state permits for self-driving vehicles, set to take effect on September 1. Tentatively, June 22. We are being super paranoid about safety, so the date could shift. First Tesla that drives itself from factory end of line all the way to a customer house is June 28. — Elon Musk (@elonmusk) June 11, 2025At its core, the robotaxi is a vehicle powered by the most advanced version of Tesla's Full Self-Driving (FSD) system and a suite of eight cameras. But unlike competitors, Tesla claims its system doesn't need expensive, pre-mapped service areas. 'It just works,' the company posted on X, promising future expansion to cities like San Francisco and Los Angeles. For Musk, this is the culmination of a long-standing promise. He envisions a future fleet, including a new 'Cybercab' and 'Robovan' with no steering wheels or pedals, that could boost Tesla's market value by an astonishing $5 trillion to $10 trillion. On June 20, Tesla was worth $1.04 trillion, the 11th most valuable company globally. By comparison, Microsoft ($3.54T), Nvidia ($3.50T), and Apple ($3.00T) top the leaderboard. Financial bulls share his optimism. 'My view is the golden age of autonomous vehicles starting on Sunday in Austin for Tesla,' said Wedbush analyst Dan Ives. 'I believe it's a trillion dollar valuation opportunity for Tesla.' Investor Cathie Wood's ARK Invest predicts robotaxis could account for 90% of Tesla's profits by 2029. If they are right, this weekend's launch was existential. 🚨BREAKING: Dan Ives says Tesla's biggest growth chapter starts Sunday with Robotaxis He calls it a $1 trillion opportunity — Muskonomy (@muskonomy) June 21, 2025But there's a huge problem: Tesla may be late to the party. Waymo, Google's self-driving unit and the current market leader, already operates in Austin with a larger service area, as well as in Phoenix, San Francisco, and Los Angeles. Zoox, backed by Amazon, is live in Las Vegas and San Francisco and is testing in several other cities. The question isn't whether Tesla's robotaxis will work. It's whether they'll work better and faster than everyone else's. This fierce competition has led skeptics to dismiss Musk's grand projections. They argue that Tesla is unlikely to dominate a market where established players already have a significant head start. 'What valuation will be attached to Tesla autonomy when it has to split the autonomous ride hailing market with others?' asks investor Gary Black, whose fund has sold all of its Tesla shares. The $TSLA debate is not about Tesla robotaxi vs Waymo or Zoox, or whether TSLA robotaxi will work at 99.99% efficacy. Of course it will work or Elon wouldn't be moving forward with the Austin robotaxi launch today. The question – which bulls painfully avoid – is when others… — Gary Black (@garyblack00) June 22, 2025This is the multi-trillion-dollar question. Is the Austin launch the dawn of Tesla's next great chapter, or is it a cautious, overhyped entry into a race it may have already lost?

Washington Post

an hour ago

• Washington Post

Tesla launches long-awaited Robotaxi in Austin

Tesla held its first Robotaxi rides in Austin on Sunday, a cautious and modest launch that came more than a decade after CEO Elon Musk first pitched the idea. A group of social media influencers and Tesla investors took the first trips on Sunday, with many of them posting videos about their experience riding in the driverless cars — albeit with a human observer in the passenger seat. Musk said the company was being 'super paranoid about safety' ahead of the launch. The vehicles were limited to certain areas in Austin and included a safety monitor. 'Culmination of a decade of hard work,' Musk wrote on X on Sunday. The tentative launch is a key step toward Musk's longtime ambition of Tesla's fully autonomous vehicles shuttling paid passengers around the country with no one behind the wheel. Investors have been eager for Tesla to unveil its fully self-driving vehicle, as Musk has staked the future of the company on the technology, saying that 'the value of Tesla, overwhelmingly, is autonomy.' Tesla chose Austin as its initial launch site because Texas has a comparatively relaxed regulatory environment for autonomous vehicles. On Thursday, a group of Democratic state lawmakers called on Tesla to delay its launch until after Sept. 1, when new state-level safety requirements for autonomous vehicles take effect. The updated law, which overhauls the state's 2017 framework, requires commercial autonomous vehicle operators transporting passengers or goods to obtain prior authorization from the Department of Motor Vehicles, among other measures. The National Highway Traffic Safety Administration (NHTSA) recently sent Tesla a letter requesting details about the safety measures in place for the Robotaxi launch. NHTSA said Friday it had received Tesla's response and was in the process of reviewing it. Tesla also received a permit this year in California to provide fully autonomous taxi rides — initially to employees — on a 'prearranged' basis, with a safety driver present. The Robotaxi rides in Austin are being conducted in Tesla's midsize Model Y vehicles, not the futuristic Cybercab without a steering wheel and pedals that Musk unveiled last year. That vehicle — envisioned as a $30,000 'lounge on wheels' — is expected to launch in the next few years. Though long the leader in the U.S. electric vehicle market, Tesla is playing catch-up in the realm of autonomous vehicles. Google's Waymo vehicles are a regular presence on the streets of San Francisco and have also been rolled out commercially in Austin, Phoenix and Los Angeles. For months the company has been testing its service with safety monitors aboard in Washington and also plans to expand to New York, Atlanta and Miami. On Sunday, the first users of the Robotaxi service generally seemed pleased with the experience as they were dropped off at supermarkets and other spots around Austin. 'No-one is in the driver seat and the safety monitor in the passenger seat does not have a steering or pedals,' Tesla fan Sawyer Merritt wrote on X after his first ride. 'It was awesome.' The Robotaxi rollout comes at a critical time for Musk, who has distanced himself from Washington and is trying to redefine his image through Tesla and his other companies. Investors initially hoped Musk's ties with President Donald Trump could boost Tesla's fortunes, causing the stock, which closed at $251 on Election Day, to soar as high as $480 in December. Those gains evaporated as Musk embraced a contentious role within the administration as the driver of deep cuts to federal personnel and grants. The stock dropped to $222 in April after the company announced a 71 percent plunge in quarterly earnings from a year earlier. The longer-term effects of Musk's ties to Trump are unclear after a testy public falling out between the two earlier this month. Tesla stock closed at $322 on Friday.