In just a few short months, the future of self-driving grew clearer and closer. It was the result of shuffling alliances among automakers, sensor- and software-providers, and the most major (Tier One) automotive component suppliers. Automakers announced ambitious timing for autonomous driving. Uber announced it’s about to start testing self-driving cars in Pittsburgh. All this despite a late spring fatal crash of a Tesla running Autopilot.
All this suggests we’ll probably see seriously self-driving cars within five years that you can buy or lease, and the luddites have a shortened time frame for complaining about the pending carnage. The lone outlier? Apple, which we’ll get to below. First, here’s a rundown of what’s happening.
Upgrading Autopilot: seeing the world in radar https://t.co/WqJfyqpDXL
— Tesla (@TeslaMotors) September 11, 2016
Tesla crash in Florida was the starting point
A fatal accident May 7 in Florida was the first time a Tesla operating under Autopilot was involved in a fatal accident. This is the crash where the driver, Joshua Brown, had a portable DVD player in the car that may or may not have been playing at the time his Model S slammed into a tractor-trailer turning left across a divided highway. The Tesla’s optical sensors apparently did not recognize the light-colored finish of the trailer against a light sky.
In July, Tesla parted ways with Mobileye, the supplier of the algorithms and microprocessors behind the optical sensors. It’s unclear who cut whom loose. Mobileye has said its current offerings weren’t meant to detect every vehicle sideways in front of the vehicle but its next-gen system, called EyeQ4, will when it arrives in 2018.
Recently, Tesla CEO Elon Musk revealed pending changes to improve the next version of Autopilot, 8.0, that shifts primary responsibility for checking the road from the optical camera in the windshield to the front-facing radar. The automatic, over-the-air update, should be ready soon.
Safety agencies and Consumer Reports have dinged Tesla for allowing extended hands-off driving currently — and even for use of the term “Autopilot.” They fear the term emboldens drivers when it really is a limited system for pacing the car in front, staying in the center of lanes that aren’t too sharply curved, and responding to some incursions into the lane the Tesla is in.
On its blog, Tesla said, in part:
While there are dozens of small refinements with Version 8 of our software … the most significant upgrade to Autopilot will be the use of more advanced signal processing to create a picture of the world using the onboard radar. The radar was added to all Tesla vehicles in October 2014 as part of the Autopilot hardware suite, but was only meant to be a supplementary sensor to the primary camera and image processing system.
After careful consideration, we now believe it can be used as a primary control sensor without requiring the camera to confirm visual image recognition. This is a non-trivial and counter-intuitive problem, because of how strange the world looks in radar. Photons of that wavelength travel easily through fog, dust, rain and snow, but anything metallic looks like a mirror. The radar can see people, but they appear partially translucent. Something made of wood or painted plastic, though opaque to a person, is almost as transparent as glass to radar.
Tesla will also use fleet learning, meaning if there’s an object that on or near the road that might be mistaken by radar for a stopped car or other hazard (a soda can with its concave bottom facing the oncoming car could do that, Tesla says), and several cars (Teslas) safely drive past the object, it will be added to a whitelist of objects that can be ignored via Tesla’s telematics. Obviously the whitelist learning would work even better if all semi-self-driving cars shared this information, either via telematics or via vehicle-to-vehicle DSRC, or dedicated short range communications.
Tesla said it will even bounce the radar off the road and under the car immediately in front to see two cars ahead. This is something Nissan has been doing with its look-ahead or predictive adaptive cruise control (image below) back in 2014 on vehicles such as the then-new Nissan Murano.
Most cars with semi-autonomous driving in the form of lane centering assist warn hands-off drivers to take hold of the wheel after about 10 seconds and suspend lane-centering assist about five seconds later. Tesla’s tough love adjustment, according to The New York Times, now will be that at highway speeds (45 mph and up), hands-off drivers will be “warned after a few minutes, and more frequently if the road turns or curves, or if there are other cars on the road ahead.”
A wave of new alliances
Automakers have been working overtime to sew up alliances with suppliers of the components and algorithms for self-driving, and the separately functioning modules that pass for simpler assisted driving today: adaptive cruise control, emergency braking; lane keep assist and its predecessor lane departure warning; and blind spot assist. Automakers that so far have projected self-driving cars within five years or less include Ford, General Motors, and Fiat Chrysler (FCA). This would mean a so-called Level 4 car, or one that can operate without a licensed driver in the car, quite possibly with no steering wheel or foot controls.
Three companies with strengths in vision systems are Autoliv, Mobileye, and Nvidia. Autoliv and Volvo cars this month announced a joint venture to autonomous driving software.
Intel this month bought the startup Movidius which specializes in machine vision. It’s a supplier to others higher up on the food chain such as FLIR (night vision sensors for cars, planes, and spies), DJI (drones), Google (self-driving car systems) and Lenovo.
Delphi, the former parts division of GM, and Mobileye entered into a partnership that would provide an autonomous driving solution that a (likely smaller) automaker could put in their cars as soon as late 2019, the two say.
Not wanting to be dependent on Google for maps, BMW, Daimler AG (ie Mercedes-Benz), and Volkswagen’s Audi unit bought the HERE real-time maps company that began as Navteq.
Automakers are also buying into ancillary technology companies that know about ride-sharing, for the time when self-driving cars are real. BMW’s iVentures subsidiary, for instance, bought into RideCell, a ride-booking firm in Seattle.
Uber Pittsburgh will have self-driving cars, with drivers
Uber this week plans to begin a limited test in Pittsburgh of self-driving cars that will pick up and chauffeur passengers. Since it’s still early for a truly self-driving car, one so good it wouldn’t need a steering wheel, there will still be an Uber driver behind the steering wheel.
Uber has announced alliances with Volvo, which will provide the modified Volvo XC90 SUVs for the aforementioned Pittsburgh test. It also bought the startup company Otto for Uber-level chump change ($ 700 million) and acquired a host of talent in robotics and autonomous driving, specializing in trucks. While truckers (some) sniff at the idea an 80,000-pound vehicle being auto-piloted, much of their driving is on interstates with mostly gentle curves. One concept is a lead vehicle with a driver (also lane center, adaptive cruise) and a dozen or more autonomous trucks following behind (with, yes, enough distance between them that cars would be locked out of the right lane).
Uber chose Pittsburgh because of the strong robotics and autonomous driving at Carnegie Mellion and for the state’s relatively open stance on testing self-driving cars, which to critics and skeptics means “life-threatening.”
But Apple pulls back on self-driving cars
Every trend line has a few outlier data points. In this case it’s Apple, which is reconsidering its unannounced plans (Project Titan) to create self-driving cars. According to a Friday story by The New York Times:
In a retrenchment of one of its most ambitious initiatives, Apple has shuttered parts of its self-driving car project and laid off dozens of employees, according to three people briefed on the move who were not allowed to speak about it publicly.
The job cuts are the latest sign of trouble with Apple’s car initiative. The company has added resources to the project — code-named Titan — over the last two years, but it has struggled to make progress. And in July, the company brought in Bob Mansfield, a highly regarded Apple veteran, to take over the effort.
Apple may have had a change of heart: Initially, it wanted to create and build an entire self-driving car, most likely an electric vehicle. Over time, it reportedly wondered if that was the company’s core competence. The pullback indicates Apple might fall back to creating the self-driving hardware and algorithms in conjunction with hardware providers, and sell that to automakers.
At the same time, Apple has a mass of loyal phone-notebook-tablet-watch buyers who’d be inclined to put an Apple car on their shortlist. The Apple name might provide (initially) a price premium.
It may also be that Apple had a hard time matching up with automakers who’d be willing to create a hardware platform (the car body, interior, and suspension) for Apple. Apple is widely reported to have approached both BMW and Daimler to work together. Apple visited BMW’s i3 factory in Leipzig, Germany, to see first-hand how the car was made. Reports this past spring say BMW and Daimler both declined to move forward because of disagreements over how much control they’d have. Reports at the time said that Apple could still link up with Magna Steyr, a Canadian-Austrian contract manufacturer of cars.