Nissan and Volvo Cars' autonomous driving: TU Automotive Japan 2016

Nissan’s ProPILOT is a simple system using a monocular camera and image analysis technology

2016/12/12

Summary


The Nissan Serena, featuring ProPILOT autonomous driving technology on single highway lanes. (Photo: Nissan)

 TU Automotive Japan 2016 was organized by TU Automotive and held from October 18 to October 19 2016 in Tokyo.  This report will describe Nissan and Volvo Cars' autonomous driving technologies based on lectures by the two companies.

 In 2016, Nissan developed autonomous driving technology for single highway lanes, and intends to release autonomous driving technology for multiple lanes that enables lane-changing on highways in 2018, followed by autonomous driving on general roads including intersections in 2020. In August 2016, the OEM released its ProPILOT autonomous driving technology for single highway lanes in its Serena minivan, which is sold in Japan. ProPILOT achieves three autonomous driving functions: maintaining speed; tracking, stopping, and maintaining standstills; and steering control that corresponds to level 2 as designated by the NHTSA.

 Technically speaking, ProPILOT is a simple system that consists of a monocular camera, image analysis technology from Mobileye, and an original algorithm developed by Nissan.   It is offered as an option at a price starting from JPY 136,080 (tax included). The version scheduled for release in 2018, which can operate on multiple lanes, features additional cameras and radars, and the version that can be used on general roads, scheduled for release in 2020, is said to incorporate LIDAR and 3D maps.

 Volvo Cars is advancing its “Volvo Vision 2020” concept with an aim for Volvo vehicles to be involved in zero fatalities or serious injuries by 2020.

 Since 2015, Volvo has equipped its 90 series with Pilot Assist autonomous driving technology, a system equivalent to level 2 like Nissan’s ProPILOT. In 2017, the OEM will conduct public road demonstration experiments with one hundred of its customers in Gothenburg, Sweden, followed by London and in China. Volvo has announced that as long as it is being used appropriately, it will bear full responsibility for what happens when a vehicle is in autonomous driving mode.



Related reports:

U.S. NHTSA: Announcement of guidance for autonomous vehicles (November 2016)
Nissan to launch Serena autonomous vehicle and Note series HV in Japan in 2016 (July 2016)



Nissan’s autonomous driving technology

 Nissan has sought to deal with the following four issues regarding environments that surround vehicles: 1) energy issues, 2) global climate change, 3) road congestion, and 4) traffic accidents.
 In response to these issues, the OEM has advanced its strategy with the following measures serving as pillars:

  1. Electrification
  2. Vehicle Intelligence

 In the past twenty years, starting with the standardization of ABS in 1996 to rear cross traffic alerts (RCTA) in 2014, numerous preventive safety technologies have been realized. Of these technologies, Nissan claims that its eight technologies including lane keep support assist are the first of their kind. The All-around Safety Shield that the OEM has been pushing has already reached completion, and serves as the foundation for its autonomous driving technology.

 Autonomous driving allows the car to handle the four actions: 1) sensing, 2) cognition, 3) decision-making, and 4) action. The reaction speed of machines is said to be over 100 times the speed of human drivers.



Safety technology realized by Nissan in the past twenty years. (Source: Nissan) Nissan’s complete All-around Safety Shield. (Source: Nissan) The four fundamental functions that are the key to autonomous driving. (Source: Nissan)


Nissan’s ProPILOT: A simple system consisting of a monocular camera and image analysis technology

 Nissan’s schedule for autonomous driving until 2020:

  • 2016: Autonomous driving technology in single lanes on highways
  • 2018: Autonomous driving technology in multiple lanes on highways
  • 2020: Autonomous driving on general roads including intersections

 In August 2016, the OEM released its ProPILOT autonomous driving technology for single lanes on highways, which was equipped in its Serena minivan, and is equivalent to level 2 on the NHTSA’s scale of autonomy (the driver still bears driving responsibility). Although it is an optional feature, roughly 70% of consumers have had the function installed. ProPILOT realizes three autonomous driving functions: maintaining speed; tracking, stopping, and maintaining standstills; and steering control. The system supports the driver by autonomously controlling acceleration, braking, and steering across all speed ranges (0 to 100 km/h); and reducing driver fatigue during highway traffic congestion and extended driving periods.

 The system features a monocular camera (from ZF TRW Automotive) for its sensor, and utilizes image recognition processing technology from the Israeli company Mobileye, as well as an algorithm independently developed by Nissan. ProPILOT has hardware with the same configuration as automatic brakes, allowing the OEM to hold prices at JPY 136,080, including tax. In the future, further cost reduction can be expected, and Nissan plans to continue to promote the benefits of ProPILOT, even after more advanced autonomous driving technology is released.

 The version scheduled for release in 2018 that will be able to operate on multiple lanes is to be equipped with additional cameras and radars, and sources say that the version that it will be possible to use on general roads, scheduled for release in 2020, will incorporate LIDAR and 3D maps.



Nissan’s autonomous driving technology implementation schedule (Source: Nissan) ProPILOT benefits (Source: Nissan)



ProPILOT use conditions

Highways and car-exclusive roads ProPILOT is for use in single lanes on highways and car-exclusive roads that are straight or have only mild curves. Use on general roads is warned against, as it may lead to unforeseen accidents.
An alarm goes off when the driver takes their hands off the steering wheel, and if the driver does not respond, autonomous steering control is turned off.



Functions and operating conditions

Functions Objects in view Overview
(When there are no preceding cars)
(1) Maintaining speed
- The vehicle will drive at set speeds (roughly 30 to 100 km/h). ((3) The vehicle recognizes lane markers to control steering).
(When there are preceding vehicles)
(2) Tracking, stopping, and maintaining standstills
The preceding vehicles The vehicle will drive at a maximum set speed (roughly 30 to 100 km/h), and will maintain distance with the car in front of it.
Maintaining standstills is a newly incorporated technology. Conventionally, when the car comes to a stop, the driver was required to step on the brake pedal. With ProPILOT, the system will continue to operate up to three seconds after stopping, and the driver can extend the operation time to three minutes by pressing a button. After three minutes, the electric parking brake turns on to maintain standstill, and the system deactivates.
(If there are lane markers on both sides of the road)
(3) Steering control
Lane markers on both sides of the road and preceding vehicles Steering support assists drivers in driving in the center of the lane. This function is the first steering control for all speed ranges. However, if the vehicle cannot recognize lane markers, or if there are no preceding vehicles and the vehicle is driving at 50 km/h or below, autonomous steering control is not provided.
The system detects steering angles with twenty times the precision of earlier generations, and controls the vehicle based on those. Steering control operates under all speed ranges from 0 to 100 km/h, and electric power steering handles the steering operations so that the vehicle remains within a single lane.
When activating turn signals for lane changes, ProPILOT is canceled, but once the operation is complete, the turn signal is neutral, and the system confirms that conditions are safe, ProPILOT will resume functioning.
Source: Nissan Serena release press kit



A simple system consisting of a monocular camera and Mobileye’s image analysis technology

Mobileye has the highest global market share for automatic-braking image-processing software The ProPILOT system featured on the Serena has a simple structure consisting of a monocular camera and image processing technology from the Israeli company Mobileye, thereby reducing costs (JPY 136,080 including tax). Mobileye has 70 to 80% of the global market share for image processing software used in automatic braking and other advanced driver assistance systems (ADASs). Nissan employs the same combination for the automatic braking system in its X-Trail.
Distance is measured with the monocular camera Mobileye developed an automatic braking control system that uses a monocular camera. Although the company hasn’t announced any details regarding how the camera measures distance, it is based on fundamentals of perspective of “the width of the road and the lane converging to one point (vanishing point) on the horizon for infinity.” The point where the tires of the preceding vehicle touch the ground will become lower than the vanishing point as the vehicle comes closer. The camera's CMOS sensor reads up to 80 to 90 m ahead. The camera is also capable of detecting pedestrians and bicycles based on the shape of objects. The company supplies the EyeQ image-processing semiconductor.
Validation with enormous amounts of accumulated data Since its founding in 1999, Mobileye has provided many automakers with image-recognition software for ADAS, and has accumulated vast amounts of data, making its verification abilities far beyond those of other companies.
ProPILOT on Nissan's Serena was tested by the OEM on all major highways in Japan during its development, and when a false recognition occurred, the software was improved accordingly.
(Note)
1. Although the overseas version of ProPILOT features a millimeter-wave radar, vehicle speeds in Japan are slower than in Europe and the U.S., leading the OEM to decide to drop the millimeter-wave radar to reduce costs.
2. Because the only sensor is a monocular camera, if it cannot detect preceding vehicles because of backlight, the system reverts driving duties back to the driver.
3. ProPILOT’s pricing (JPY 136,080) is calculated from the difference between the Serena Rider edition (Autech 30th anniversary special edition) (priced at JPY 3,480,840) and the version with ProPILOT (JPY 3,616,920). Both models feature the same equipment aside from ProPILOT.


Volvo’s S90, V90, and XC90 feature Pilot Assist level 2 autonomous driving as a standard feature


The Volvo S90 featuring Pilot Assist (Detroit Auto Show 2016)

 Volvo Cars is advancing its concept of “Volvo Vision 2020” with its aim for zero fatalities or serious injuries involving Volvo vehicles by 2020. In Japan, the OEM acquired approval for use of automatic braking capable of full stops for the first time in 2009. In 2011, pedestrian detection functions were added, followed by cyclist detection functions in 2013. In 2016, nighttime detection of pedestrians and cyclists were added to enhance automatic braking. Newly registered Volvo vehicles featuring automatic braking in the seven years from 2009 to 2015 have a 69% reduction in accident rates in comparison to vehicles without automatic braking systems.

 At the Detroit Auto Show 2016, Volvo Cars exhibited the S90 and V90. The two models are equipped with second-generation Pilot Assist autonomous driving technology that is equivalent to level 2 on the NHTSA’s scale of vehicle autonomy. Pilot Assist features a millimeter-wave radar and camera, and can drive autonomously on roads where lane markers are easily recognizable in the center of a lane at preset speeds of up to 80 mph, or can also track preceding vehicles.

 Volvo’s Pilot Assist does not have automatic lane-changing functions (while the Mercedes-Benz E-Class and Tesla Model S/Model X have autonomous driving systems that are also positioned at level 2, they are able to automatically change lanes when the driver operates the turn signals). Furthermore, drivers are required to keep both hands on the steering wheel.

 The system (first generation) was already featured on the Volvo XC90 in 2015, but was limited to speeds of up to 30 mph, and also required a preceding vehicle to function. The XC90 will feature the same second-generation Pilot Assist as the S90 and V90 in 2017.






Volvo Cars: Verification testing of Autopilot autonomous driving technology with 100 customers to begin in 2017

 From 2017, Volvo Cars will choose commuter routes in Gothenburg, Sweden’s second largest city, and conduct a “Drive Me Project” public road demonstration experiment for its autonomous driving vehicle based on the XC90 with 100 of its customers.

 The test vehicles will feature a camera, radar, laser scanner, 3D map, and a computer Volvo calls the “Autonomous Driving Brain.” In contrast to the previous Pilot Assist, the test vehicle will have technology called “Autopilot.”

 With Pilot Assist, drivers are responsible for monitoring and supervising the vehicle, as well as intervening in the event of emergencies, but Autopilot frees them to do other things. With regards to safety, as long as autonomous driving functions are used properly, Volvo Cars will bear responsibility. The OEM feels that a product that claims to feature autonomous driving but places responsibility on the driver in the event of an accident is not a complete product.

 Moreover, with Autopilot, the driver can switch between autonomous and manual driving at will. In the “Drive Me Project,” drivers will be required to drive if they go outside of designated areas.

 The first autonomous driving test vehicle rolled out in September 2016, and after strenuous testing by Volvo’s engineering department, the vehicles will be handed over to customers starting in 2017 for verification testing.

 For test runs, in addition to technical challenges, the drivers are to use the vehicles in their daily lives to seek to find any issues that require addressing.

 In 2017, similar testing will be conducted in London and China.

 Through verification testing, the automaker plans to increase the technology’s level of completion, and aims to release the system to the market around 2021.

 Volvo Cars has stated it requires concentrated knowledge of various fields to develop autonomous driving technology. In August 2016, the OEM signed an agreement with the major ridesharing company Uber to develop next-generation autonomous driving technology. In September, the automaker also agreed to establish a software development company for autonomous driving with Autoliv, a major automobile safety system developer.



Autopilot verification testing to begin in 2017 in Sweden, London, and China (Photo: Volvo Cars) Pilot Assist and Autopilot comparison (Photo: Volvo Cars) Autonomous driving technology featured on Autopilot (Photo: Volvo Cars)



Vision for autonomous vehicle interiors: Concept 26

 In November 2015, Volvo Cars announced its Concept 26 vision for the interior layouts of future autonomous vehicles at the Los Angeles Auto Show, and also exhibited the concept at the Detroit Auto Show 2016. Drivers feel that they want to hand over long-distance driving and commuting to autonomous driving systems, and the layout was named Concept 26 because the average time spent in a vehicle for commuting is 26 minutes. Drivers and passengers are freed from driving duties and are able to spend their time in the vehicle doing other significant tasks.

 When in autonomous driving mode, the steering wheel recedes and sits lower than normal, the seats recline, and a large screen appears from the dashboard to provide entertainment.



Concept26: Layout of manual driving mode (Photo: Volvo Cars) Concept26: Layout of autonomous driving mode (Photo: Volvo Cars)

Keyword

Nissan, Volvo Cars, Autonomous driving

<Automotive Industry Portal MarkLines>