Advanced Driver Assistance Systems (ADAS) by Bosch, Denso and Renesas

International Automotive Electronics Technology Expo 2015 lectures and exhibits



Bosch's rear mid-range radar sensor
Bosch's rear mid-range radar sensor detects vehicles behind and supports the driver in lane change maneuvers (photo courtesy of Bosch)
Bosch's rear mid-range radar sensor
Bosch's rear mid-range radar sensor also supports the view-blocked driver in backing maneuvers from a parking space (photo courtesy of Bosch)

 This report summarizes lectures and exhibits regarding Advanced Driver Assistance System (ADAS) technologies that were presented by Bosch, Denso and Renesas Electronics at the 2015 International Automotive Electronics Technology Expo held on January 14 to 16, 2015.

 The three companies plan to improve their safety and driver assistance technologies and commercialize new technologies in phases while looking ahead to achieving autonomous driving in the future.

 Bosch suggested two approaches with the understanding that automated driving starts with highway driving and parking assist functions. According to Bosch, autonomous driving would require "surround sensing technology," "safety and security technology" and "legislation." The company introduced map data and system architecture including electrification that supports the automation technologies.

 Denso gave lectures on its recognition (surround sensing) technology. The company plans to rely on a signal processing technology called Multiple Signal Classification (MUSIC) and scanning type Light Detection and Ranging (LIDAR) that help increase the accuracy of recognition. Denso also introduced plans to use quasi-zenith satellites to increase the accuracy of positioning a vehicle to the level of 10cm.

 Renesas Electronics Corporation, a semiconductor manufacturer, outlined the latest trends in the development of autonomous driving technology. Driving safety and other driver assistance technologies have become highly advanced, more objects can be detected and recognized, and the recognition speed has increased as has the power consumption of associated devices. As a result, the automobile industry is facing a stricter demand for functional safety than before. Renesas Electronics has developed the "R-Car" which is an ADAS-compliant system-on-a chip (SoC: a system LSI), and RH850 Series of 32-bit microcontrollers that provide all-in-one solution for driver assistance systems. These devices have been developed to reduce power consumption of such systems. Renesas Electronics has also developed Wireless Access in-Vehicle Environment (WAVE), a new communication solution under the "Connected Car" approach.

Related reports:
Technologies that will be key to autonomous driving (Nov. 2014)
Autonomous Driving Technology: Telematics Japan 2014 (Nov. 2014)
ITS World Congress 2014: Exhibits and demonstrations (Oct. 2014)
ITS World Congress 2014: CTO plenary session overview (Sep. 2014)

Bosch's initiatives for autonomous driving

 Bosch gave a lecture on "Bosch's way toward automated driving."

 The company suggested "Automated," "Connected" and "Electric" as the keywords of future mobility.


Future Mobility

Automated Mobility Connected Mobility Electric Mobility
* Automation technology is used initially to enhance safety. * Cloud service * Contributes to safety * Potential networking of car repair shops * Electrification technology to support autonomous driving * Contributes to Clean Driving


Automated and connected-social benefits

A Bosch survey in 2012 shows 60% are in favor of autonomous driving as long as it can be switched on and off by the driver.
Reduced congestion  Reduced congestion, less waiting time at intersections
Higher fuel efficiency  Synchronized traffic flow achieving 23 to 39% improvement in highway fuel economy
Gain in productivity  Driving time of 56 minutes per day can be used for other purposes (U.S.)
Democratization of mobility  Increasing number of senior citizens can enjoy driving
Improved safety  90% of all accidents are caused by human errors


Bosch: Autonomous driving starts with highway driving and parking assist functions

 With the understanding that autonomous driving starts with highway driving and parking assist functions, Bosch presented its roadmap to autonomous driving using existing technologies in those areas. The company stressed that "surround sensing," "safety and security" and "legislation" are the keys to autonomous driving along with "map data" and "system architecture" that support them. The system architecture includes electrification that increases functional processing capacities for autonomous driving.


Roadmap of highway autonomus driving functions

(Availability of advanced sensing technology and safety gets more critical from left to right)

Partially automated (under driver's supervision) "Connected"
Highly automated Fully automated
ACC, Lane Keep Assist Integrated Cruise Assist Highway Assist Highway Pilot Auto Pilot
Single sensor Sensor Data Fusion (note) Sensor Data Fusion + Maps
Longitudinal and lateral control Partly automated longitudinal and lateral control within a lane Automated longitudinal and lateral control; lane change upon the driver's confirmation Highly automated longitudinal and lateral control including lane change without the driver's constant monitoring Commuting between home and work in urban areas without the driver's monitoring

(Note) Data received from plural sensors are combined to correct defects of other sensors and calculate correct position.

Roadmap to parking assistance functions
In series production 2015 "Connected"
2018 2020 onward
Park steering control Park maneuver control Automatic/remote park assist Auto park pilot Valet parking
Ultrasonic sensors Ultrasonic sensors + cameras Ultrasonic sensors + cameras + map
Automated steering with the driver applying accelerator and brake The system controls steering and brake while the driver applies accelerator Partly automated longitudinal and lateral control under the driver's monitoring (in or out of the vehicle) Fully automated longitudinal and lateral control without the driver's monitoring The connected vehicle is autonomously parked with the intervention of maps, vehicle-to-vehicle and road-to-vehicle communications

 Bosch presented three prerequisites for automated driving, map data and system architecture that support them, taking examples from highway driving and parking functions mentioned above.


Prerequisites for Automated Driving

Surround Sensing Safety and Security Legislation
Radar, camera and other sensors are combined for 360-degree view sensing Defense from technical failure and cyber attacks Need for adapting legislation according to technical progress
Robust reliability at view angles otherwise undetected by radar or camera Security leaks may lead to safety risk (close relations) * Legislation regarding autonomous driving differs by state in North America * Initiative in Europe by VDA (German Association of the Automotive Industry) * Levels of autonomous driving being redefined and legislation studied in Japan
Driver monitoring is a key element for autonomous driving (system control can always revert to the driver) <Safety> Protect from technical glitch and minimize influence of failure <Security> Blocking cyber attacks


Map data and system architecture supporting the three prerequisites

Map data  Highly automated driving requires accurate, up-to-date map data. The system must be established for processing and delivering collected information via cloud services. Map data are critical to efficient routing for long-distance travel.
 Data provides information about road conditions outside the sensor range and prepares the vehicle control for impending conditions (ex: Avoid emergency stop before lane end at merging traffic).
System Architecture  Reliable fail-safe components and systems are indispensable. They will feature duplicated steering, braking and behavioral stability systems so that their functions will stop in phases rather than shutting down at once in an event of malfunction.
 An electrical and electronic (E/E) architecture is required to meet increased processing power and autonomous driving functions. Bosch offers a broad variety of electrification components such as ESP, electric power steering and iBooster.


Sensors and associated electric devices

 Bosch believes that autonomous driving functions will bring about a major change to automobile architecture. The company has a broad range of elemental technologies associated with autonomous driving and is preparing for their market launch.

 Introduced below are the sensors for Surround Sensing and associated electric devices exhibited by Bosch at the 2015 International Automotive Electronics Technology Expo.


The brake pedal force Dual-pinion type electric power steering
The brake pedal force is assisted by the motor of iBooster instead of vacuum from the engine. Dual-pinion type electric power steering exhibited by Bosch


Bosch's surround sensing technologies

Long-range millimeter-wave radar sensor (LRR4)  The LRR4 is capable of detecting other vehicles at a distance of roughly 250 meters at a radar aperture angle of 40 degrees. This allows the adaptive cruise control (ACC) to be used in speeds of more than 160km/h. Commercial production of the LRR4 began in 2014 and the sensor is used in Porsche and other high-end vehicles.
Mid-range millimeter-wave radar sensor (MMR)  The MMR is capable of detecting other vehicles at a distance of roughly 160 meters at a radar aperture angle of 45 degrees. It is more cost effective than the LRR4. This allows the automatic emergency braking and ACC to be used at speeds up to 160km/h. Shipment of the MMR started in 2013.
Rear mid-range radar sensor (MRR rear)  The MRR rear is capable of detecting other vehicles behind at a distance of roughly 100 meters maximum at a radar aperture angle of 150 degree maximum. Two sensors, concealed at both sides of the rear bumper, are used to monitor the area alongside and behind the vehicle to support lane changes and parking. Shipment of the MRR rear began in 2014.
Multi-purpose camera (MPC)  The MPC was developed to implement road sign recognition and object detection and improve radar-based functions such as ACC and lane keeping support.
Stereo video camera (SVC)  The SVC has two complementary metal oxide semiconductor (CMOS ) color image pickup devices for 3D measurement range of more than 50 meters in a horizontal field of view of 45 degrees. The video sensor complements radar technologies and creates detailed images from the sensor data to support the radar functions. The 3D detection of the vehicle's surroundings provides the basis for the autonomous driving functions of the future.

Source: Bosch's Exhibits and press releases


Bosch: Electric devices that are essential part of ADAS

iBooster  iBooster is an electro-hydraulic brake booster in which the brake pedal force is amplified by a motor rather than by the vacuum from the engine. It is highly responsive and applies the brake instantly at high pressure. This reduces the braking distance in an automatic emergency braking, by 1.5 meters when the vehicle is traveling at 30km/h. Bosch claims that up to 50% of rear-end collisions could be avoided if all cars are fitted with the iBooster system.
Electric power steering  The electric power steering (EPS) is another critical system for autonomous driving. Bosch exhibited its dual-pinion EPS that is commonly used in Europe (Bosch also supplies column type and paraxial drive type EPS systems).



Denso: Sensing technologies that support driver assistance systems

 Denso gave a lecture on "Sensing Technologies for Advanced Driving Support Systems." The company classifies system configuration of safe and secure driving supporting system into "driving environment recognition (surround sensor)," human machine interface (HMI)," "information and communication" and "vehicle motion control." The lecture focused on "driving environment recognition (surround sensor)" in particular. The lecture started by introducing safety standards in different countries and how the stricter New Car Assessment Program (NCAP) is leading to the commercial application of preventive safety and other driving support technologies.


Trends of regulation and assessment (EURO NCAP)

Preventive safety systems are included in NCAP criteria in countries such as Japan, U.S. and Europe. The current status of the leading program, Euro NCAP, is described below.
2014  Forward Collision Warning (FCW)/Autonomous Emergency Braking (AEB), Lane Departure Warning (LDW) were introduced.
2016  AEB pedestrian (during daytime) under consideration
2018  AEB pedestrian (at night), AEB bicycle (in crossing at intersections), AEB vehicle (in crossing at intersections, turning ) under consideration


Denso: Sensing for driving environment recognition

 According to Denso, recognizing driving environments will need "surround sensing," "V2X," "cloud communication" and "locator" depending on the distance from the vehicle to the object and whether the object is visible to the driver.

 Denso introduced technologies being developed to increase the performance of surround sensing. Among them were Multiple Signal Classification (MUSIC) that increased accuracy of millimeter wave recognition, color cameras, and scanning type Light Detection and Ranging (LIDAR).

 Incidentally, Toyota announced in September 2014 the progress of its development of autonomous driving technologies addressing safe driving assistance. They included Single Photon Avalanche Diode/Light Detection and Ranging (SPAD LIDAR) that was developed jointly with Toyota Central R&D Labs. According to Toyota, SPAD LIDAR contributes to improving the performance of surround sensing. As a result of substantial downsizing and cost reduction efforts, it also fits in a small space.


Sensing for driving environment recognition

Surround sensing (up to 100m ahead from the vehicle)  Surround camera, sonar, millimeter-wave radar
Out-of-range and blind spots  V2X (Vehicle to vehicle, Vehicle to infrastructure), Cloud communication, Locator


New technologies addressing enhanced surround sensing

MUSIC to increase accuracy of millimeter wave recognition  Denso faced a technical challenge regarding Adaptive Cruise Control (ACC) and Autonomous Emergency Braking (AEB) before millimeter-wave sensors could be used on compact vehicles. The challenge was to reduce their size and cost without affecting the resolution that would result in poorer recognition capability.
 As the first automotive application, Denso adopted a high-resolution signal processing technology, MUSIC. This made what was impossible with compact sensors possible: (1) Distinction between two vehicles ahead running at the same speed and (2) distinction between the vehicle ahead and roadside railing.
Color cameras  The use of color cameras made it possible to recognize rear lamps of the leading vehicle and pedestrians. In addition to the white markings used on Japanese roads, the cameras can also recognize the blue markings used in South Korea, the Botts' Dots (round raised pavement markers about 10cm in diameter that are used in North America) and the yellow lines used in Germany to identify road construction sections. They can also recognize traffic signs while driving at speeds of 150km/h to 200km/h.
Scanning type LIDAR  LIDAR is the name of a remote-sensing technology based on electromagnetic waves (ultraviolet, visible, and near infrared rays) having wavelengths shorter than radar. Their reflected light is analyzed to measure the distance to objects such as poles, edge stones, roadside trees, etc. Its high resolution power enables accurate measurement of distances to small objects and pedestrians.
 Denso has developed a mapping method (grid mapping) in which the map of the road ahead is divided into small grids to show the probability of the presence of an obstacle in individual grids. Denso's LIDAR system detects obstacle-free grids and presents them to the driver (or an autonomous driving system) as a safe path.

 Denso also introduced technologies for recognizing remote objects from the vehicle. They included ADAS locators and high-precision positioning using quasi-zenith satellites.


Collection of information about remote (or invisible) objects from the vehicle (information communication)

V2X  Vehicle-to-vehicle and vehicle-to-infrastructure communication supports safe driving.
Cloud Communication  Big data analysis provides information for safety and security.
ADAS Locator  The locator uses signals from GPS, gyro sensors and vehicle speedometer to tell the present location of the vehicle. It provides "Road Ahead Information (ADAS Horizon)" for roughly 1km ahead based on the "current position" and "map." Denso has fabricated its prototype.
Precise positioning by Quasi Zenith satellite  High positional accuracy in the order of 10cm is necessary for safely passing another vehicle. The currently available "correction by GPS and gyro sensors" produces an error of about 122cm from the reference. Denso has verified that positional accuracy within the range of 10cm can be achieved by using the compensation signal from a quasi-zenith satellite (an artificial satellite that remains in a geostationary orbit) and accurate positioning algorithm developed by Japan Aerospace Exploration Agency (JAXA).


Driver Status Monitor (DSM)
Function, performance  The DSM's function has been evolved from "monitoring the driver (facial direction, drowsiness) to "inspecting the driver (eyes, facial expression). This represents an evolution from "facial image recognition" to "facial image recognition and data analysis."
System  Denso has developed a system in which the ECU controls the camera and lights, performs image processing, and monitoring of the driver's conditions. This is followed by several actions based on the result; activate the pre-crash safety system, warn the driver by sound and/or vibration, change the air conditioning settings, give driving hints to the driver, etc. (announced in April 2014). Denso is currently in the process of developing methods of image processing of the driver's facial expressions and estimating the level of drowsiness.



Renesas Electronics develops collision-free cars using system-on-a-chip solution

 Renesas Electronics lectured on "Renesas ADAS solution for no-accident driving" and introduced three industrial trends toward enhanced safety along with Renesas' solutions. The company's approach to (1) "driver assistance" is to balance out the increasing computational load and power consumption by defining separate roles between the CPU that extracts information and the image processing engine. Its approach to (2) "autonomous driving" is to use its know-how regarding control systems for functional safety in R-Car, which is the company's high-performance system-on-a-chip (SoC: system LSI). Renesas' approach to (3) "connected cars" is to provide new communication technology, WAVE solution, that implements interoperability.


Three futures toward collision-free cars

(1) Driver assistance
Change in the content of driver assistance  The content of driver assistance is changing from passive safety (seatbelts, airbags) to active safety (lane departure warning, automatic emergency brake, collision avoidance by steering control).
 The point of interest (POI) is expanding to driving lanes, vehicles, pedestrians and their motions. More decisions need to be made resulting in increased computational load and power consumption. Predicting the pedestrians' motion, in particular, requires far more power than before.
Renesas' approach  Renesas has developed a dedicated accelerator and a central processing unit (CPU) to balance out the increasing recognition processing and the associated power consumption. While the two share memory, the CPU extracts information and meaning and the Image Renderer (IMR) and the image recognition engine IMP-X4 (collectively referred to as the dedicated accelerator) handle image processing. The accelerator increases the speed of processing by 10 to 100 times compared to when processing is done by a general-purpose CPU. The higher efficiency reduces the power consumption.
(2) Automatic driving
Change in the content of autonomous driving  The content of automatic driving is changing from "visual assistance" to "Traffic Jam Assist" of today and "driving assistance for the aged" of tomorrow.
Leap from prototyping to production  Developing a production model from the automatic driving prototype presents a number of barriers in terms of performance (balancing between recognition capability and power consumption), costs (marketability costs) and reliability (functional safety). The highest classification of safety measures, level D, is required under the Automotive Safety Integrity Level (ASIL) defined within ISO26262. A Micro-processing Unit (MPU) that meets both performance and functional safety requirements has not been developed as yet.
Renesas' approach  Renesas will continue to improve functional safety while balancing out the recognition capability and power consumption. The company already has know-how regarding functional safety which will be integrated into the R-Car, a high-performance System-on-a-Chip (SoC: system LSI).
(3) Connected car
New communication technology WAVE  Electronic Toll Collection (ETC) and LTE/3G are commonly known in today's communication technology. The next-generation vehicle-to-vehicle and vehicle-to-infrastructure high-speed communication technology, Wireless Access in Vehicle Environment (WAVE), is expected to be introduced and become a new mainstream in 2016 or so. Cross-validation of interoperability will be the key to success of the new technology. The technology was developed by plural companies and must be validated by two or more companies.
 Renesas has developed a WAVE solution and is testing its interoperability with international standardization organizations and conducting proving tests with OEM/Tier1. Renesas is participating in Plugtest (interoperability test) by European Telecommunications Standards Institute (ETSI). Renesas' technology is rated high under radio characteristics and connectivity.
Renesas' approach  Renesas will provide its WAVE solution for implementing connected cars. The company is already mass-producing the device for use in Japan (760MHz) and developing a global version (5.9GHz).


Surround Monitoring System and RH850 series of safety microcontrollers

 Renesas exhibited and demonstrated the surround monitoring system and the RH850/P1x-C series of all-in-one safety microcontrollers that meet all driver assistance needs.

 The surround monitoring system was presented as follows: The ADAS started with Lane Departure Warning (LDW), Automatic Emergency Brake (AE) and high-beam assist using front-mounted cameras. Today, the ADAS provides 360-degrees views including rear views along with functions to prevent overlooked risks. These increase the point of interest (POI) and require the use of high-definition cameras. As a result, the system's computational load increases and higher levels of functional safety are required. Renesas has developed unique low-power consumption technology while meeting those new challenges.


Surround Monitoring System Surround Monitoring System
Demonstration of Surround Monitoring System evolving from Surround View System
Demonstration of ADAS
Demonstration of ADAS on cars fitted with RH850/P1x-C series of all-in-one microcontrollers


Renesas' goals for ADAS progress

Current generation Next generation Subsequent generation (Connected)
ADAS * High-resolution 3D   surround monitoring * Simultaneous 4-camera   pedestrian recognition * Automatic driving   (on special road) * Collision prevention * prediction of pedestrian   movements * bicycle recognition * Automatic driving   (on regular roads) * Cloud-connected services * Prediction of moving   objects at intersections
Renesas' technologies and products WAVE solution
R-CAR (SoC, System on a Chip)
(Current 32-bit MCUs) RH850 (32-bit MCUs for automotive control systems)


Renesas Electronics: Surround Monitoring System

 In September 2014, Renesas started sample shipment of the R-Car V2H, the first of the company's ADAS-ready system-on-a-chip (SoC: system LSI). Building upon the currently available Surround View System, the SoC enables the construction of a Surround Monitoring System of higher visibility and resolution. Images from plural cameras are synthesized real-time after switching the field of view depending on the potential risk situation to deliver panoramic, overhead view to the driver. Commercial production of the chip will start in the second half of 2016 with a plan to produce half a million chips a month starting in October 2017.
 It is said that 30 to 40% of car accidents occur in parking area where the driver tends to concentrate on parking maneuver, feel pressed, hasten, be inattentive or careless. The currently available Surround View System "patches up" images from four cameras and leaves the rest to the driver's decision. The new system features image recognition technology that detects obstacles and pedestrians and supports the driver in the presence of overlooked risks.
 The R-Car V2H has the following characteristics:  (1) ADAS-ready functions in a single low-power chip.  (2) Availability of open-source image recognition library drastically increases the efficiency of software development.  (3) The use of the Ethernet AVB (Audio Video Bridging), considered the top choice for signal transmission from high-resolution cameras, enables a next-generation onboard camera system.

Source: Renesas' exhibits, press release Aug. 28, 2014.


Renesas Electronics: Commercial application of RH850/P1x-C Series of all-in-one safety microcontrollers

 Driver assistance systems are growing advanced rapidly toward autonomous driving. Renesas Electronics has developed the ready-to-use RH850/P1x-C Series, a higher-end version of the RH850/P1x Series of 32-bit automotive microcontrollers. The company has integrated, in a single chip, the four solutions that are required in a driver assistance system. Sample shipment of the new device starts in February 2015. Its commercial production is slated to start in the second half of 2016 with plans to increase monthly production to 2 million pieces in 2020.
(1) Safety  Safety solution: The device complies with ASIL D, the highest safety level stipulated in the ISO26262 functional safety standard for road vehicles.
(2) Security  In the era of connected cars, communication with social infrastructure is becoming a mainstream. This requires robust security measures to ensure that the system is protected from unauthorized external access.
(3) Sensors  The system collects information from many sensors such as cameras and laser devices. To make this possible, RH850/P1x-C Series features large memory capacity and powerful CPU to ensure fast processing speeds.
(4) Network  The RH850/P1x-C Series features a full array of communication functions including Ethernet in addition to CAN, LIN, CSI and FlexRay. This allows sensor fusion or complex control of chassis systems using data obtained from a variety of sensing systems.
 Applications for the RH850/P1x-C Series may include driver assistance systems. They may be used in conjunction with the R-Car family of Renesas SoCs having surround image recognition capability to detect lane markings, assist the driver in parking maneuvers, and detect leading vehicles to assist the driver. The R-Car SoC would perform image recognition based on information from image sensors, and the connected RH850/P1x-C Series would make judgments and apply control as appropriate. Such systems could be implemented using only a small number of components.

Source: Renesas' exhibits, press release Nov. 6, 2014.

<Automotive Industry Portal MarkLines>