The future of car-mounted camera-based systems

Use of monitoring and sensing cameras to rise sharply

2013/01/16

Summary

 The car-mounted cameras debuted in the form of monitoring cameras for providing back-view and side-view, and later front-view as well. In addition to the use as a monitoring camera, the front monitoring camera was soon used as a sensor for keeping distance from another vehicle in front and staying inside the lanes. The number of car-mounted cameras topped approximately 10 million units in 2011 and their market is said to be worth around 35 billion yen.

 Efforts for technical development toward functional improvement and cost reduction of car-mounted cameras are expected to increase in line with such movements as legal requirement of back-view cameras on passenger vehicles in North America, similar requirements of lane departure preventive systems and collision preventive braking systems in the United States and Europe, and their addition to the new vehicle assessment list. Backed by the users' new awareness resulting from the functional improvement of camera-based systems, the number of car-mounted cameras is expected to increase sharply to 40 million in 2015 and to more than 50 million in 2020.

 The auto industry sees the car-mounted camera feature as one of their car's appealing and differentiating performances and is likely to engage in fierce competition in developing camera-based systems.

 To gain insights into the industry's trend, MarkLines interviewed development staff of Fuji Heavy Industries making news with the unique EyeSight feature that uses a stereo-camera, and the representative of the Japanese corporation of Mobileye that is providing mono-camera based recognition technology for OEMs. Details of these interviews are introduced in this report under separate sections about the trends among OEMs and suppliers.

(References) Regulatory and other moves
(1)  "Kids Transportation Safety Act" (K.T. Safety Act) in North America
An NHTSA's initiative requiring the use of back-view cameras to prevent accidents involving children when backing up from private garages in North America.
(2)  Required use of LDW+FCW (NHTSA) in North America
In 2011, NHTSA required the use of LDW (Lane Departure Warning) + FCW (Forward Collision Warning).
(3)  NCAP (New Car Assessment Program) in Europe
The Vehicle-AEB (autonomous emergency braking against vehicles) will be added to the NCAP requirements in 2014 and the Pedestrian-AEB (autonomous emergency braking against pedestrians) in 2016.


Product and technological development among OEMs

Subaru EyeSight Ver.2: Product differentiation with a stereo-camera system

 Findings from an interview with Mr. Yutaka Hiwatashi, Subaru Technical Research Center, Fuji Heavy Industries, who has been involved in the development of the stereo-camera system from an early stage, revealed the history of development, unique characteristics of the stereo-camera, and its future directions.

(1) History of development

History of development of EyeSight (Japanese text only)
History of development
End of the
1980s
Dawn of the history of stereo-camera technologies
1999 First-generation ADA fitted in the Legacy
Four functions Distance controlling cruise control
Distance warning
Lane departure warning
Curve warning and down-shift control
2001 Second-generation ADA fitted in the Legacy
Five functions Four above functions + VDC preview control
2003 Third-generation ADA fitted in the Legacy
Eight functions Five above functions plus
Follow-up monitor
Grip monitor
Lead vehicle start alert
Characteristics Addition of millimeter-wave radar
To assist the driver at night or in foggy weather conditions beyond the stereo-camera's capability
2008 EyeSight Ver.1 fitted in the Legacy
Eight functions Collision avoidance and impact mitigation
  Pre-collision throttle management
  Pre-collision braking control
  Pre-collision braking assist
Driver's fatigue mitigation
  Adaptive cruise control with all-speed range tracking function
  Lead vehicle start alert
Preventive safety functions
  Distance warning
  Lane departure warning
  Lane sway warning
Characteristics Newly-integrated stereo-camera
For system cost reduction and functional improvement
2010 EyeSight Ver.2 fitted in the Legacy
Functions Performance improvement of pre-crash braking
"No collision" at differential speed less than 30km/h if all other conditions are met
Characteristics For further cost reduction and performance improvement

(Note) ADA: Active Driving Assist


 When Subaru engineers embarked on the pursuit of vision-based automotive safety systems, they chose the stereo-camera as the sensor because of the belief that functions similar to those of human eyes were critical to the development of a car that recognized constantly changing traffic environments (as shown the picture below) and avoided impacting accordingly.

Concept of ADA (Japanese text only)
ADA: Active Driving Assist

(2) Market penetration of EyeSight Ver.2

 Sales of Subaru-branded passenger cars were 11,887 units in November 2012, approximately 2.5-folds more than the result (4,841 units) in the same month a year earlier (according to Japan Automobile Dealers Association - JADA statistics). Even more significantly, the Impreza, Forester and Legacy, all of the Subaru group, made to the Top 30 model sales chart at 10,672 units combined.
The EyeSight Ver.2 is available nearly equally on all three models. For instance, the EyeSight was fitted in 86.6% (of orders as of one month after vehicle's launch) of the new Forester launched in November 2012, in 82.3% of the new Impreza XV launched in September 2012, and approximately 90% of the Legacy and Outback sold from December 2011 to February 2012. These percentages prove the high level of interest in "cars that do not collide" and the JADA statistics seem to indicate that the availability of the EyeSight was one of the important reasons for buying the new cars.

 

(3) Characteristics of the stereo-camera system

1) Change in the system configuration

Characteristics of the stereo-camera system (Japanese text only)
Characteristics of the stereo-camera system


  * The 1st-, 2nd- and 3rd-generation ADA had the camera fitted away from the image processer. The camera was fitted on the rearview mirror and the image processor was located under the front seat.
  * EyeSight Ver.1 and Ver.2 have the imaging, recognition and control computing functions all centralized inside the camera itself to reduce the system cost while improving accuracy.
  * Image signals from the right- and left-side cameras are digitalized and the main stereo image processing is performed by the newly-developed 3D image processing engine (supplied by Hitachi) while recognition is performed by an image recognition microcomputer.

2) In-house development of image recognition algorithms
  * Every development is being done entirely in-house from the dawn toward the end of the 1980s and up to date.
  * The company has a competitive edge with massive know-how concerning three-dimensional object recognition algorithms and road surface condition (lane, obstacles, etc.) recognition algorithms.
  * In developing the 2nd-generation EyeSight, new algorithms were developed in-house that would realize "cars that do not collide" at a differential speed less than 30km/h, and the algorithm was implemented in the Ver.1 hardware.

 3) Another in-house development theme was the know-how concerning the camera fixture, alignment and inspection.
  * For example, a new method was developed for completing alignment within the line pitch when fitting the camera in off-line condition in the factory.

 4) The company has filed for a large number of patent protection regarding its know-how:
  * About 200 patent applications regarding the stereo-camera
  * About 120 patent applications regarding the three-dimensional object recognition algorithm including pedestrian detection, and about 20 patent applications regarding the road surface condition recognition algorithms
  * More than 20 patent applications regarding the camera fitting, alignment and inspection

 

(4) Future directions of development

Future directions of development (Japanese text only)
Future directions of development

 The interviewer was impressed by the interviewee's high level of confidence in their stereo-camera-based system development and the notion that advanced technologies were truly valuable only after they were put to practical applications.

 Consequently, the company is likely to aim for further improvement of the recognition performance benefiting from the ranging function of the stereo-camera through the following:

1) Continuation of the in-house development of the stereo-camera image processing technology addressing the following:
  * Greater field of view: Wider view angles and longer detection ranges
  * Evolution of object recognition and processing technology: Improved recognition performance of detectable objects and increase kinds of detectable. Both of which will contribute to product differentiation from other systems in the form of mono-cameras and the fusion between mono-cameras and radar.

 (Note) The intent of continued development is also evident in the comment made during the press conference announcing the all-new Forester (November 2012) "we have nearly completed technical development to raise the differential speed for collision avoidance from 30km/h to 50-60km/h."

2) It appears the company will rely on the advancement of the auto industry's technical evolution for camera-related technologies other than the front-mounted cameras and stereo-cameras:
  * Use whatever meets the company's spec requirements
  * In other words, the company will concentrate all its development resources to stereo-camera systems.

 

Product and technical development among OEMs

 Development efforts for camera-based systems among OEMs are listed in the table below. They include camera-based systems, radar-based systems and also systems that combine cameras and radar.

Company Object of detection System description
Toyota Areas around
the vehicle
Multi-angle monitor (with four mono-cameras)
 Multi-camera systems supplied by Fujitsu Ten and Alpine are implemented in the Alphard, Vellfire, Estima, Prius, etc.
Front LKA (Lane Keeping Assist)
 A mono-camera system detects lanes and controls the vehicle to stay inside.
 Used in the Alphard, Vellfire, Crown, SAI, etc.
Pre-crash safety system
 Millimeter-wave radar and stereo-camera are combined to assist in avoiding collision by automatically applying brakes if the relative speed from another vehicle/object is less than 40km/h. A near infrared projector is used to detect pedestrians at night.
 Used in the LEXUS LS.
Adaptive high-beam system (AHS, a mono-camera built into the rearview mirror)
 When the system detects the taillight of the vehicle in front or headlamp of the oncoming vehicle, the system automatically blocks the unwanted portion of own beams to prevent blinding the other driver.
 Used in the LEXUS LS.
Blind spot monitor (BSM)
 Millimeter-wave radar is used to detect another vehicle running behind that is otherwise undetectable with the side mirror and alerts the driver to assist when changing lanes.
 Used in the LEXUS LS.
Honda Areas around
the vehicle
Multi-view camera system (with four mono-cameras)
 Detects obstacles around the vehicle. Used in the Odyssey, Step WGN, etc.
Front High-beam support system (with a mono-camera built into the rearview mirror)
 Detects headlight of the oncoming vehicle or tail lamp of the car in front and switches the beam from high- to low-beams.
 Used in the Accord, etc.
Forward collision warning system
 A lead vehicle detection system using a mono-camera.
 Used in the Accord Coupe for North America launched in 2012
Autonomous Emergency Brake: Economy type (automatic braking and alert at low speeds)
 A lead vehicle detection with laser radar. Being developed (unveiled during Honda Meeting 2012).
Autonomous Emergency Brake : High-function type (with ACC and lane recognition function)
 Millimeter-wave radar and mono-camera. Detects objects including pedestrians. Stops the vehicle running at 60km/h or lower speeds. Being developed (unveiled during Honda Meeting 2012).
Nissan Areas around
the vehicle
AVM (Around View Monitor) (with four mono-cameras)
 Images from four cameras are synthesized and viewing point-converted to provide overhead-view images.
 Used in the Serena, Elgrand and more.
MOD (Moving Object Detection) implemented in the Altima for North America. A rearview camera detects any obstacles while backing up from the garage.
A rearview camera warns the driver of lane departure. Used in the Altima for North America.
 Lane recognition by a rearview camera.
Blind spot warning. Used in the Altima for North America.
 A wide-angle rearview camera detects a vehicle in rear sides and warns the driver.
The above functions are implemented in the rearview camera system for product differentiation regarding the KT Act requiring a rearview camera.
Front LDW (Lane Departure Warning) / LDP (Lane Departure Prevention)
 A mono-camera detects lanes and prevents the vehicle from running off lanes.
 Used in the Infiniti and other models for North America.
Forward collision avoidance assist system
 A mono-camera and radar assist the driver in avoiding collision with the vehicle ahead (warning and braking control).
 Being developed (unveiled during Advanced Technology Briefing 2012).
Autonomous emergency steering
 A mono-camera, radar and laser scanner search for an obstacle-free direction and select emergency braking and emergency steering to avoid collision. Being developed (unveiled during Advanced Technology Briefing 2012).
Subaru Front EyeSight
 Used on the Legacy, Impreza, Forester, etc.
A stereo-camera system detects front obstacles and lanes. The system has the pre-collision braking, adaptive cruise control with all-speed range tracking function, lane departure warning and other functions.
 Introduced first in 1999 as ADA (Active Driving Assist) .
 Introduced as EyeSight Ver.1 on the Legacy in 2008 after downsizing and cost reduction.
Revised as EyeSight Ver.2 in 2010 after further functional improvement and cost reduction (selling price 100,000 yen) on more models.
Mazda Front, rear sides i-ACTIVSENSE
 Used on the Atenza.
Front-mounted millimeter-wave radar (76GHz) and a mono-camera detect the vehicle in front and activate, as appropriate, the front obstacle alert, automated braking control (SCBS: Smart City Brake Support), lane departure warning, and Acceleration Control for Automatic Transmission.
 Rear-side-mounted millimeter-wave radar (24GHz) is also used to detect vehicles approaching in rear sides (RVM: Rear Vehicle Monitoring).
Volvo Front City Safety
 Implemented on the XC60. A laser radar-based vehicle collision avoidance system.
Human Safety
 Used on the S60. A millimeter-wave radar and a mono-camera based collision avoidance system against vehicles and pedestrians.
BMW Surround Surround View
 Used on the 3 Series. A parking assist system using five cameras (right and left sides of the front bumper, right and left side mirrors, rear) to provide a synthesized image.
Front SmartBeam DFL(Dynamic Forward Lighting)
 Implemented on the 3 Series.
 A CMOS camera (supplied by Gentex and mounted on the rearview mirror) detects headlamps and tail lamps of other vehicles.
FCW (Forward Collision Warning)
 Used on the 1 Series.
 A frontal collision avoidance system with a mono-camera (Mobileye EyeQ2).
LDW (Lane Departure Warning)
 Used on the 1 and 3 Series.
 Detects lanes and warns the driver of lane departure.
Mercedes
-Benz
Front, rear,
rear sides
Surround vision (to be implemented on the next S-Class models)
 Front Long Range Radar + Mid-range Scan, Front Short Range Radar, Rear Multimode Radar
 Rear-side Short Range Radar × 2
 Front stereo-camera, front infrared camera
Front obstacle avoidance (steering, braking)
 Stereo-camera and radar used to detect vehicles and pedestrians
Intersection accident avoidance (radar)
 Pedestrian and animal alert
Lane keeping assist (including rear-side vehicle warning)
 Rear collision avoidance, damage mitigation
Headlight control
Audi Front Variable headlight range control
 Used on the A3.
 A car-mounted camera detects light sources on other vehicles and controls the host car's beams accordingly.
Active lane assist
 Used on the A3.
 A car-mounted camera detects lanes and assists the driver with lane departure warning and steering control.


 The sensor configurations and system functions of some of the OEMs listed above are described below.

 

(1) Toyota

 System configuration of the pre-crash safety system implemented on the LEXUS LS in which a front-mounted stereo-camera and millimeter-wave radar are used as sensors. The system not only detects another vehicle in front, it also has a near infrared projector for detecting pedestrians as well. System configuration (Japanese text only)
System configuration of the pre-crash safety system

 

(2) Nissan

 Special efforts are being made to develop higher value-added rearview cameras. Nissan unveiled, at Advanced Technology Briefing held in October 2012, a new rearview camera that realized all three functions.

1) Moving Object Detection (MOD)
    Detects any moving object when backing up and alerts the driver.

2) Lane Departure Warning (LDW)
    The rearview camera detects lanes and warns the driver of lane departure.

3) Blind Spot Warning (BSW)
    Detects and alerts the driver of other vehicles in rear sides when attempting lane change.

 

(3) Mazda

 Sensor configuration of the i-ACTIVSENSE. The front-mounted mono-camera is combined with millimeter-wave radar to detect other vehicles ahead. The system also has a pair of quasi-millimeter wave radar to detect vehicles approaching from rear sides. Sensor configuration of the i-ACTIVSENSE

 

(4) Mercedes-Benz

 A front-mounted stereo-camera is combined with front long-range radar, front and a pair of rear-side short-range radar, rear-mounted multimode radar and a near/far infrared camera to detect obstacles and pedestrians in all directions of the vehicle. Mercedes-Benz

 

 



Parts Suppliers

Mobileye: Increasing use of mono-camera based safety systems on automobiles

(1) Company profile

Main office Netherlands
Development site Israel (approx. 300 engineers)
Characteristics World leader with image recognition technology based on monocular cameras. Has SOC dedicated to image recognition with newly-developed algorithms.
* EyeQ1 and EyeQ2 have been adopted on vehicles. EyeQ3 with improved collision avoidance performance has been developed.
Applicable system Lane Detection/Vehicle Detection/General Object Detection/Pedestrian Detection/Traffic Sign Recognition/ Headlamp Detection
Customers
(taken from Mobileye's website )
OEMs
Opel, BMW, Chrysler, GM, Ford, Hyundai, Jaguar, Peugeot Citroen, Volvo, Yulon Motors, Scania, two Japanese OEMs
Tier 1 Suppliers
Autoliv, Calsonic, Delphi, Gentex Corporation, Leopold Kostal, Magna Electronics, Mando vorp, TRW, etc

 

(2) Interview with Mr. Norio Ichihashi, Chairman, and Mr. Shotaro Kawahara, CEO, Mobileye Japan, about the present product development and future plans

1) Start of the company
   * When asked by an OEM for advice about developing a collision avoidance system using a stereo-camera, the engineers replied "such a system can be implemented with a monocular camera." This inspired the engineers to develop recognition technologies using mono-cameras.
   * They established a company in Israel to develop such a mono-camera based system.

2) Available functions

2007, 2008 2011, 2012 2014 2016
Implemented functions LDW
 (GM,BMW)
LDW+CMB
 (Volvo)
LDW+IHC+TSR
 (BMW)
LDW+FCW
 (GM etc.)
LDW+IHC+TSR+FCW
 (BMW, Opel etc.)
    LDW+LKS
    (Hyundai, Ford)
CAR AEB PED AEB
Regulations North America NHTSA required the use of LDW+FCW Europe NCAP-Vehicle AEB Europe NCAP-Pedestrian AEB
(Note)  LDW  Lane Departure Warning
LKS  Lane Keeping Support
CMB  Collision Mitigation by Braking
FCW  Forward Collision Warning
IHC  Intelligent Headlamp Control
TSR  Traffic Sign Recognition
AEB  Autonomous Emergency Braking
PED  Pedestrian


 

 * The system was first used on vehicles in 2007 by Volvo, GM, and BMW. Today, it is used by nine OEMs that were driven by the NHTSA regulations in the United States requiring the use of the LDW + FCW systems.

 * The system is likely to find its way to more vehicles (up to 16 OEMs) as the European NCAP's new requirement is introduced in 2014.

 * The system is implemented in two styles, one that uses a mono-camera only and a fusion system that uses a mono-camera combined with radar. They are used selectively according to the OEM's product differentiation policy such as using the fusion type on higher-end models.

 

3) Characteristics of the Mobileye mono-camera system

Image recognition SOC Comprises a customizable image processing CPU (VMP=Vector Microcode Processor) and a general-purpose CPU (originally ARM, replaced by MIPS)
Software The internally-developed software comprises three hierarchies (core > application > parameter).
The user normally realizes specifications for its product by selecting applications and adjusting parameters.
The core software and application software are updated every six months based on analytical results of drive data collected from the user.
Recognized objects Present:
 Lead vehicles, pedestrians, obstacles, light source (headlamp, backlight), traffic signs (speed limits)
 (Objective recognition and estimation of its position (distance))
Extendable in future:
 Animals, traffic signs (of varying shapes), traffic signals, etc.
Improved accuracy of range detection Range detection by EyeQ2 was occasionally unstable compared to that of millimeter-wave radar and its use in adaptive cruise control (ACC) was limited in terms of applicable vehicle speeds. This shortcoming was corrected and the next-generation system (EyeQ3) is fully functional at all vehicle speeds.

 

4) Drive database

a way of collecting and analyzing massive drive data * Product development at Mobileye is unique in that it has a way of collecting and analyzing massive drive data and using the results to improve and develop recognition algorithms.

 * Mobileye operates a special database system that allows the company to incorporate image data from driving tests, performed under various meteorological conditions in all parts of the world, to gain insights for developing vehicles in which their vision cameras are installed.

 * The company analyzes data to extract information to improve recognition algorithms accordingly. The improved software is made available to all users.

 * The vast amount of collected data and know-how fed back from market applications is a major advantage to Mobileye and its image database is growing constantly. The company is leading all others in terms of the system accuracy and coverage.

 

5) Direction of technical development

Technical development Evolution of recognition technology
  * Extended recognizable objects
  * Free space recognition
    * Acquisition of depth map by a mono-camera
  * Detection of road bumps and potholes
    * Detection of sidewalk bumps
Extended scenes of application
  * Extended application of recognition technology as in the higher vehicle speeds for ACC
Self-Driving Car development
  * Initiate the development of self-driving car with several camera counted around the vehicle
  * Possible application includes driver assist in traffic congestion.
    * Volvo has announced plans to use the new technology in 2014. Other OEMs are studying the feasibility of use.
Product strategy The company will maintain the current business configuration in which common technologies are provided to several OEMs through Tier 1 suppliers.
The "front monocular sensing camera" is the basic product but Mobileye will make sensor fusion available if demanded by OEMs (as has actually been the case).
The need for rearview camera systems will be met by a new multi-functional product that incorporates inputs from the rearview camera into the "front sensing system" which is the company's basic product. In other words, the rearview capability will be incorporated as a variation of the basic product (this will be achieved when image data transmission on Ethernet becomes a public practice).

 

Development efforts among parts suppliers

Tier1 Tier2 Camera
Denso For lane recognition, etc. Has radar fusion technologies.
Has developed stereo image processing ECU.
Aisin AW Detection of road surface conditions using rear-mounted camera
Used with G-Book mX Pro and other navigation systems
Fujitsu Ten For surround-view monitors
Used by Toyota
Honda Elesys For lane recognition, etc.
Provided for Honda
Clarion For surround-view monitors, rearview monitors, etc.
Provided for Nissan, Fuji Heavy Industries, etc.
Alpine For surround-view monitors
Provided for Toyota
Panasonic For surround-view monitors, rearview monitors, etc.
Provided for Japanese OEMs
Hitachi AMS Stereo-camera (including image processing LSI)
Used on EyeSight
Valeo For surround-view monitors, lane recognition, etc.
European OEMs
Continental Mono-camera (lane recognition, etc.), stereo-camera (pedestrian detection)
Integration of laser radar and mono-camera being developed (SRL-CAM)
Bosch Mono-camera (lane recognition, etc.), stereo-camera (downsizing)
Fusion system comprising a mono-camera and radar has been developed (for Audi, etc.)
Mobileye World leader in image recognition technology using mono-cameras
Technologies being provided for many Tier 1 suppliers. Used by nine OEMs today.
Started selling collision preventive assist systems for aftermarket application


 Japanese suppliers started with the development of monitoring cameras. Their recent goal is to develop sensing cameras for control purposes.

 European suppliers are more experienced than the Japanese with the development of sensing cameras for control purposes.

 Mobileye cooperates with a wide range of suppliers regarding monocular sensing sensor technologies. In contrast, others such as Denso, Bosch, Continental and Valeo are developing their systems independently without teaming up with Mobileye.

 

 



Direction of evolution

(1) Present condition of systems and their future trends

Sensors Present and future
System name Stereo-
camera
Mono-
camera
Milli-
meter-
wave
radar
Laser
radar
Ultra-
sonic
Infra-
red
sensor
Pre-crash safety
(automatic emergency brake)
(obstacle recognition technology)
EyeSight will continue the stereo-camera only system configuration
Used on the LEXUS. Their use was announced by Mercedes-Benz and will remain as a high-function version.
Mobileye is confident about the realization of its mono-camera only system. Many OEMs will likely use it as a low-cost version.
More functional than the mono-camera only system. This system configuration will likely continue as a high-function version.
Today's mainstream but is likely to step down in front of camera-based systems.
Pedestrian detection and warning Pedestrian detection technologies will be established in conjunction with camera-based systems for other purposes.
(○) Sensor fusion was once considered to be in high demand but is likely to be replaced by camera-only systems.
Use will continue for night-time detection. The infrared light projection type with a daytime camera for light reception will increase.
Lane detection and warning / lane keep control Pedestrian detection technologies will be established in conjunction with camera-based systems for other purposes.
Pedestrian detection technologies will be established in conjunction with camera-based systems for other purposes.
Lane detection systems with rearview cameras will increase as a low-cost version.
Approaching vehicle detection
(headlamp control)
Pedestrian detection technologies will be established in conjunction with camera-based systems for other purposes.
Sign and signal recognition Pedestrian detection technologies will be established in conjunction with camera-based systems for other purposes.
Parking assist
(guidance)
This type has expanded the market in Japan.
Combined use with ultrasonic sensors will likely become the mainstream.
These will likely become the mainstream.
Use has increased in Europe and other markets. Combined use with camera-based systems will likely become the mainstream.
Parking assist
(moving object detection)
Nissan is leading others.
Fiercer competition is expected for compliance with the K.T. Safety Act.
K.T. Safety Act compatible
(back-view monitor)
Being developed by many companies to prepare for the K.T. safety Act enforcement in North America. Unit prices of $159-$203 with display, $58-$88 without display are often heard.
Nissan has introduced moving object detection, lane departure warning and other systems for product differentiation.
Development race is expected for product differentiation using a standard feature of rearview cameras.

(Note) ◎: Mainstream Sensor, ○: Currently used sensor, (○); Sensor for some systems

(2) Trends of monitoring camera development

 The use of the rearview cameras and surround-view monitors is increasing in Japan. The K.T. Safety Act is being introduced in North America. Parking assist systems are growing popular in Europe. These market conditions will lead to the following:

1) Increased use of monitoring cameras

2) Fiercer competition in incorporating product differentiating technologies in monitor camera systems (multi-functionality such as Nissan's moving object detection and lane keeping assist technologies).

 

(3) Trends of control sensing camera development

 Advancement and cost reduction of camera technologies will have significant impact on front obstacle detection for driver assist technologies:

1) Sensor fusion with radar technology will become less necessary and OEMs will be able to choose between camera-only and sensor fusion systems to suit the specific level of system requirements.

2) Multi-functionality from a single camera system

3) OEMs will be able to choose between mono-camera and stereo-camera systems according to their system requirements and strategy.


 

(Reference) Comparison: Mono-camera systems vs. Stereo-camera systems

Item Mono-camera systems Rating Stereo-camera systems
Image recognition accuracy * Equal performance in principle = * Equal performance in principle
Ranging accuracy * Calculated from the ground surface image and the camera position * Ranging based on parallax
Rain and fog compatibility * Equal performance to human eye Equally
low
( = )
* Equal performance to human eye
Integrity of single system * Lane markings detection is fully performed.
* Difference will likely remain between camera-only systems and millimeter-wave radar fusion systems in terms of collision alert and collision avoidance control
(up to OEM's use policy)
* Commercially realized by Subaru (EyeSight)
* Camera-only systems will likely become the mainstream and lead systems comprising stereo-cameras and millimeter-wave radar.
Ease of inspection* and adjustment * Easier than stereo-camera systems * Harder than mono-camera systems because of the need for optical axis alignment and brightness balancing of two cameras.
Cost * Fewer part count than stereo-camera systems * Best cost reduction efforts are done such as sharing of the image processing and recognition subsystems but still cost higher by the cost of an imaging element than the mono-camera systems.
Others * Mobileye is the technical leader in the field of camera-only systems * Subaru owns a vast amount of know-how and intellectual property rights that leave others far behind.

 

<Automotive Industry Portal MarkLines>