3D Maps for Autonomous Driving: Standardization and Update Method Studies Advance
Autonomous Vehicle and ADAS Japan 2016 (3)
HERE's HD Live Map
The "Autonomous Vehicle and ADAS Japan 2016" conference was organized by TU Automotive and held in Tokyo from July 11 - 12, 2016. This report will summarize efforts to study and prepare for full-scale creation of 3D maps, based on a lecture by Ichiro Hatano, General Manager In charge of Public Relations for the Autonomous Driving Systems Business Development Division at Pioneer Corporation, titled, "A fusion of laser technology and car navigation technology: The role Pioneer plays in the realization of autonomous driving", and on a panel discussion done afterward titled, "Defining the Precision 3D maps (dynamic maps) required for autonomous driving.".
* Japanese map companies, surveying companies, and nine automotive companies established the "Dynamic Map Planning Co., Ltd." in June 2016. They divided 3D map contents into cooperative and competitive areas, and started a study to standardize the cooperative areas.
* Pioneer has developed a small, low cost 3D-LiDAR. It was made to be equipped in general vehicles, and developed together with map maintenance vehicles. The company has proposed a "data ecosystem" for updating 3D maps to the latest state through information collected from those vehicles. Pioneer has also announced it is working on a demonstration test for the "data ecosystem" together with the European map company "HERE".
* Panel discussion: The discussion included statements such as the opinion that standardization should be promoted as soon as possible, which was countered by a view that careful consideration is needed because the distinction between the cooperative and competitive areas is not always clear. At the same time, HERE has been promoting standardization globally.
Promoting autonomous driving through government programs, accelerating the advancement of dynamic maps
One of the goals for the Japanese government's "SIP: Cross-ministerial Strategic Innovation Promotion Program," which is being spearheaded by the Cabinet Office, is to support the practical application of autonomous driving. The policy is for research and development of each area to proceed in order to enable semi-autonomous driving (level 2) by 2017, semi-autonomous driving (level 3) by 2020, and the commercialization of fully autonomous driving by 2025.
Five major issues were raised for autonomous driving systems: 1) Improving map information (dynamic map), 2) HMI (Human Machine Interface), 3) Information security, 4) Pedestrian accident reduction, and 5) Next generation urban transport.
Investigating cooperative areas for standardization at "Dynamic Map Planning Co., Ltd."
Mitsubishi Electric and six map and survey-related companies established "Dynamic Map Planning Co., Ltd." together with nine Japanese automotive manufacturers. The information used in high precision 3D digital maps for autonomous driving was divided into the cooperative and competitive areas, and the company is aiming to standardize data specifications for the cooperative areas shared by each automaker. Creating a 3D map for automatic driving takes an enormous amount of man-hours, and it would be difficult for OEMs to undertake such a project without partners, so standardization and open architecture are essential. The six participating companies are made up of a measurement equipment manufacturer (Mitsubishi Electric), map companies (Zenrin, INCREMENT P, Toyota Mapmaster), and surveying companies (Pasco, Aisan Technology).
(Note) Dynamic maps refer to 3D maps that provide information necessary for autonomous driving, and are continually updated and maintained. . They achieve the following functions: Specifying the position of vehicles and peripheral objects, generation of planned driving routes, serving as an intermediary for travel routes with nearby vehicles, etc.
Dynamic Map Planning Co., Ltd. Challenges
|* A common underlying data structure for 3D maps, methods for updates and position reference point management|
|* Common base data for 3D maps and an operational system concept for the cooperative areas of dynamic map (measuring method, charting method, server system, distribution method, security system, etc.)|
|* Maintenance plan (schedule/necessary cost), offer price|
|* An interface with equipment including government systems for road management, dynamic data systems, and automaker's probe systems|
|* An interface with similar overseas systems|
|* Standardization activities (domestic and international)|
|* A utilization draft of common base data for 3D maps in other applications such as infrastructure maintenance, and disaster prevention and mitigation|
|* Map update methods by means of probe information|
6 companies that participated in the establishment of "Dynamic Map Planning Co., Ltd."
|Company name||Investment ratio||Relationship with maps for autonomous driving|
|Mitsubishi Electric||18%||The company has developed an on-board measuring device called MMS (Mobile Mapping System). It is equipped with a maximum of 6 cameras and four laser scanners that measure 3D coordinate data and record video of the road and surroundings continuously. Mitsubishi Electric has sold the system since 2007. The new planning company president is from this company.|
|Zenrin||17%||Japan's largest map production company. It has also started production of 3D maps for autonomous driving.|
|Pasco||17%||A vendor for products such as satellite photos, aerial photography, city 3D data maps, and map digital data. It also uses Mitsubishi Electric's MMS.|
|6%||A provider of analysis technology for determining more accurate positions to "survey design," "construction," and "land and house investigation" industry customers. It also has multiple MMS units.|
|INCREMENT P||6%||A provider of maps and car navigation systems that is a wholly owned subsidiary of Pioneer.|
|6%||A maker of map databases for car navigation systems. Map databases are sold for purposes other than car navigation systems like mobile phones.|
|(Note) In addition to this, nine automotive manufacturers (all Japanese automotive manufacturers except for Mitsubishi Fuso Truck Bus and UD trucks) are participating, and each has invested 3.3%.|
MMS (Mitsubishi Mobile Mapping System)
|In 2007 Mitsubishi Electric released a mobile mapping system that can easily retrieve high precision 3D space data. It has been utilized for purposes such as public surveys, research, and the creation of road ledger diagrams necessary for road maintenance. It can also be used to create 3D maps for autonomous driving.|
|3 GPS antennas, an Inertial Measurement Unit (IMU, equipment to estimate the current position with GPS), laser scanners, and integrated units are installed on a vehicle roof. It can hold up to 6 cameras and four laser scanners. The system takes measurements with a high degree of accuracy even on general roads (maximum speed 60 km/h) and highways (maximum speed 80 km/h) without traffic restrictions.|
Dynamic Map Planning Co., Ltd.'s business model
The new planning company has announced the following business model. It will proceed with standardization based on the assumption that data will flow in this manner.
Surveying companies and map companies will utilize equipment such as the Mitsubishi Electric's MMS to create 3D maps (part of the common base data) based on acquired data and sources including road information. Semi-dynamic information (information on accidents, traffic jams, etc.) will also be added, and the 3D maps provided to map companies and automotive manufacturers. Even after that, maintenance will continue through probe information (information from cars driving on the road with equipment including LiDAR equipped). Automotive manufacturers and map companies will then use their respective in-house know-how (competitive areas) to complete and apply the 3D maps in autonomous driving.
According to the new planning company, the question of how to gather dynamic, semi-dynamic, semi-static, and probe information is an important issue for future investigation. For dynamic information (information for the vehicle surroundings, pedestrians, traffic signals, etc.) in particular there is an influential view that principally speaking it is possible for this information to be incorporated into all of the vehicles equipped with maps via the cloud, but "sensors and decision making ability installed in the vehicles themselves will be used to handle events that occur in real time."
Pioneer: Developing an "Eco data system" through its small 3D-LiDAR
Pioneer has utilized signal processing technology and optical technology cultivated through its work on products like optical disks to develop a small 3D-LiDAR driving space sensor that has the high performance required for advanced driver assistance and autonomous driving. The principle verification prototype was completed in September 2015, and development and in-vehicle demonstration experiments aimed at attaining high performance, small size, and low costs have started.
In 2016 the company has been engaged in developing a 3D-LiDAR for map maintenance vehicles. These are low cost, compact, and simple items that are cheaper than the LiDAR that is currently in use.
Pioneer aims to commercialize the 3D-LiDAR for general vehicles from around 2018. This equipment is incorporated into areas including the headlights and bumpers, and it is used both in the autonomous driving of the vehicle itself, as well as to send information about the vicinity to the cloud. Since road conditions are constantly changing, maps are living things in a manner of speaking. Map data will need to be processed for discrepancies (updated to the latest conditions) using information on areas collected in real time from these general vehicles. Pioneer refers to this "sophisticated and efficient map operation and maintenance system" that can be run at a low cost as a "data ecosystem." It seems that Pioneer's 3D-LiDAR is mainly targeted at the data ecosystem rather than mapping vehicles.
Joint "Data ecosystem" demonstration experiments by Pioneer and HERE
A HERE HD Live Map, which not only has road width, but also displays white lines, lanes and nearby 3D objects in addition to the road.
HERE, the top vendor for car navigation maps, has an 80% market share in the U.S. and Europe. The standard car navigation maps for vehicles operated by human drivers cannot be used for autonomous driving. Because of this, HERE has been focusing on 3D maps for autonomous driving and ADAS. Surveying vehicles equipped with LiDAR made by U.S.-based Velodyne have been working towards the creation of 3D High Definition (HD) maps.
In addition, the company is collecting road information from general vehicles equipped with sensors such as LiDAR and cameras with the same aims Pioneer has for its "data ecosystem," and an "HD Live Map" system that updates map data to the latest conditions was announced in January 2016. Changing road conditions are immediately ascertained and joint experiments with global automotive manufacturers and parts manufacturers will be boosted further so that map data is regularly updated. The partnership with Pioneer is part of such efforts.
Pioneer and HERE jointly agreed in May 2016 to conduct a demonstration experiment as a step towards the establishment of a data ecosystem. The roles of the two companies are as follows:
< Pioneer > Utilizing a 3D-LiDAR that is under development and vehicle location technology, Pioneer will extract the data necessary for autonomous driving.
< HERE > The company will utilize the cloud and its high-precision maps for autonomous driving to collect and analyze the data extracted by Pioneer (such as sensor data). HERE will then investigate mechanisms to deliver updated map data to vehicles.
HERE was the map department of the Finnish company Nokia, but it was purchased in 2015 by an industry association composed of Daimler, BMW, and Audi. This association is open, and it is expected that the number of investors will increase, with the members set to welcome new entrants.
Panel discussion: Defining the Precision 3D maps (dynamic maps) required for autonomous driving.
After the lecture by Mr. Hatano, a panel discussion was carried out with the title of "Defining the Precision 3D maps (dynamic maps) required for autonomous driving." Standardization of items such as data specifications are required for the creation of 3D maps, and many opinions were offered on the subject of how to proceed with this process, as well as on what role "Dynamic Map Planning Co., Ltd." should play as it spearheads this undertaking.
Panel discussion participants
|Ichiro Hatano||Pioneer Corporation||General Manager, In charge of Public Relations, Autonomous Driving Systems Business Development Division|
|Satoshi Yamada||Toyota Mapmaster Incorporated||Director|
|Mandali Khalesi||HERE Japan||Head of Asia-Pacific, Automotive|
|Michio Takegawa||Zenrin Co., Ltd.||General Manager, ADAS Strategy Planning Office|
|Fusatoshi Abe||INCREMENT P Corporation||General manager, New Business Development Department|
|(Note) Companies in blue text participate in "Dynamic Map Planning Co., Ltd." The moderator was Pioneer's Mr. Hatano.|
< About the establishment of Dynamic Map Planning Co., Ltd. >
* There was an explanation of "Dynamic Map Planning Co., Ltd.", which was established in June 2016, from Mr. Takegawa of Zenrin (a member of the new planning company). Since the creation and maintenance of 3D maps is very expensive, the companies separated various aspects into cooperative and competitive areas. The concept for the cooperative area is to lower costs so that all participants can use the maps.
< Distinction between competitive and cooperative areas, timing to promote standardization >
* Several panelists offered the opinion that the current distinctions between competitive and cooperative areas are not clear. Mr. Yamada of Toyota Mapmaster stated that, "the distinctions between cooperative and competitive areas are still unclear, and we want to calmly and carefully identify them. In car navigation systems, there is a sequence of events where the maps and applications are developed side by side. I expect that even for autonomous driving, once the way the maps are used is worked out, the matter of the two areas should become clearer."
* Mr. Abe of INCREMENT P stated that he would like to immediately promote standardization and work out this distinction.
< HERE expects rapid standardization >
* (HERE's Mr. Khalesi) HERE expects the speedy advance of standardization in Japan, and since many Japanese cars are sold overseas, they are also expected to meet global standards. HERE has already proposed a single format for HD Live Maps for autonomous driving. Since the amount of map information has been rapidly increasing, there is an urgent need for this information to be standardized and efficiently stored in the cloud. If it cannot be standardized in cooperative areas, further differentiation will also be impossible.
* HERE hopes to offer the service as a platform.
|(Note) HERE proposed standardization in June 2015 to continue to aggregate analysis of sensor data from vehicles driving on the road in the cloud. Various formats are currently used for each automobile manufacturer.
In June 2016, HERE submitted a standard data format named the "SENSORIS Innovation Platform" to ERTICO (ITS Europe), from which it received approval. More than ten automotive manufacturers and parts manufacturers have already participated in SENSORIS (Japanese manufacturers Pioneer, Aisin AW, others are Daimler, Bosch, Continental, etc.). The number of participating manufacturers is also expected to increase in the future.
HERE aggregates sensor data in a common language from hundreds of thousands of vehicles driving on the road in the cloud, and continually updating 3D maps in near real time is key for the realization of autonomous driving.
* Audience members offered opinions like, "Won't it be cheapest if HERE's system is introduced as is?", and "We should already be working to keep this technology from being developed too specifically for the Japanese domestic market."
< (Question from the audience) Will MMS be used by the new planning company? >
* (Mr. Takegawa, Zenrin) MMS has seen considerable use until now, so I expect that it will be used widely in the future as well.
< The relationship with maps for car navigation systems >
* (Mr. Abe, INCREMENT P) It is conceivable that maps for car navigation systems can be linked with and improve high precision maps for automatic driving (such as displaying parking lot entrances in detail). On the other hand, maps originally for car navigation systems, those for ADAS, and autonomous driving-use maps are on a different level from one another. In addition, in the case that there are plans for a road to be opened, while it is possible for a car navigation-use map to have a note added saying "scheduled to be opened soon," in autonomous driving-use maps such an entry cannot be made because the road has not been opened yet for the measurements to be taken.
These were the main opinions expressed and questions asked during the panel discussion, which lasted for about 1 hour. It was clear that there were subtle differences in perception among the participants with regards to the details.
Keywords: ADAS, autonomous driving, maps, LIDAR
<Automotive Industry Portal MarkLines>