Autonomous Driving and AI - Risks and Opportunities

Initiatives for automated vehicle - Mercedes-Benz, VdTÜV, and IAV



  This report contains a Spotlight article from Springer's automotive technical magazine "ATZ" and "MTZ" Springer is a German company affiliated with MarkLines.

About Springer Fachmedien Wiesbaden GmbH

  Established in 2010 as a subsidiary of Springer Nature Company, Springer Fachmedien Wiesbaden GmbH is one of the world's leading scientific and technical publishers. It mainly publishes technical journals and books for engineers and social scientists, in addition to holding various technical events.

About Automotive Technology Magazine ATZ

  Motorwagen-Zeitschrift (Motored Vehicles Magazine) was founded in 1898 as an automotive technical magazine. From 1929, the name of the magazine was changed to "ATZ (=Automobiltechnische Zeitschrift: Automotive engineering magazine)". In addition to being published in German, the English edition has been available since 2001.



Material from: Richard Backhaus, Correspondent of ATZ/MTZ/ATZ electronics, Autonomous Driving and AI - Risks and Opportunities, IN THE SPOTLIGHT article from ATZ electronics worldwide [05/2020] [the websites of Springer Fachmedien Wiesbaden GmbH, which are available at] reproduced with permission of Springer Fachmedien Wiesbaden GmbH.





Autonomous Driving and AI -
Risks and Opportunities

Artificial intelligence and machine learning are widely regarded as essential to the development of self-driving vehicles. But how are these technologies to be deployed to actually replace drivers, and where does work still need to be done?


 The ability to process the sensor signals generated by a car’s own driver assistance systems, as well as additional car-to-X communication data will not suffice to implement the higher levels of driving automation (SAE 4 and 5) for everyday road traffic. This is because driving a car requires an intelligent interpretation of overall traffic situations. Discussions on how to address this problem often include reference to the promise of Artificial Intelligence (AI). While such an approach may seem revolutionary, it is actually only an extension of computer programming. While programmers used to devise and join explicit rules of calculation to form algorithms, we now have the additional option of deriving the rules from vast amounts of data. “Every computer that executes algorithms to perform tasks is essentially a case of AI. That is nothing new. What people commonly refer to as AI is often a special application of AI known as Machine Learning (ML). The use of ML in the development of self-driving vehicles has expanded dramatically in recent years; and ML is already an integral part of the many driver assistance systems used, for instance, to detect road signs,” Prof. Lutz Eckstein explains. As the Director of the Institute of Automotive Engineering (ika) at RWTH Aachen University in Germany, Eckstein heads up a team whose work centers on the further development of this very technology.

 The principle behind ML is to train computers to make decisions based on large volumes of data. For instance, the algorithms at work in today’s driver assistance systems serve mainly to process sensor data to detect or recognize objects. Going beyond this, ML can also help when it comes to predicting how other vehicles will behave, thereby providing a basis for a more holistic assessment of traffic situations and enabling decisions on the appropriate driving maneuver. Engineers at Mercedes-Benz expect AI to establish itself as a standard technology for self-driving vehicles within the next ten years. “AI can be used in this area for environmental perception, interpreting traffic situations and planning actions. ML can then be used to acquire knowledge based on experience. That is to say, the machine learns to recognize patterns from a vast number of stored examples. This enables machines to use statistical records to also detect and comprehend even the most complex of circumstances,” says a company representative. While that will significantly improve the quality of object detection, localization and risk assessment for automated driving, representatives of the German Association of Technical Inspection Agencies (VdTÜV) point out that the learning process is so computation-intensive that it would essentially have to be performed by manufacturer-operated host systems and not within the vehicle. That is why ML is deployed in the backend to train the system.


Camera image and semantic segmentation with the help of deep learning

“When it comes to offline learning, the function of the AI algorithm is changed exclusively within a defined training phase prior to vehicle operation. In the later operative phase, the function of the components remains constant, which means that the validation can take place largely in the development period,” says Richard Goebelt, Head of the Transport and Mobility Division at VdTÜV. In contrast, cyclical learning involves a stationary AI algorithm that records information in the background and that can then be used at a defined point in time for additional offline learning. Once such iterations are complete, the algorithm needs to be re-evaluated and rolled out to the vehicles in the field. “Connections to cloud services make sense when it’s a matter of being able to develop the AI quickly. Driving data is recorded in a multitude of vehicles and uploaded to the cloud,” says Mirko Taubenreuther, Senior Vice President Automated Driving Functions at the Germany-based IAV. There, they are then available to one or more AIs for purposes of learning. “This enables a cloud-linked AI system to benefit from a variety of data sources. And the system’s performance is further enhanced as the volume of training data increases.” 
 Proceeding from the cloud-service end, over-the-air data transmission can be used to update and improve vehicle-based algorithms long after their initial, in-vehicle deployment. This, however, is an area where alternative approaches to the use of ML are available. 
Experts at the ika, for instance, envision a scenario in which the cloud service provides an assistive real-time perception of the vehicle environment and cooperative on-road decision making. “We’re currently working on this approach together with various colleagues from other universities - for instance, in the framework of the UNICARagil lighthouse project sponsored by the German Federal Ministry of Education and Research (BMBF). One other option would involve an extensive relocation of vehicle-based processing algorithms to the cloud,” says Eckstein. 
This would represent a promising solution for driverless vehicles that are used, for instance, in the area of public transportation. However, others regard this latter approach as somewhat far off on the horizon. Engineers at Mercedes-Benz, for instance, suggest that cloud-based AI applications such as interpreting traffic scenarios are not currently viable, essentially because vehicles need to be able to react in real time to critical situations, which can only be done today with the use of onboard sensor and processing systems.

Result of a survey conducted in Germany on the prerequisites that are to be met to ensure the safety of AI applications [2]




 Extraordinary driving situations, such as those triggered when other drivers fail to abide by traffic laws, present a challenge in the area of traffic prediction. Self-driving vehicles must be prepared for such situations and at least be capable of stop-ping safely in case of doubt in order to minimize the risk of an accident. For instance, such problem-solving compe-tence is essential whenever an illegally parked delivery vehicle is blocking the lane ahead of us. In this situation, a fully automated vehicle would have to decide whether to wait a few minutes (or even hours) or simply cross the double yellow line (a traffic infraction!) to pass the delivery vehicle. The dilemma here is that drivers can decide on their own whether to take the risk of committing traffic infractions, while a self-driving vehicle cannot. “A self-driving vehicle’s repertoire of driving maneuvers should generally not include maneuvers that violate the rules of the road. After all, the rules of the road rightfully have the status of essential safety criteria, and thereby should not be overridden arbitrarily,” says Taubenreuther. Experts at Mercedes-Benz also point out that current legislation forbids self-driving vehicles to violate traffic laws. As a work around, one could imagine delegating authority to a person at a control room who is in contact with the vehicle. According to IAV and Mercedes-Benz, ad hoc commands could be issued by a monitoring authority to resolve the decision-making limitations of self-driving vehicles. The responsibility for such actions would thereby remain with an individual (or legal entity) who could also be held legally accountable.

Result of a survey in Germany on the problem of who is responsible when an automated vehicle causes an accident [2]

 Working on the basis of a systematic assessment of self-driving applications, the Advisory Council for Automated and Connected Driving at the Association of German Engineers (VDI) has issued specific recommendations for handling such issues.These recommendations also address various legal matters [1]. In the interest of traffic flow, the Advisory Council suggests that priority should be given to harmonic merging into traffic. “That being said, the resulting tolerance cannot be quantified,” says Eckstein who, as the Chairman of Vehicle and Traffic Engineering at VDI, was a coauthor of the publication. In general, the authors emphasize the importance of establishing a broad-based consensus on the driving repertoire of self-driving vehicles so that they indeed lead to an improvement in overall traffic flow and safety.

 The experts also agree that emergency vehicles operated, for instance, by the police and fire department will have to be connected to all other vehicles on the road, that is, via car-to-car communication. Moreover, many experts recommend the use of exterior detectors in self-driving vehicles to ensure their capacity to detect the acoustic signals sent by emergency vehicles. "At our Institute, we’re deploying AI to enhance the capacity to detect typi-cal signal patterns such as sirens with the use of exterior microphones. Such extraordinary driving situations are a case where self-driving vehicles, like their human- operated counterparts, would of course be permitted to violate the rules of the road,” says Eckstein. In this respect, Goebelt points out the related legal limitations, “Essentially all driving maneuvers entered in the system are possible. This also includes driving maneuvers that are technically illegal. This underscores the need for action on the part of lawmakers, namely, to harmonize legislation govern-ing road traffic and that governing vehicle registration. It is essentially incumbent on lawmakers to determine the permissible range of driving maneuvers.”


Dr. Marco Zeuner
Senior Manager Autonomous Driving – Perception and Functional Testing at Mercedes-Benz


ATZelectronics _ What are the advantages of joining AI and cloud computing?

ZEUNER _ Driving tasks are currently executed by vehicle-based AI systems with no connection to a cloud service. Some of the beneficial applications that cloud computing will enable include: software updates to enhance existing AI systems that are based on data derived from real traffic situations; remote support services to resolve vehicle stalemating; and an array of driver-experience functions such as our MBUX.

What is the procedure for certifying an AI application or an AI update for automated driving functions?

Autonomous driving is a safety-critical function that requires comprehensive verification and validation, as well as approval according to established and certified procedures. Mercedes-Benz has many years of experience when it comes to introducing safe AI systems in the area of driver assistance - for instance, road-sign detection. We’ll be able to rely on this experience as we move forward to more complex autonomous-driving functions.



In the framework of the UNICARagil project, the ika is working with other institutes to develop viable options for relocating AI systems for autonomous driving to cloud services


This admonishment coming from the VdTÜV calls attention to a problem that is currently just as unresolved as the issue of certification and periodic inspections of self-driving vehicles. Moreover, experts at ika would hasten to add that the problem presents a challenge that begins as early the development process. It is therefore necessary to select an AI architecture that will permit sufficient monitoring of each artificial neuronal network both in terms of learning success outside the vehicle and during vehicle operation. Going forward, the VdTÜV plans to promote mandatory independent inspections of individual systems with their algorithms and data.

“In addition to current functioning, future inspection procedures should include the impact and further development of the system in operation,” says Goebelt. The VdTÜV points out that it enjoys broad-based public support for its initiative. This is evidenced by the results of a recent survey the VdTÜV commissioned the IPSO market research and consulting firm to carry out, indicating that 84 % of respondents (out of a total of 1000 individuals above the age of 16) support the idea of mandatory inspections by an independent agency to help ensure the safety of all important AI systems [2]. The VdTÜV’s demands regarding to the legal framework center on three main areas. First, the scope of existing legislation will need to be expanded, given that software, even if it is relevant to safety, is generally not regarded as a product within the terms of EU product-safety legislation.

Second, provision will have to be made for AI-system evolution. “The integration of AI software into products can significantly alter product functioning throughout a long service life. This applies especially to systems that require frequent software updates or that rely on ML. In such cases, new risks can arise that were not present when the system was originally commissioned,” says Goebelt. Third, the definition of safety will need to be expanded because the use of AI in products and services can lead to risks (for example relating to cybersecurity) that are not explicitly addressed in current EU legislation.




In light of the unresolved issues, it is likely that we have a long way to go before self-driving passenger vehicles become a common feature of our streetscapes here in Europe. IAV engineers suggest that it would be possible from a technical perspective to implement full automation in as early as two to three years, primarily enabled by the high-capacity sensor systems and computers that are available of within reach. “When one takes account of the legal aspects associated with product conformity and vehicle registration, however, it’s not likely that self-driving vehicles will be plying the streets of Germany for another five to ten years. This has to do with the many issues of certification and market approval for AI-based functions that remain unresolved, as well as the need to extensively revise the corresponding regulations,” says Taubenreuther. Eckstein voices a similar opinion, “While we will quickly have applications such as fully automated parking in production vehicles, I personally don’t expect full automation for regular urban individual driving before the year 2030.”

Richard Backhaus


[1] Dietmayer, K.; Eckstein, L.; Form, T.; et al.: Automatisiertes und autonomes Fahren. VDI-Handlungsempfehlung 2019. Online: automatisiertes-und-autonomes-fahren, access: February 20, 2020

[2] Verband der TÜV e. V. (ed.): Sicherheit und Künstliche Intelligenz. Studienbericht. Online:, access: February 20, 2020



“AI is a development method like many others. It introduces a wide range of new opportunities, and also harbors many risks. I think that it’s incumbent on us to present nothing but a clear-headed account of the facts to the public, and that emotionally charged accounts are likely to do a serious disservice by generating overly optimistic expectations, or even considerable anxiety, depending on the particular audience. This is because the use of AI in the automobile industry is ultimately about autonomous driving, and thereby whether Europe is setting itself up to fall behind Asia and the USA in an important development. And yet with safe and reliable automated driving systems based on transparent methods of AI development and seamless certification and monitoring procedures, Europe could actually stand to gain a competitive advantage.”

Richard Backhaus
ATZ | MTZ | ATZelectronics.


Springer, ATZ, Autonomous, ADAS, AI, Machine Learning, Computer, Cloud Service, OTA, V2X, Daimler, Mercedes-Benz, VdTÜV, IAV, ika, UNICARagil

<Automobile Industry Portal MarkLines>