Nissan: Connected strategy in the age of autonomous cars

Adoption of Microsoft's Connected Vehicle Platform, aiming for fully automated driving in 2022



Easy Ride
Field testing of the Easy Ride driverless vehicle jointly developed with DeNA (Source: Nissan)

This report is based on a presentation given at TU Automotive Japan 2017 (in October 2017) by Mr. Masaru Muramatsu, the head of Nissan's Connected Car & Telematics Development Group, the HD Map Development Group of Nissan's AD & ADAS Advanced Technology Development Department, and its Connected Car and Autonomous Drive Division. This report outlines Nissan's autonomous vehicle connected car strategy, focusing mainly on the "Connected Car in the Autonomous Era".

The Nissan Intelligent Mobility innovation roadmap consists of three core elements: 1) Nissan Intelligent Power, which is represented by EV technology, 2) Nissan Intelligent Driving, represented by autonomous driving technology, and 3) Nissan Intelligent Integration, represented by Connected Car technology. Nissan believes that the cars of the future will be electric, autonomous, and connected through a combination of the fruits of advancements being made in these three areas of innovation. Nissan is aiming to realize fully autonomous (driverless) driving in 2022.

Renault-Nissan has adopted Microsoft's Cloud Azure-based Connected Vehicle Platform. The platform contributes to fully-autonomous driving using features such as state-of-the-art advanced navigation systems, remote monitoring of vehicle status, and over-the-air (OTA) software program updates. At the same time, the platform aims to free the driver from driving tasks to enrich their time spent in an automated vehicle by providing a mobile connection to the outside world and services.

Nissan will introduce its Seamless Autonomous Mobility (SAM) platform, a remote monitoring system for automated vehicles, in technical field testing of the Easy Ride driverless vehicle mobility service that has been jointly developed with DeNA since the end of 2017. In January 2018, Nissan announced that it plans to conduct additional testing to further advance its Seamless Autonomous Mobility (SAM) platform that was jointly developed with NASA by conducting a working demonstration of the technology. SAM represents a major milestone for its Nissan Intelligent Integration infrastructure technology, which is considered key to accelerating the deployment of autonomous mobility services. In Japan and the U.S. there are legislative proposals to require remote monitoring systems for field testing of driverless vehicles on public roads.

Mr. Muramatsu also mentioned that, from 2020, 5G communication technology will gradually evolve to a point where vehicles will acquire new functions and characteristics and result in the availability of new services and business.

In January 2018, Nissan announced its plans to collaborate with five companies including Continental AG and Qualcomm, to conduct its first Cellular Vehicle-to-Everything (C-V2X) trials in 2018. Nissan aspires to provide new safety and convenience services in a timely manner, once the 5G technology is available.

Related reports:
Renault-Nissan-Mitsubishi Alliance: Projecting sales of 14 million units in 2022 (October 2017)
Tokyo Motor Show 2017: Exhibits from Nissan, Honda, and Mitsubishi (December 2017)



Renault-Nissan: Introducing EV, autonomous driving, and connectivity technologies on vehicles for the mass-market

The Renault-Nissan Alliance has set a goal for itself to become a leader in each of the three core areas of innovation that comprise the Nissan Intelligent Mobility concept: 1) Nissan Intelligent Power (represented by EV technology), 2) Nissan Intelligent Driving (represented by autonomous driving technology), and 3) Nissan Intelligent Integration (represented by Connected Car technology). In addition, these technologies will enable innovation in the area of new mobility products and services such as car and ride sharing.

The Alliance plans to lead the industry in offering EV, autonomous driving, and connectivity technologies, not only on luxury cars, but at affordable prices on major segment vehicles for the mass market.

The future vehicles of Renault-Nissan will combine these three technologies.

Nissan Intelligent
The three elements of Nissan Intelligent Mobility (Source: Nissan)



Advancing autonomous driving technology, from Standalone to Connected

Following the release of the Nissan Serena in August 2016, which features ProPILOT single-lane highway driving assist technology, the Renault-Nissan Alliance plans to adopt various levels of automated driving technology on 40 models by 2022. The autonomous driving technology deployment schedule is as follows:

2018: Highly automated vehicles for highway driving (autonomous driving on multi-lane highways, including merging onto the highway and lane changing; the driver always needs to monitor the environment)
2020: Highly automated vehicles for urban areas (the driver always needs to monitor the environment)
2020: Highly automated vehicles for highway driving (driver involvement when needed)
2022: Fully automated vehicles (driver involvement unnecessary, enabling driverless operation)

Nissan Connected Car
Nissan Connected Car Intelligence from Standalone (On-board intelligence) to Connected (Source: Nissan)

Mr. Muramatsu showed how Nissan's Connected Car Intelligence changes from a Standalone (on-board intelligence) to a connected system, as shown in chart on the right. For example, the ProPILOT system installed in models such as the Serena and Leaf allows automated highway single-lane keeping and is controlled only by on-board intelligence installed on the vehicle, but the technology to be introduced by Nissan in the future will be connected to communicate with the cloud, enabling remote intelligence and social intelligence to firmly anchor mobility services.

Renault-Nissan tie-ups with major IT companies such as Microsoft strengthens the software development capability of the Alliance, as Renault-Nissan will need to develop huge volumes of software to build such a connected system. As of October 2016, Renault-Nissan announced that its connected car team will consists of 300 personnel at both Renault and Nissan, with plans for the Alliance to hire an additional 300 employees, for a total of 900 people. Nissan has established a new development center for this project in Meguro-ku, Tokyo.

In January 2018, the Alliance announced the launch of Alliance Ventures, a new corporate venture capital fund that plans to invest up to USD 1 billion over the next five years focused on next generation mobility, including vehicle electrification, autonomous systems, connectivity, and artificial intelligence. According to Mr. Carlos Ghosn, the chairman of Nissan, "This investment activity is aimed at attracting the world's most promising automotive technology start-up companies to the Alliance." Renault (40%), Nissan (40%) and Mitsubishi (20%) will jointly fund Alliance Ventures.



Microsoft's Connected Vehicle Platform and its contribution to automated driving

In January 2017, the Renault-Nissan Alliance became the first automaker to adopt the Microsoft Connected Vehicle Platform, a set of services based on the Microsoft Azure cloud. Microsoft says that by using its Connected Vehicle Platform, OEMs can benefit from the huge sums that it has already invested in the Azure cloud. Adoption of Azure will enable Renault and Nissan to build a common connectivity platform to consolidate their existing separate platforms.

Unlike Waymo, the autonomous car development subsidiary of Alphabet Inc., Microsoft is not a competitor building cars, so it can be considered a trusted partner. This platform is not an in-car operating system or a "finished product", but a system jointly designed with an automaker tailored to the customer's needs and existing embedded hardware and software.

The platform aims to address five core scenarios: predictive maintenance, improved in-car productivity, advanced navigation, customer insights (deepening understanding and strengthening customer relationships), and help building autonomous driving capabilities.

The platform seeks to realize highly automated and driverless driving, for example, by being constantly connected to the Azure cloud to receive the latest information on roads and surrounding environment as well as OTA software updates. Advanced navigation, especially in area-restricted Level 4 automated vehicles, will show areas where it is possible to drive in automated mode using 3D highly automated driving (HAD) maps provided by the system and displayed as a Geofence (area surrounded by virtual boundaries). Manual driving mode is required for vehicle operation outside of the Geofence areas.

Renault-Nissan has not announced the vehicle models that will adopt the Microsoft Connected Vehicle Platform or the details of the technologies to be realized.

Contributing to improved driving time productivity in fully automated vehicles

When fully automated driving is realized, the driver will be freed from the driving task and can spend the driving time relaxing, or use the time for work or sleeping. So, the above "improved in-car productivity" relates to making the vehicle's interior a home or office space by the use of features such as the Cortana virtual personal assistant, and the work productivity tools Office 365 and Skype for Business.

Cortana: An in-vehicle personal assistant that can understand the driver's preferences, providing a variety of driver support services managing the driver's personal schedule, sending reminders, giving wake-up calls, conducting web searches and tell us the results. With Cortana drivers can benefit from advanced in-vehicle voice recognition and intuitive Human Machine Interface (HMI).

Office 365: A group of Office productivity software and related services such as Outlook, Word, Excel, PowerPoint, provided on a subscription basis at a JPY 7,000 to 18,000 per year.

Skype for Business: An instant messaging, voice calling, telephone conferencing, and video conferencing.



Introducing Seamless Autonomous Mobility (SAM) remote monitoring system at the Oppama Plant, Easy Ride service

Teleoperation for
Autonomous Vehicles
An operator remotely monitoring and controlling the vehicle "Teleoperation for Autonomous Vehicles", as announced by Phantom Auto at CES 2018
(Source: Phantom Auto)

CEO Ghosn has stated that although autonomous driving technology is evolving daily with AI, what is currently available has not reached a level where the system can respond appropriately to unanticipated driving situations (e.g. traffic accidents, new construction areas). This current technological challenge is hindering the realization of autonomous driving. As a result, Nissan has developed a remote monitoring system called Seamless Autonomous Mobility (SAM) that is unveiled at the CES 2017. It is said to be the ultimate in Nissan Intelligent Integration.

For at least the first few years after the introduction of autonomous vehicles, a remote monitoring system such as SAM will be required (while AI technology learns how to successfully cope with difficult autonomous driving situations) for vehicles to operate safely, enabling the early deployment of fully automated vehicles for the mass market.

In December 2016, Nissan introduced the Intelligent Vehicle Towing (IVT) system, which transports finished vehicles with driverless towing cars from the Oppama assembly plant to the facility's wharf where they are loaded onto ships. The towing vehicle is equipped with an array of two cameras and four laser scanners that automatically guide it to the designated unloading point, but instances may occur where the system is unable to negotiate its route unaided. In this case, for example, if two driverless towing cars intersect and are unable to navigate the situation, the SAM control center will intervene to assess the situation, determine the right of way, and send instructions for the new route to be taken by each of the towing vehicles. The SAM platform is also introduced in the section below entitled "Easy Ride".

In January 2018, Nissan announced plans to conduct a working demonstration in the U.S. of their Seamless Autonomous Mobility (SAM) platform jointly developed with NASA to further advance the technology. In the future, Nissan is also considering offering the technology to third-party organizations for applications such as ride-hailing services.

At the CES 2018, Phantom Auto, a U.S. venture company, announced its "Teleoperation for Autonomous Vehicles" concept, which is similar to that of SAM, and conducted a demonstration of its remote monitoring system. Phantom Auto expects to receive orders for their service from at least two OEMs and will be able to announce its first customer soon.

For details on Seamless Autonomous Mobility and Intelligent Towing Vehicle, refer to the previously report:
Seamless Autonomous Mobility (SAM): Responds to unpredictable situations during autonomous driving



Joint field testing of Easy Ride driverless vehicles with DeNA

The Renault-Nissan Alliance is jointly conducting field tests of driverless vehicles with DeNA in Japan and Transdev in France. These tests will allow the Alliance to provide vehicles for new mobility services such as driverless vehicle dispatch systems, public transportation, and car sharing.

In January 2017, Nissan and DeNA started collaborating to develop a new mobility service, named "Easy Ride", using autonomous driving technology for driverless vehicles. Technical field testing using driverless autonomous vehicles (a modified "Leaf") initially began in 2017. The companies plan to conduct public field tests of the new Easy Ride robo-vehicle mobility service for 2 weeks in the Minatomirai district of Yokohama, Kanagawa Prefecture, Japan from March 5, 2018, to March 18, 2018.

The goal of the Easy Ride service is to allow customers to use a dedicated mobile app to complete the whole process from setting destinations and summoning vehicles to paying the fare, including the selection of recommended destinations and sightseeing routes depending on their objectives. With "more freedom of mobility" as its concept, Easy Ride is envisioned as a service for anyone who wants to travel freely to their destination of choice in a robo-vehicle. Nissan and DeNA aim to provide the service as a new infrastructure option that supplements existing transportation in each area of the country, and are targeting to start offering the mobility service on a full-scale basis in the early 2020s.

Furthermore, Nissan and DeNA plan to offer vehicle remote monitoring services with control centers to manage and support automated driving 24 hours a day.

As a trial example of the remote monitoring system, Sony developed the New Concept Cart SC-1 (shown in photo below-right) aiming to provide a new mobile experience, and started demonstration experiments in Okinawa from September 2017. In addition to being able to be driven in manual mode, the SC-1 can also be monitored and operated remotely from as far away as Tokyo when the system is connected to the cloud using NTT Docomo's 5G communication technology. The maximum speed of the SC-1 is 19 km/h in EV driving mode.

Easy Ride Control center New Concept Cart
Started field testing of the driverless vehicle developed with DeNA (Source: Nissan) Control center remotely monitors and supports Easy Ride (Source: Nissan) Sony started field testing of the New Concept Cart, a remote controlled automated vehicle (Source: Sony)

In Japan and the U.S., remote monitoring systems will likely become a standard requirement to receive permission to test driverless vehicles on public roads

In June 2017, the Japanese National Police Agency announced the permit standards for field testing of driverless vehicles on public roads controlled by a remote monitoring system.

In the previous version of the permit standard to test fully automated and driverless vehicles on public roads it was necessary that an on-board occupant be present to intervene to control the steering wheel or apply the brakes in the event of an emergency. Under the new standard, driverless driving tests on public roads are allowed on the condition that the remote monitoring system operator have a driver's license and is constantly monitoring the driving environment to immediately apply the vehicle's brakes in the event of emergency. In the event of an accident, the remote monitoring system operator is to be held responsible as the legal driver.

Field testing of driverless vehicles equipped with a remote monitoring system have been conducted throughout Japan from December 2017.

The California Department of Motor Vehicles (DMV) in the U.S. is considering legislation for the testing and deployment of automated driverless vehicles (SAE Levels 3 to 5) in California. One of the key points of the legislation is to permit the testing of driverless vehicles on public roads. In the latest (November 2017) draft of the regulations, the remote operator must be actively monitoring the vehicle's operations and there must be a communication link between the vehicle and the remote operator to provide information on the vehicle's location and status and allow two-way communication between the remote operator and any passengers. Also, in the event of an emergency, the system must inform the remote operator to immediately take over physical control of the vehicle.

These proposed regulations are close to being finalized and are expected to be adopted in 2018.



The outlook for new automated driving and connected services using 5G in the 2020s

In the 2020s, 5th generation (5G) ultrafast wireless communication technology is expected to be realized. 5G will likely be able to send 100 times faster than the current 4G technology, using 100 times more simultaneous connections, and shorten the time needed for data transmission to about 1 millisecond, a tenth (1/10) of the present level.

Established in September 2016, the 5G Automotive Association (5GAA) is a cross-industry organization of companies from the automotive, technology, and telecommunications industries (ICT), including Nissan, working together to develop end-to-end solutions for future mobility and transportation services.

With automated driving it will be possible to apply huge volumes of data to derive, for example, the latest navigation map and environment information. Also, the application of AI technology to automated driving has the advantage of allowing the vehicle to instantly judge, for example, a driving situation to avoid an accident. In the aforementioned example for Sony, real-time monitoring and vehicle operation from a remote location will also be possible.

In January 2018, Nissan announced its plans to collaborate with five companies including Continental AG and Qualcomm, to conduct Cellular Vehicle-to-Everything (C-V2X) trials in 2018. The objective is to validate and demonstrate the benefits of C-V2X using direct communication technology, operated in 5 GHz band, defined by the 3rd Generation Partnership Project (3GPP) in their Release 14 specifications. The trial cases are designed to focus on Vehicle-to-Vehicle (V2V), Vehicle-to-Infrastructure (V2I) and Vehicle-to-Pedestrian (V2P) direct communications, as well as Vehicle-to-Network (V2N) operations over cellular network-based wide area communications utilizing LTE-Advanced (LTE-A). Nissan would like to provide new safety and convenience services in a timely manner, once the 5G technology is available.

According to Mr. Muramatsu of Nissan, as communication technology evolves the vehicle's connected technology is also likely to advance. Early telematics services used mobile phones for communication. With the popularization of smartphones, Nissan introduced the current Infiniti InTouch and NissanConnect connected services. Mr. Muramatsu showed that the 2020s will be an era in which more advanced automated driving and connected services will become available with 5G communication technology and full-scale utilization of the cloud.

Nissan, Autonomous driving, Connected, Microsoft, Azure, Easy Ride

<Automobile Industry Portal MarkLines>