NVIDIA: The rapid evolution of Deep Learning and its applications

Autonomous driving, AI manufacturing, and the Brain of the AI City



NVIDIA CEO Jensen Huang

NVIDIA CEO Jensen Huang

(All figures in this report are provided from NVIDIA)

 This report presents a summary of the lecture entitled “Deep Learning Opens the New Era of Robotics”, presented by Mr. Naoya Yamoto, Senior Account Manager of NVIDIA’s Industry Division. The lecture was given during the 11th Ota City Manufacturing Technology Exhibition and Business Meeting on July 6, 2018 at the Ota City Industrial Plaza PiO.

 NVIDIA was founded in 1993 and is involved in a diverse array of business areas such as gaming, artificial intelligence (AI), and autonomous driving. In the computer world, Microsoft launched Windows 95 in 1995, which was followed by mobile phones, advances in the popularization of cloud technology, and today we are experiencing the age of AI through deep learning technology.

 Around the year 2012, the development of neural network technology, the establishment of big data collection and analysis methods, as well as the performance improvements of the GPU (Graphics Processing Unit) developed by NVIDIA formed the 3 essential conditions to enable the dramatic growth of deep learning that continues to this day.
AI technology has evolved significantly through deep learning and is now used in a wide range of applications.

  In the area of autonomous driving, NVIDIA offers a computing platform (known as NVIDIA’s end-to-end platform) that covers the whole area of autonomous driving such as collection of driving data, training neural network models by deep learning, driving simulation, and driving operation.

  In addition to autonomous driving, AI has permeated into every area of all industrial sectors and has begun contributing to improvements in productivity. While various examples were introduced during the presentation, this report covers AI applications cited such as FANUC’s AI manufacturing, Musashi Seimitsu Industry’s AI visual inspection, Komatsu’s AI construction, and “The Brain of the AI City” presented by NVIDIA.



Overview of NVIDIA

 NVIDIA was founded in 1993 and in 1999 developed the GPU (Graphics Processing Unit) for computer graphics computations. It currently employs 12,000, of which 8,000 are engineers, with more than half involved in software development. The company is referred to as the “AI computing company”, focusing on business areas of such as gaming (a $100 billion industry), artificial intelligence (a $3 trillion industry), and autonomous driving (a $10 trillion industry), leveraging the same “NVIDIA GPU” platform for applications in all business fields.

The advent of the GPU computing era, and the rapid development of deep learning


 Looking back at the history of computer usage over the past 20 years, the PC revolution started with the launch of Microsoft’s Windows 95 in 1995, followed by the mobile phone (e.g. Apple iPhone) that put a computer in every pocket, advancing to the popularization of cloud that turned every mobile device into a supercomputer. Now we are entering the era of AI through deep learning, which is making it possible to solve the most complex of problems faced by computers to date.

 Deep learning has dramatically evolved since 2012 and its practical applications continue to advance. In the ILSVRC (ImageNet Large Scale Visual Recognition Challenge, a global competition for image recognition) held in 2012, a team using deep learning was able to demonstrate good results with an error rate of 16.40% (the error rate for the 2nd place and below teams were above 26%). By 2016, the accuracy rate increased to 97%, which is higher than the success rate of 95% for humans.

 Around 2012, in addition to the growth of neural network technologies as described above, the improvements in big data collection and processing power (e.g. memory) and the performance improvements in the GPU (Graphics Processing Unit) developed by NVIDIA formed the 3 essential conditions that accelerated the growth of deep learning. In deep learning, the larger the amount of input data, the greater the effectiveness of the computation results obtained. However, this requires a massive number of calculations, which have been made possible by improvements in GPU performance. GPUs used for calculations other than graphics processing are called GPGPU (general-purpose computing on GPU). According to NVIDIA, performance improvements that can be achieved by increasing the density of CPUs are reaching its limit. However, it is likely that GPUs will continue to offer performance improvements because of the GPU’s superior parallel computing architecture.

Structure of deep learning

 In AI (Artificial Intelligence), machine learning (where the AI itself learns) is a subset of AI and deep learning is a subset of machine learning.

 Deep learning utilizes “artificial neural networks” consisting of multiple layers. The artificial neural networks are said to be computational models resembling the structure of the neurons in the human brain.

 In deep learning, there are 2 phases, the “training phase” and the “inference phase”.

 In the “training” phase, 1) results are generated based on the input data, 2) differences from the expected results (right answer) are calculated as errors, 3) using the error, the weighted parameters for every layer of the artificial neural networks are updated, and 4) this process is iterated until the error diminishes, increasing the accuracy of the artificial neural network.

AI applies the learned results to the new data and makes a guess (known in AI lexicon as “inference”) being able to solve complex problems.

Deep learning is a subset of AI. The structure of an artificial neural network Flow of the deep learning training process
Deep learning is a subset of AI. The structure of an artificial neural network Flow of the deep learning training process



NVIDIA offers AI platform NVIDIA DRIVE for autonomous driving

 With respect to autonomous driving, NVIDIA has built its own self-driving test car named the BB8 (built based on various vehicles such as Ford and Lincoln) and has been conducting real-world road tests. It also partners with global automakers and Tier 1 suppliers; in Japan, with companies such as Toyota, Pioneer, and Zenrin.

 NVIDIA developed the AI platform NVIDIA DRIVE for automated and autonomous vehicles. The platform (known as an end-to-end autonomous computing platform) covers the whole area of autonomous driving such as collection of driving data, training neural network models by deep learning, driving simulation, and driving operation. Further, since it is an open platform, the NVIDIA DRIVE can be used by autonomous vehicle developers to enhance their own autonomous driving systems. It is also scalable to various development objectives, from Levels 2 and 3 of autonomous driving capability to Level 5 full automation capability, which could be referred to as robo-taxi capability.

 It is a serious challenge to accumulate enough driving distance in real world conditions to test and validate autonomous driving software, including the learning models used to improve the robustness of algorithms, so NVIDIA is focusing its efforts on its driving simulation systems.

 In March 2018, NVIDIA announced its NVIDIA DRIVE SIM simulation system and its NVIDIA DRIVE Constellation validation system using actual hardware. The systems are used to simulate a wide range of test environments including rare and challenging driving and environmental conditions such as inclement weather like rainstorms and snowstorms, limited visibility at night, and all types of roads and terrains. The learning model is then validated within those test environments. Furthermore, it would be possible to validate whether a vehicle could recognize conditions the same as under real world driving conditions by connecting the simulation system to a vehicle’s on-board AI computer equipped with the learning model. NVIDIA will start offering the system to some of its partners in the third quarter of 2018.




AI at the Monozukuri (manufacturing) site: FANUC, Musashi Seimitsu Industry

FANUC: AI manufacturing

 In October 2016, NVIDIA and FANUC Corporation announced the two companies will collaborate to implement artificial intelligence on FANUC’s manufacturing IoT platform “FANUC Intelligent Edge Link and Drive (FIELD) system”.

  In FANUC’s FIELD system, the use of artificial intelligence will make robots and machining centers smarter, with FANUC aiming to increase productivity in the manufacturing industry.
 By using artificial intelligence to analyze the operational status of robots, it will be possible to detect in advance any signs of malfunction that until now were not predictable.


Musashi Seimitsu Industry: AI-based visual inspection

 The visual inspection process in the manufacturing industry has been heavily dependent on the skill levels of human operators, so automation has been sought to improve the speed and accuracy of the process.

  In the current process, the burden on the human operator is significant and there is considerable variation in the inspection results depending on the operator’s skill level and other conditions. Musashi Seimitsu Industry has introduced NVIDIA Jetson processors to achieve unmanned/labor-saving process, improved inspection speed, and ensure stable inspection accuracy by using AI to automate the visual inspection process.

(Musashi Seimitsu Industry)

Other examples of AI applications: Komatsu, and the Brain of the AI City

 Various examples from the presentation include activities such as “Komatsu AI Construction” and “The Brain of the AI City”.


Komatsu AI Construction

 In December 2017, Komatsu announced the introduction of its AI platform centered around the NVIDIA Jetson processor to expand its “Smart Construction” program. With the GPU communicating with drones and cameras, visualization of an entire construction site can be realized, allowing for a 360-degree view to identify people and machines in the area to prevent accidents such as contacts and collisions. The platform also recognizes changing conditions in real time to provide construction equipment operators with accurate information and instructions.


The Brain of the AI City

 By 2020, 1 billion cameras around the world are expected to be operational, and it will be impossible for humans to track the massive number of images from these cameras. Using AI-based video analytics, the aim is to achieve safe and efficient cities. The NVIDIA Metropolis platform developed for the AI city will be introduced to harness its power of real-time facial recognition, license plate searches, and search for specific people. NVIDIA has over 25 partnerships throughout the world, led by Alibaba, Dahua, Hikvision, and Huawei (all 4 firms are Chinese).

Komatsu AI Construction THE BRAIN OF THE AI CITY
Komatsu AI Construction THE BRAIN OF THE AI CITY



NVIDIA, GPU, Deep learning, Autonomous driving, AI

<Automobile Industry Portal MarkLines>