Tesla's New Neural Net Patent Makes it Process Images More Like Human Sight

Tesla filed a new patent for its Neural Network, describing a system that makes it process images more like humans do with their sight. The Neural Network is the heart of the automaker’s Full Self-Driving Suite and Navigate on Autopilot. It is and will be instrumental to Tesla’s goal to build a fully autonomous vehicle and, in extension, its Robotaxi fleet.

The patent is titled "Data Pipeline and Deep Learning System for Autonomous Driving." The upcoming upgrade to the company's Neural Network was described in the patent’s Background, which states: 

“Deep learning systems used to implement autonomous driving typically rely on captured sensor data as input. In traditional learning systems, the captured sensor data is made compatible with a deep learning system by converting the captured data from a sensor format to a format compatible with the initial input layer of the learning system. 

"This conversion may include compression and down-sampling that can reduce the signal fidelity of the original sensor data. Moreover, changing sensors may require a new conversion process. Therefore, there exists a need for a customized data pipeline that can maximize the signal information from the captured sensor data and provide a higher level of signal information to the deep learning network for deep learning analysis.”

What has the "dog or muffin" test got to do with #AI & computer vision? Read out our latest blog post to find out https://t.co/c70uEaGpeC pic.twitter.com/NPgoOwSoRp

— Blippar (@blippar) January 26, 2017

The patent further breaks down Tesla’s Neural Network upgrade with extensive passages. With the patent, the automaker’s AI will be able to interpret and identify images better due to the new way the Neural Network will be processing individual pieces of data. 

Neural networks aren’t specific to Tesla alone. There are a bunch of companies trying to develop their own AI, and each of them needs to go through specific tests to develop. For instance, machine learning engineers often run image recognition tests to see if their AI can differentiate between objects and organisms in the real world. 

These tests often yield entertaining results. One image recognition test that has become worthy of meme culture involved a neural network being somewhat challenged with pictures of corgi butts and loaves of bread. FreeCodeCamp revealed that another test would be having AI differentiate between Chihuahua faces and muffins. There loads of other hilarious examples, and Tesla's AI isn't immune.

Tesla’s Neural Network has come far in the last couple of years, but it is still a few steps away from perfect. Just recently, Autopilot confused a small boy wearing an orange shirt for a traffic cone. So there are still a couple images that can be difficult for Tesla’s “brain” to decipher. 

The new patent will attempt to help Tesla’s Neural Network to differentiate between objects and organisms. 

With the latest upgrade, the electric car maker’s AI will process information in several layers, like shape, features, and color. Then the Neural Network reconstructs the image as a whole layer by layer, much like the human eye and the brain. 

Tesla Autopilot Confuses Boy In Orange Shirt For A Cone In Brazil https://t.co/OHCjIpEXdY via @insideevs.com

— InsideEVs (@InsideEVs) December 16, 2019

In a previous Tesmanian article, Tesla’s FSD system was compared to the brain and human senses. This patent follows the same mindset, with the Neural Network acting like a brain for Tesla’s fleet. 

The human eye is much like the cameras, sensors, and radar found in a Tesla. The eye captures an image, translates it into electrical codes, then sends it to the brain to interpret. According to Brain Facts, studies have found that the movement, depth, size, shading, texture, color, among other aspects of an object are processed separately in the brain before coming together to make one complete image. The system outlined in Tesla's new patent seems to mimic this process. 

Multi-layering is not new to AI. Tesla most likely already adopts a multi-layering processing system for its Neural Network. This patent could refer to a more refined, more efficient version of it. 

The brain is also known to fill in the gaps of specific images, which is why optical illusions have become so entertaining. According to the patent, the Neural Network will be able to do something similar to this. It will be able to distinguish an image from the data collected globally by other Teslas' sensors. 

Tesla has long been a proponent of a more natural, organic way for AI to collect data, steering away from using LiDARS for their autonomous fleet. With this new patent, the next-gen automaker seems to be doubling down on its stance.  

Featured Image Credit: Tesla/YouTube