Highly modern networking

At the NIPS conference in Long Beach, California, in December, Audi subsidiary Audi Electronics Venture presented an innovative pre-development project: a mono camera that uses artificial intelligence to generate an extremely precise 3D model of the vehicle environment.

Birte Mußmann (copy) & AUDI AG (photo)

The new Audi A8, for example, which made its debut at the Audi Summit in Barcelona last year, is already designed for SAE Level 3 conditional automated driving: Its optional Audi AI traffic jam pilot* takes over driving in slow-moving traffic on multi-lane roads up to 60 kilometers per hour, provided national legislation allows it and the driver selects it. A requirement for automated driving is constant precise mapping of the surrounding environment. Artificial intelligence (AI) is a key technology in this area, set to enable major progress in the future.

At the Conference on Neural Information Processing Systems (NIPS), a project team from Audi subsidiary Audi Electronics Venture GmbH (AEV) presented a mono camera that uses artificial intelligence to generate an extremely accurate 3D model of the environment. The technology, developed independently of a specific Audi model, makes it possible to capture the vehicle’s surroundings precisely. A conventional front camera acts as the sensor, covering a roughly 120 degree angle in front of the car and delivering fifteen images per second at a resolution of 1.3 megapixels. The images are then processed in an artificial neural network by means of semantic segmenting, in which each pixel is classified into one of thirteen object classes. This enables the system to identify and differentiate objects such as cars, trucks, houses, road markings, people and traffic signs within system limits. The system also uses artificial neural networks to obtain information on distances between the car and other objects. In this case, visualization is performed via ISO lines—virtual boundaries defining a constant distance. The combination of semantic segmenting and depth estimates produces a precise 3D model of the actual environment.

Audi engineers had previously “trained” the neural network by means of “unsupervised learning,” i.e. machine learning without the use of pre-sorted, pre-classified data. In this approach, the system learns by generating knowledge from observations of situations and scenarios. For this, the artificial neural network was fed with numerous videos of road situations recorded with a stereo camera. The network is able to learn and understand the rules resulting from this, which it then uses to produce 3D information from the mono camera images. This knowledge can subsequently be applied to new situations. The findings from the pre-development project are a further important step toward the next level of production automated driving systems.

Alongside AEV, two partners from the Volkswagen Group also presented their own AI topics at the NIPS conference Audi booth. The Fundamental AI Research unit of Volkswagen IT’s Data:Lab is focused on unsupervised learning and optimized control through variational inference, an efficient method for representing probability distributions. In addition, an Audi team from the Electronics Research Laboratory in Belmont, California, demonstrated a solution for purely AI-based driving on parking lots and highways. Lateral guidance of the vehicle is carried out completely by artificial neural networks. Using artificial intelligence, the system learns how to generate a model of the environment from camera data and steer the car. This approach requires neither highly precise localization nor highly accurate map data.

* The assistance systems/automated technologies described here are still under development and not available in production vehicles. The Audi AI traffic jam pilot is not yet available as an optional extra. Your Audi partner will be pleased to inform you about the exact date of introduction. In most countries, piloted/automated driving is currently not legally approved for use on public roads. Please also note in general that assistance systems can only support the driver within the specified system limits. Responsibility for remaining sufficiently alert and performing driving tasks rests at all times with the driver.