XPeng rolls out AI-powered OS with 2K pure vision ADAS, CEO promises L4 autonomous driving by 2025 (2024)

XPeng Motors is rolling out cutting-edge AI technology that it hopes will transform the automotive industry from “electrification to smartification.” The Chinese EV automaker held an AI Day event earlier today, detailing the impressive capabilities of XOS 5.1.0, “AI Tianji,” which it hailed as “the industry’s first AI-powered in-car OS.”

It’s been about a month since XPeng Motors ($XPEV) began touting its latest operating system’s (OS) capabilities during the Beijing Auto Show. XOS 5.1.0 focuses on artificial intelligence (AI) driven technologies to power smart vehicles and accelerate their evolution.

During the auto show, XPeng showcased a number of AI-centric technologies in the works that will offer its drivers an EV that can deliver more human-like decision-making and safer driving experiences via neural network learning and end-to-end data iteration from both vehicle and cloud processing.

XPeng also introduced plans to integrate and deploy what it calls the “industry’s first mass-produced 2K pure visual neural network large model in vehicles.” That newsconfirmed previous rumorsthat XPeng was abandoning LiDAR sensors in favor of pure vision, similar to Tesla FSD.

That being said, XPeng says its new system enables autonomous driving with capabilities comparable to LiDAR but uses pure vision to create a granular 3D naked-eye experience. We learned more about this technology and then some following XPeng’s AI Day event. Here’s a rundown.

XPeng rolls out exciting advanced tech during AI Day

There is much to unpack from XPeng’s AI Day in Beijing, China, earlier today when company founder, chairman, and CEO He Xiaopeng spoke to a live audience.

To begin, Xiaopeng outlined three core characteristics of this new breed of smart EV, led by AI large models in its OS: Active learning, rapid growth, and personalized experiences.The core of this technology involves the “feeding and training” of AI large models with unlimited rules to develop and grow their understanding, perception, and decision-making abilities in complex driving scenarios.

XPeng says it has already developed the necessary core algorithms for perception, positioning, planning, and decision-making while establishing vehicle-end and cloud-end data processing capabilities. As such, XPeng’s new AI-powered OS will enable rapid algorithm iteration based on actual data and offer future EV drivers enhanced autonomous driving capabilities that are continuously improving via over-the-air OTA updates.

A considerable part of XPeng’s OS and ADAS technology rolling out is enabled by “XNet,” the automaker’s new neural network. XNet is supported by XPeng’s planning and control large model, “XPlanner,” and a large language model called “XBrain.” Combined, XPeng says the integrated system enhances intelligent driving capabilities by 2x.

The XNet deep visual neural network used the industry’s first mass-produced pure vision 2K occupancy network to safely operate a given XPeng EV’s autonomous driving system with capabilities comparable to LiDAR. This new 2K pure vision network reconstructs the world with over 2 million grids to create 3D representations of drivable spaces and recognize every detail of static obstacles along a vehicle’s path.

XPeng shared that the perception range of pure vision 2K twice as long, covering an area equivalent to 1.8 football fields. Furthermore, the ADAS system can identify over 50 types of objects. With the autonomous driving system in place using AI and neural networks, XPeng has empowered the OS to learn and improve on its own using “XPlanner” and “XBrain.” Per the automaker:

XPlanner, like a human cerebellum, continually trains with vast amounts of data, making driving strategies increasingly human-like. This reduces driving jerks by 50%, improper stops by 40%, and passive human takeovers by 60%, providing an ‘experienced driver’-like smart driving experience that significantly enhances user comfort and safety.

With the introduction of the AI large language model architecture, XBrain, XPENG’s autonomous driving system gains human-like learning and comprehension capabilities, significantly improving its handling of complex and even unknown scenarios as well as its reasoning and understanding of real-world logics.

For example, with the support of XBrain, the autonomous driving system can recognize turn zones, tidal lanes, special lanes, and road sign text, as well as various behavioral commands such as stop and go, fast and slow. This enables it to make human-like driving decisions that balance safety and performance.

With the help of these new AI technologies, XPeng’s CEO told the crowd in China that the automaker will achieve Level 4 autonomous driving in the country by 2025. The automaker is also testing the end-to-end capabilities of its XNGP driver assistance technology globally.

Once we get the full English translation of XPeng’s AI Day event, we will be sure to add the video below. There’s much more to unpack, including the automaker’s AI assistant, Xiao P, Smart co*ckpit capabilities, and safety features like AI Valet Driver, AI Parking, and AI Bodyguard.XOS 5.1.0, AI Tianji, is available to all eligible XPeng EV drivers beginning today.