To develop fully autonomous passenger vehicles, it is clear that a combination of technologies are needed, including the underlying maps that are used for localization and safety. By providing autonomous driving systems with better information about roads and their surroundings, maps can provide key information for AI to use to make advanced and safe driving decisions.
At the GTC keynote, NVIDIA has announced a new mapping platform that can be used to get that enhanced location information to autonomous driving platforms. NVIDIA Drive Map is a multimodal mapping platform designed to enable the highest level of autonomy while improving safety.
DRIVE combines the accuracy of DeepMap survey mapping (the mapping company that NVIDIA acquired in 2021) with AI-based “crowdsourced” mapping.
Hybrid Data Collection Methods
To collect the map and localization data, the system utilizes cameras, lidar and radar. By overlapping these systems; the DRIVE Map system can incorporate a level of redundancy and versatility required to get the best data possible in a wide range of scenarios that could be encountered while driving.
In addition, these multiple modes of mapping/capture can be used as separate data layers, accessible by AI ‘drivers’ to localize independently, which is crucial for the safety requirements of the highest levels of autonomy.
The camera localization layer consists of map attributes such as lane dividers, road markings, road boundaries, traffic lights, signs and poles.
The radar localization layer is an aggregate point cloud of radar returns. It’s particularly useful in poor lighting conditions, which are challenging for cameras, and in poor weather conditions, which are challenging for cameras and lidars. Radar localization is also useful in suburban areas where typical map attributes aren’t available, enabling the AI driver to localize based on surrounding objects that generate a radar return.
The lidar voxel layer provides the most precise and reliable representation of the environment. It builds a 3D representation of the world at 5-centimeter resolution — accuracy impossible to achieve with camera and radar at present.
Once localized to the map, the AI can use the detailed semantic information provided by the map to plan ahead and safely perform driving decisions.
Rather than choosing to completely crowd-source these maps or a dedicated survey-grade mapping effort, the DRIVE Map will be constructed through a combination of both. This approach combines the best of both worlds, achieving centimeter-level accuracy with dedicated survey vehicles (from the DeepMap survey map engine), as well as the freshness and scale that can only be achieved with millions of passenger vehicles continuously updating and expanding the map (via the AI-based crowdsourced engine). The collected map data will be aggregated and loaded in to the NVIDIA Omniverse and used to update the map, providing the real-world fleet fresh over-the-air map updates within hours.
Leveraging Omniverse for Maps and Digital Twins
NVIDIA also hopes to use DRIVE Map to accelerate autonomous vehicle deployment, by generating ground-truth training data for deep neural network training, as well as for testing and validation. These workflows are centered on Omniverse, where the real-world map data is loaded and stored. Omniverse will maintain “an Earth-scale representation of the digital twin” that is continuously updated and expanded by survey map vehicles and millions of passenger vehicles.
Using automated content generation tools built on Omniverse, the detailed map can be converted into a drivable simulation environment that can be used with NVIDIA DRIVE Sim. Features such as road elevation, road markings, islands, traffic signals, signs and vertical posts are accurately replicated at centimeter-level accuracy.
With simulations that incorporate real-world physics, AV developers can use the simulated environment to generate training scenarios that aren’t available in real data.
Ultimately, the digital twin could provide fleet operators a complete virtual view of where the vehicles are driving in the world, assisting remote operation when needed. As a scalable platform, DRIVE Map equips the AI driver with the understanding of the world needed to continuously advance autonomous capabilities.
