Early this month, prior to co-founder and CEO Jensen Huang’s keynote address in which he discussed the present and future of generative AI and OpenUSD, NVIDIA announced a new extension of their existing partnership with lidar sensor manufacturer Hesai Technology. The new collaboration between the two companies will integrate the latter’s lidar sensors within the former’s Drive and Omniverse ecosystems. According to the announcement, the combination of Hesai’s sensors and NVIDIA’s AI, simulation, and software development platforms “will unlock new possibilities for the autonomous driving industry.”
Getting into more specifics, Hesai will be integrating its AT128 lidar sensor into the NVIDIA platforms. The AT128 is the company’s long-range lidar for advanced driving Assistance Systems (ADAS), and it will be integrated into both NVIDIA DriveWorks and NVIDIA Drive Sim.
DriveWorks is a software development kit (SDK) from NVIDIA aimed at assisting with autonomous vehicle software development. A modular, open, and customizable platform, it allows developers to innovate their own solutions without having to spend the time to develop the basic functionalities. This can include incorporating just a single module or multiple, depending on the needs of the developer. With this latest collaboration between NVIDIA and Hesai, the latter’s lidar sensors are integrated into this workflow at the developer level to create more efficient and reliable autonomous driving systems.
After developers create their autonomous driving systems in DriveWorks, that software can be tested within the Omniverse with Drive Sim. Drive Sim provides a physically accurate simulation environment for this testing to take place at scale. The platform’s Neural Reconstruction Engine (NRE) uses AI to bring real-world data directly from data collection in the real world into a digital simulation environment, where developers can also modify different scenarios, add objects to the environment, and apply randomizations for better testing. Hesai is the latest lidar sensor to be added to the platform, joining RoboSense, which was added earlier this year.
Although the development of autonomous vehicles hasn’t quite come at the pace some may have expected a decade ago, that progression is still coming, to the point where we are starting to see driverless taxi services working 24/7 in some parts of the United States. That said, there is also plenty of public pushback coming against the early versions of these systems that are still working on kinks and need further testing. That testing is, of course, necessary to know the vehicles are safe for mass production, but it can also be dangerous for drivers, cyclists, and pedestrians when tested in the real world.
Drive Sim works to at least mitigate some of these concerns by providing a hyper-realistic digital environment to complete some of this testing before vehicles hit the road, and partnerships with companies like Hesai help boost this testing by using the types of lidar systems that are used in the vehicles.
"By combining our expertise in lidar technology with NVIDIA's world-class simulation and software development platforms, we can provide developers with invaluable insights and resources for unlocking the full potential of lidar in autonomous driving applications. This announcement marks a significant step forward in advancing the autonomous driving industry," said Bob in den Bosch, SVP of Global Sales at Hesai Technology, in a press statement.
"This collaboration will provide autonomous vehicle developers with seamless tools and flexibility along the AV pipeline," Glenn Schuster, Senior Director of Sensor Ecosystems at NVIDIA, added. "Together, we aim to push the boundaries of lidar technology and accelerate the deployment of safe and efficient autonomous driving."
