As we celebrate the end of a challenging year and look ahead to what 2022 holds, many of us are thankful for the good health we maintained while many of our friends and family members were not as fortunate. The pandemic has imposed many daily challenges and created a sense of urgency that we had never witnessed before. This was driven by the need to rebuild our economies and regain some sense of normalcy and has inspired the need to bridge the vast digital divide between advanced geospatial professionals and those who require these solutions but are less technologically savvy.
Regardless of where people fall on that continuum, the need to work and communicate virtually has permeated all sectors of global society. “Zoom” became a common household term for all ages and abilities, while that platform and others like Microsoft Teams has provided vital corporate lifelines around the world. Curbside grocery pickup and virtual doctor visits also became the norm, while fitness centers have been replaced with remote Zumba classes and home gyms.
These digital demands presented great opportunities for the advancement of cloud-based data sharing and processing, which directly and indirectly resulted in improved 3D processing and modeling. Despite the pandemic changing how we conduct business, geospatial sensor manufacturers and technologies continued their upward trend, although more modestly than expected. In the following sections, I will revisit my predictions in geospatial trends made at this time last year, while looking ahead to what’s expected in the industry this year.
Virtual Collaboration Rooms and Mixed Reality
Although developments in virtual and augmented reality started long before 2021, the lowered costs of data processing, computing power, algorithms, data compression and the cloud processing platform have advanced these technologies from video games to engineering and environmental solutions with direct societal benefits.
The release of Microsoft Mesh, a software platform for virtual collaboration, has taken VR and AR to a new level. Within Mesh, team members who are continents apart can engage via eye contact, facial expressions and gestures as 3D avatars in a shared space, collectively discussing and implementing modifications to a 3D replica of their project or design. Team members can participate in a Mesh session using Microsoft HoloLens 2, VR headsets, mobile phones, tablets or PCs via a Mesh-enabled app. HoloLens 2 is a self-contained holographic computer that enables hands-free and heads-up interaction with digital models via eye goggles.
These accessible platforms generate the need for 3D data while providing a new means of data modeling and interpretation. The geospatial industry reaps the technological fruits of these platforms when these applications involve 3D data modeling for engineering and environmental projects. Bentley, for example, has released its mixed reality platform, SYNCHRO XR. SYNCHRO XR is an app designed for visualizing 4D (3D data plus time) construction digital twins that also uses HoloLens 2.
A similar approach was introduced this year by NVIDIA with its NVIDIA Omniverse platform, which is described as “a scalable, multi-GPU, real-time reference development platform for 3D simulation and design collaboration, based on Pixar's Universal Scene Description and NVIDIA RTX technology.” Omniverse enables a variety of client applications, renderers and microservices to share and modify representations of virtual worlds.
Deep into Miniaturized Sensors
Apple achieved a great milestone in 2020 with the introduction of a lidar sensor on its iPhone 12 Pro, and it continued that capability on the iPhone 13 Pro. As a result, we have witnessed more surveyors and field technicians using smartphone-based lidar. A technician who is repairing a faulty circuit in an underground manhole can use this easy lidar tool to document and create an above-ground or underground 3D model. There were many articles published last year that summarized applications and data accuracy verification of lidar point cloud from iPhone 12 and iPhone 13, providing an expanding and accessible way for people to benefit from geospatial technologies.
Miniature lidar has been used on automated vehicles for a few years, and similar technology has been adopted for advanced home cleaning robots like the Samsung Jet Bot AI+ Robot Vacuum Cleaner. The cleaning robot is equipped with a lidar system and SLAM navigation hardware to scan and map rooms in a building, guided by geofencing to address defined cleaning needs. The manufacturers of these vacuums are not using the laser for range finding and obstacle avoidance; the laser is part of a true mapping system.
In addition to Samsung, Dreametech, Narwal and iRobot have followed a similar design and added lidar to their products. These developments should be welcomed by our industry as they illustrate how miniaturized lidar systems can be used, with slight modification, onboard mapping drones. These expanding applications will propel more advances in lidar manufacturing and will help bring down the cost of lidar systems.
BIM and GIS, the Foundation for Digital Twin and Metaverse
2021 brought a noticeable focus on the digital twin, with the industry learning more about its value to society. We also started hearing the term “metaverse,” which is synonymous, or at least close, to digital twin. Often misunderstood and confused with a 3D model, a digital twin is much more complex. It encompasses multiple concepts—such as scan-to-BIM, 3D modeling and GIS—to produce the desired environment.
Over the course of the year, the industry has learned that a digital twin:
- Is a dynamic, up-to-date replica or representation of a physical object, an asset or system with a complete collection of data in one place
- Evolves with the flow of real-time input from sensors and other sources
- Is not a static 3D model or simulation; it continues to evolve with added data and information
This advancement is important because, without a full understanding of the digital twin and its value, the term would remain an abstract buzzword. The tri, digital twin, BIM and smart city concepts all share the overarching goal of managing data and information as a smart and interactive means of improving data accessibility for better decision-making and asset management.
The digital twin accelerates asset operational readiness as it transforms an asset’s life cycle with the use of maintenance and performance data. Figure 1 illustrates the stages in an asset’s life cycle when viewed through the digital twin lens. In that figure, design details do not end up as a static archive in someone’s office but evolve into a living link to inform the asset owner or the operation manager to produce informed decisions.
Experts believe that about 70% of the return on investment in an asset design ecosystem is realized through asset operation and management. Today’s asset owner wants digital data at the outset, so project designs and models do not end up trapped in files (i.e., PDFs, spreadsheets, etc.); they live in dynamic objects that are easily accessed and managed. Without the digital twin approach, analog, unclassified and disconnected data present management challenges. This connection between the digital and physical worlds offers enhanced life cycles, informed decision-making and predictive capabilities.
Although our knowledge of these innovative concepts and their applications grew at a healthy pace in 2021, there still needs to be more outreach to educate clients and the industry about their collective benefits. Fortunately, a growing number of agencies and property owners understand the ROI and are willing to invest in building a digital twin for projects or assets. Architecture, engineering and geospatial (AEG) firms have a great opportunity to serve their clients by collecting, modeling, and managing data and building an integrated digital twin/BIM environment. This provides for better access to data and information to support efficient and effective asset management. The digital twin also opens huge opportunities for the geospatial community since it relies on services like scan-to-BIM, lidar scanning, GIS and other mapping services.
High-Definition Maps for Autonomous Driving
The concepts of smart cities and intelligent transportation systems continued to make progress in 2021. In particular, developments in autonomous driving made strides as an increasing number of car manufacturers worked toward putting their autonomous vehicles on the road in the next five to 10 years.
While connected and autonomous vehicles are equipped with a variety of sensors and artificial intelligence algorithms to help navigate the road, they also require high-quality maps to see beyond the vision of the onboard sensors and to assist during inclement weather. These maps are not like the navigation maps you see on a smartphone or a standard car navigation system; they are high-definition maps.
High-definition maps provide the car control center, i.e. the brain, with extensive, precise information. This includes the lane number the car is in, freeway exit lanes, pedestrian crosswalks, bridges, overpasses, tunnels, locations of traffic control devices, smooth 3D trajectories for road edges and boundaries that are accurate to the centimeter level, meter-by-meter road grade and road superelevation, among other details.
Most self-driving vehicle manufacturers are planning to develop these maps for their vehicles. The exception is Tesla, which ascribes to autonomous navigation using sensors alone. Opponents of this sensors-only approach cite the following advantages to utilizing high-definition maps, saying they:
- Have a longer range than sensors, with views for miles that wrap around corners and curves
- Help the car drive smoothly around curves versus the jumpy updates of live sensors
- Provide more accurate positions than sensors
- Enable continual updates and improvements; sensors are static unless you get a new car
- Can be used for car geofencing
- Are not impacted by weather conditions like sensors
- Can be used to check and verify the health of sensors, providing an extra layer of safety
- Can correlate traffic lights to lane assignments
However, although these are proven benefits, high-definition maps for the global roads network do not yet exist for autonomous vehicles. Tremendous work and global cooperation are needed to remap the roads network to this level of detail. Some car manufacturers are attempting to build these maps themselves but are finding the job daunting. Additionally, these car manufacturers are producing maps according to their own specifications, which not only leads to a duplication of efforts but also creates an inherent and troubling lack of standardization.
The geospatial industry has an immense opportunity to work with national and global transportation agencies to put forward a set of standards to govern the production of high-definition maps to support autonomous vehicles. The industry can not only ensure the maps are created appropriately but can earn a substantial market share of the services and technologies needed to produce high-definition maps for a global roads network.
Drones, Drones Everywhere
While unmanned aircraft systems have been used for engineering and geospatial mapping applications for a few years, last year finally saw industry standardization. On the regulatory side, the Federal Aviation Administration issued positive modifications to Part 107 guidelines regarding flying restrictions over people and flying at night. This standardization has helped drones become integral to professional aerial data acquisition operations. Improved business practices and advancing technologies have helped drones prove that, when it comes to small projects, there are no other acquisition platforms that can compete with their capability and affordability.
The other strong dynamic that 2021 brought to drone operations was the increase in UAS-based lidar. This healthy growth was helped by the release of DJI lidar system, Zenmuse L1. Zenmuse L1 integrates a Livox Lidar module, a high-accuracy inertial measurement unit, and a camera with a 1-inch CMOS on a 3-axis stabilized gimbal. Zenmuse L1 is offered with DJI newest drone, the Matrice 300 RTK, and DJI Terra. DJI Terra is DJI’s suite of software for mission planning, data capture, and data processing and analysis. The differentiator for the DJI lidar solution is the lower cost as compared with other market offerings, which attracted the attention of small businesses and solo operator companies.
It is worth mentioning here that while the Zenmuse L1 provides a reasonable solution for many applications, it will not be suitable for applications that require the highest accuracy. Zenmuse L1 is marketed with 5-centimeter vertical accuracy and 10cm horizontal accuracy (as RMS). While 5cm vertical accuracy is suitable for many applications, it will not meet standards for road design as required by departments of transportation. I expect this growing trend in collecting lidar data via drones will continue through 2022 and in the years to come.
From Coastal and Deep into the Sea
As I predicted last year, bathymetric survey and data collection made industry news in 2021. That is partly due to the direction the global economy is taking to support and protect the blue economy. Multiple aspects of our life—from fisheries, aquaculture, maritime transport, coastal and marine and maritime tourism to the emerging utilization of marine spaces to harness coastal wind energy—contribute to the evolving blue economy and require updated approaches and technologies.
Here in the United States, deep ocean survey and advanced techniques for coastal and deep ocean mapping are in high demand, propelled by two high-profile initiatives of the National Oceanic and Atmospheric Administration (NOAA): National Strategy for Ocean Mapping, Exploring and Characterizing of the United States Exclusive Economic Zone (also known as the NOMEC Strategy) and Alaska Coastal Mapping.
On the coastal and inland water mapping front, mainstream lidar manufacturers continued to make innovations and enhancements for topographic and bathymetric systems. Recently we witnessed some creative developments in bathymetric lidar initiated by industry members outside traditional manufacturers. An example of this was the recent introduction of the first high-altitude topo-bathy lidar system, based on Geiger-mode technology. The Bathymetric Unmanned Littoral LiDar for Operational GEOINT (BULLDOG) system was developed by the Woolpert Maritime Research Lab for the Joint Airborne Lidar Bathymetry Technical Center of Expertise (JALBTCX).
Additionally, deep ocean hydrographic surveys witnessed good progress in the field of uncrewed deep water mapping technologies. Companies like SailDrones and other manufacturers are expected to continue to innovate to meet demand.
Whirl Around the Coastal Regions
Speaking of coastal wind energy, with the current U.S. administration backing renewable resources, that industry is expected to flourish in the coming years. Last year, the nation’s first large-scale offshore wind farm was approved about 12 nautical miles off the coast of Martha’s Vineyard, Mass. More projects are either already approved or in their final stages of approval. This development found a soft-landing spot within the geospatial industry since the planning, design and construction of wind farms requires geospatial and surveying services, including bathymetric lidar survey.
Data Democratization: Big Data Needs Big Tools
In my opinion, the field of data democratization grew sluggishly in 2021 as businesses struggled to survive a challenging year. We did not see clear signs from the mainstream geospatial industry on the growth of creative methods and tools needed to mine, extract and convert geospatial data into knowledge, and therefore we are still limited in our development of tools that can gather information from lidar point clouds and imagery using automated methods. Despite that, the industry continued its interest in data mining and machine learning as it relates to application development.
Outside the geospatial industry, crowdsourcing, big data and data science continued to grow. This was largely fueled by a hot market for tech companies selling location-based and social tracing data to the highest bidder, whether for business purposes, marketing or otherwise. The diverse sectors utilizing these location-based technologies—including planning, construction, utilities, transportation, government and energy—will continue their growth and potential applications in 2022 and the years to come.
AI and Deep Learning are Living in the Cloud
In my previous articles on annual geospatial trends, I acknowledged that progress was made by agencies moving computing powers to the cloud. More companies are offering business models for cloud data hosting and processing. However, Amazon, Google and Microsoft continue to lead that market and offer users and developers sophisticated platforms like serverless cloud computing.
As I mentioned before, these platforms enable developers to run apps and services without having to manage and operate costly and complicated server infrastructure. One interesting development is the increased use and attention to the Microsoft platform Azure. Azure is a cloud platform that contains more than 200 products and cloud services designed to enable businesses to build, run and manage applications across multiple clouds, on-premises and at the edge, with the tools and frameworks of their choice.
One very positive characteristic of Azure is that it is not limited to Microsoft software and applications. It supports off-the-shelf and open-source technologies, so a user can employ the tools and technologies that suit their needs. Azure supports the virtual work environment, enabling businesses to run any centrally hosted applications using their data source and operating system and device. Because it is virtual, the Azure environment eliminates the need to set up a workstation with company applications and software for every employee individually.
As for geospatial application-based AI within the industry, on the commercial software level, Esri continues to push the envelope to help users streamline workflows with AI tools within ArcGIS. For example, in 2021, Esri added three ready-to-use geospatial AI models in the ArcGIS Living Atlas of the World. The first two models, which use satellite imagery, extract building footprints and perform land-cover classification. The third model classifies point clouds. The interesting thing about these models and algorithms, besides their ability to deal with huge amounts of data, is that they are self-trained and require no additional data training by the user.
On the AI-based services side, only modest activities were observed this year. Companies like Airwork offered a cloud-based solution for extracting information from imagery and lidar point clouds. Although this Airwork offering has been welcomed by the surveying and mapping industry, massive work still needs to be done with AI, machine learning and deep learning to marry automation with GIS and geospatial applications to support mining the tremendous amount of data today’s sensors acquire. In my opinion, without federal and public funding to entice creativity in this field, GIS and the geospatial applications will see slow adaptations of AI, ML and DL capabilities.
Lidar for Everyone
During 2021, lidar use and technologies continued to advance. But, aside from the bathymetric development noted earlier, there was not the big breakthrough from a major lidar system manufacturer I had hoped for this year in the development of AI-based applications. This development would help system users derive information from the massive and growing number of point clouds with minimal manual processing. However, this lack of investment may have been a casualty of the pandemic and its impact on the economy.
The miniature lidar system was the segment that advanced the most in 2021, driven by car manufacturers, gadget manufacturers like smartphones and drone use. Most upscale models of cars are now equipped with lidar sensors, which are important for autonomous vehicles. This is leading car manufacturers to invest heavily in miniaturized lidar by acquiring manufacturing companies that produce small lidar or indirectly supporting its development.
Our industry reaps the benefits from this investment, as Velodyne is one of the biggest providers of miniature lidar for the automobile companies. Velodyne revealed during Auto Guangzhou 2021, held in China in November, that its latest innovations support advanced driver assistance systems (ADAS), autonomous vehicles, robotics, smart city infrastructure, delivery, industrial applications and more. Fortunately, Velodyne has not yet been acquired by a car manufacturer and still offers affordable lidar systems for our drones.
Shadow of the Pandemic
This is my fifth annual trends article, which began as a blog on the Woolpert website in 2018. In the last couple of years, the pandemic has adversely affected many segments of our industry while providing some relief to others. This has made it more difficult to accurately predict where our industry and technology are heading. However, one thing has become even clearer during this difficult time: Our success as an industry requires the collaboration of multiple tiers of government, the private sector, public utilities, community activists, building owners, average citizens, etc., to truly advance.
We will get through this, and we will be stronger for it individually and together. Happy New Year!
About the Author: Woolpert Vice President and Chief Scientist Qassim Abdullah, Ph.D., PLS, CP, has more than 40 years of combined industrial, R&D and academic experience in analytical photogrammetry, digital remote sensing, and civil and surveying engineering. When he’s not presenting at geospatial conferences around the world, Qassim teaches photogrammetry and remote sensing courses at the University of Maryland and Penn State, authors a monthly column for the ASPRS journal PE&RS, and mentors R&D activities within Woolpert.
This article is running on Geo Week News and both PE&RS and Lidar Magazine.
