Spend even just a few minutes reading about 3D capture and you’re likely to see a lot of very heated talk about accuracy. That’s because the accuracy specification of a scanner is among the most common concerns for buyers on the market, whether they are a newcomer or an old pro. It’s also one of the most commonly misunderstood parts of 3D capture.
Amir Rubin with Paracosm’s PX-80
Paracosm president Amir Rubin is no stranger to this phenomenon, having spent years talking to potential buyers about the accuracy of his company’s handheld SLAM-based lidar scanner, the PX-80. “I think people are trained by industry literature,” he says. “It’s the first question that everyone is trained to ask, What’s the accuracy? But then if you counter with, what do you mean by accuracy? Then, you realize people don’t know what they’re asking for.”
Rubin explains that we often view accuracy as a simple number (like +/-3 cm) that can tell us how “good” a 3D scanner is. However, he notes, “the reality is that there’s not even a really well agreed upon standard or notion of what accuracy is for any kind of laser scanning system.”
When you start to examine how accuracy is measured and consider all the ways that field work quickly complicates the idea, his point becomes difficult to argue.
The problem of standardization
When we think about the accuracy specs for tripod-based scanners, we tend to view them as an indication of relative accuracy. To put another way, most of us read a 2 cm accuracy spec as telling us that each individual laser measurement performed by the scanner will return a number within 2 cm of the truth. The benefit of this way of thinking about accuracy is clear: These numbers allow you to make easy comparisons between one scanner and another.
It all starts to break down, Rubin says, when you realize that vendors don’t all arrive at these numbers the same way. Are they calculating accuracy by taking multiple scans and averaging the results? Where do they measure the zero point of the scanner? Are they accounting for the reflectance of the surfaces they are scanning? That last one makes an especially huge difference, because you can perform a scan with a system that has very high accuracy specs and get very different results depending on whether the surface is reflective or not.
PX-80 scan data captured in an industrial plant.
Accuracy in the field, where we work in 3D
Even if all of these accuracy specs were standardized across devices, Rubin argues, that wouldn’t “reflect the reality of scanning in the field.” Where the accuracy specs are measured in 2D, work in the field happens in 3D.
He offers a quick illustration to show how real-world scanning complicates things. Imagine you have a laser scan of the room you’re sitting in right now, and you want to use that data to measure the distance between one wall and another wall. “What does that mean, exactly?” Rubin asks. “Does it mean that I measure between a point on one wall, and a point on the other wall, by clicking on actual points from the point cloud? Or should I fit a plane to the surface of each wall, almost like creating a BIM model over the laser scan, and then take a measurement between those two objects? By the very nature of modeling a wall, or fitting a plane to it, I’m doing a type of noise averaging off the raw point clouds themselves.”
This question—whether to measure off of the points themselves, or to model a plane over the points first and measure from there—complicates how you define the accuracy of a scan. Both methods derive a measurement from the same scan, but each one will return different results. So, how do you define the accuracy of the scan?
It only complicates things further when you consider everything else that goes into a real 3D scan in the field. If you’re in a complex environment that requires multiple tripod-based scans, you’ll need determine how tightly those different scans are registered together when you’re determining accuracy. Or if you’re scanning an entire building, Rubin asks, what if there’s a serious error in the middle, like a warped or noisy section? “If I measure point to point from one edge to another edge, across that section, it’s very quote unquote ‘accurate,’ but the middle of the scan is all messed up. How is that captured in the definition of a scanner’s accuracy?”
A simpler kind of accuracy measurement
Rubin argues that, despite all these complications, we can still define accuracy in a very simple way: “Does it capture scans that let me solve the problems, or answer the questions, I have about my environment or real-world space?”
If you’re scanning a construction site to check the progress against the BIM design model, you probably don’t need to worry if the scanner is accurate to the millimeter. Instead, you should learn whether it gathers data that tells you what you need to know. If you’re scanning a factory floor to move a piece of equipment, or to perform a renovation, you only need to determine if the data is going to make it possible for you to complete your task.
“If the answer is yes,” says Rubin, “then the scan has done its job and the scanner is plenty accurate.”
An overview shot of an industrial plant capture performed with a PX-80
Good enough accuracy means higher flexibility
This kind of accuracy can’t be easily described with a simple number, but thinking this way offers a lot of benefits. For one, it will help you to evaluate your needs better.
If it turns out that you do need the highest absolute accuracy on the market to perform your task, like when you’re measuring the flatness of a concrete floor, or recording the locations of every bolt in a room, you’ll need to get a tripod-based scanner to get the detail you need to complete your project. If you don’t need that kind of detail—and many projects don’t—you can get a device that trades accuracy for other benefits. A SLAM-based scanner like Paracosm’s PX-80, for instance, offers lower accuracy specs than a tripod-based scanner but also costs less, captures RGB imagery, enables you to proceed without ground control points or GPS, and helps you capture a large area over the course of a few hours.
“What we’re arguing,” Rubin says, “is that it doesn’t matter whether your tripod scanner has millimeter accuracy if it’s going to take you three weeks to capture the entire project. It would be much more valuable to have a scanner that lets you capture in one day or one afternoon if that scan lets you answer the really critical questions right away. For us, that’s the basic premise for using SLAM-based handheld scanning.”
Rubin explains that SLAM-based handheld systems like the PX-80 are ideal for those projects that are focused on the gathering more holistic, big-picture information. That means tasks like recording the locations and circumference of trees in a forest, capturing the landscape between two buildings to plan a walkway, or surveying a whole office building to capture every cubicle, pipe, conduit, and closet.
When you’re doing this kind of work, “you have to ask, what am I trying to achieve and do I really need “millimeter” accuracy for that task? Or would I be better served by something that really quickly captures the condition of this entire space with results that are good enough? That’s your accuracy benchmark—you shouldn’t judge it off a spreadsheet.”
To download sample PX-80 data, or to learn more about the device, click through to Paracosm’s demo site here.
