SiteSee has made a name for itself by using machine learning to perform accurate object recognition against 3D reality models. The company sends up UAVs to capture a reality model of your large infrastructure asset, and then automatically extracts virtually all the information that an owner/operator could need to make informed decisions about the maintenance of the structures—from current condition to future operational parameters.
Recently, SPAR3D caught up with CEO Lucio Piccoli to discuss SiteSee’s capabilities and technology. We had a wide-ranging conversation that touched on the future of machine learning, the limiting factors for the development of object recognition (hint: it’s not what you think), and the future of the 3D capture industry in the age of cheap, cheap commodity sensors.
Antenna head being processed with Artificial Intelligence.
What SiteSee can do
The first thing you’ll notice when looking at SiteSee’s software is that it offers the basic features you would expect from a 3D viewer—viewing on any internet-connected device, access control, on-the-fly measurements, and an array of visualization tools.
But you’re coming to SiteSee for the object recognition, right? Though less than a few years old, the solution does not disappoint. Feed it 3D models of power transmission lines, it will automatically identify insulators and compare them against the design model to determine deviations and perform automated corrosion assessments. If you feed it models of telco towers, it will tell you antenna make, model, height, and more. It can analyze the current condition of the tower and identify damages and inconsistencies. It can learn to recognize most of what you need to know about your structure’s current state.
This kind of data is useful in itself, but it also enables what SiteSee to predict the future with a reasonable degree of certainty. The software uses the data to track changes over time, and given a baseline for your structure it can use predictive analysis to broadcast future failures.
For those of you not in power transmission or telco, Piccoli offers good news: SiteSee has plans to expand.
Object recognition is already better than humans, and it keeps improving
When I ask about the accuracy of the object recognition, Piccoli says “we’ve just done some towers for a US client and we’re getting 94% accuracy on detecting objects, 98% with another. With some we’re doing in Australia, we’re getting 100%.”
Piccoli surprises me by explaining that the specific accuracy numbers aren’t usually important to his customers. That’s because the solution is already significantly “better than what humans are doing—humans miss far too many things. It’s difficult when you have pictures, and even though some humans are using 3D reality models, they still miss too much stuff. Now we highlight it all and they only have to worry about this little thing the system missed, or if it’s actually correct.”
It may disappoint you to know that you’ll have to look at the model at all, but the good news is that SiteSee has a solution for that too. The company is turning this human review process into a mechanism for bringing the object recognition closer to 100% accuracy.
“One of the big features we’re adding,” Piccoli says, “is the ability for experts with our clients to manually annotate what we miss, what we incorrectly classify. Then we do a continuous training loop.” In other words, SiteSee is exploiting the basic principle of machine learning, that computers can learn to perform specific tasks by being fed right kind of data. Feed a machine-learning algorithm an extremely large set of 3D models with tennis shoes in them, and it will learn to identify tennis shoes pretty well. Offer the algorithm human input that tells it when when it’s right, and when it’s wrong, and will improve even further.
“We know we’re going to get it close to 100% with the continuous learning capability,” Piccoli says.
SiteSee measurement accuracy analysis.
Eliminating humans
Human input is the key to improving machine learning, but humans are also the biggest factor holding it back. “When we started this 18 months ago,” Piccoli says, “education about 3D modeling was the biggest issue we had. That’s all changed. What we’re facing right now is issues about the quality of capture. The hardware, the sensors, they’re fantastic and they give us such great results, but we are doing automated capture with humans involved, and humans introduce too much variability, too many problems. That’s our number one.”
The solution, of course, is to get humans out of the loop. “We are working on a lot of improvements in automated capture to reduce this variability. We’re working with two autonomous flight-plan software providers to get the drone to do it exactly as we want, every time, without fail. No humans involved. That’s what we want.”
A happy future of commodity 3D
Piccoli admits that the company got into 3D capture as a means to an end, to get the data they needed to feed their algorithms. However, he’s happy that 3D data is getting exponentially cheaper, and even becoming a commodity.
“We would be really, really happy for it to become such a commodity that we don’t have to do it,” he says. “We don’t care where it comes from, it’s not our business. It’s a painful, painful business. Anyone who thinks they’re going to make money out of building reality models? That’s gone, that opportunity moved about two years ago. We simply want to consume more.”
He argues that 3D reality modeling is incredibly clever, but the real value is getting software to go on top of that and bring value. That’s why he hopes that 2023 will find SiteSee looking more like the Google of infrastructure—a repository for the world’s 3D infrastructure models, and a suite of tools that chew up this data, spitting out every useful piece of information a human could possibly need.
For more information, see SiteSee’s website here.
