In January of 2014, tech giant Apple filed a patent for a 3D-scanning sensor. The application, which was published this past Wednesday, was called “3D depth point cloud from timing flight of 2D scanned light beamed pulses.” They might as well have just called the thing “laser scanning.”
The gist of the patent is that Apple has developed their own LiDAR sensor to address the weakness points of currently available LiDAR solutions, including “excessive power consumption, limited x-y resolution, limited depth resolution or accuracy, limited frame rate, and long product development cycles.”
One possible application of an Apple LiDAR is the much-rumored “Apple Car.” Another possible application comes from the patent itself: a “host device like a consumer electronics product” (iPhone, iPad, or maybe even a dedicated scanner?).
The Apple Car probably won’t look like this.
Considering that their LiDAR sensor has to be different from other LiDAR sensors on the market in order to receive a patent, what makes it special? I’m not 100% sure, since patents feel like they’re written in a form of English I’ve never seen before, but here’s what this reporter has been able to gather by poring over patent #9,285,477.
It Will Take 3D Video
The first sentence of the patent summary lays this one out pretty clearly. “An embodiment of the invention is an optical remote sensing system that can generate a digital, time-varying 3D point cloud…” [emphasis mine]
The sensor can be used to take measurements of static 3D objects or environments, but that’s unlikely to be the point. A point cloud that varies in time gives us something more exciting: A 3D video. This makes the sensor useful for applications like machine-vision for autonomous cars as well as slightly less intensive applications like consumer 3D movies.
It Has a Framed Field of View
Google’s tango handheld consumer scanner.
The sensor appears to capture a rectangular field of view instead a full 360-degree circle the way many traditional LiDAR sensors do. Good points of comparison for Apple’s LiDAR seem to be Google’s Tango and DotProduct’s DPI-8, handheld 3D scanning devices that capture only what you point them at.
Scanning with Apple’s LiDAR means capturing a sequence of “frames,” or 3D point clouds that cover the entire field of view. String these 3D frames together and you’ve got a 3D video, much like stringing 2D photographs together gives you a 2D video. According to the patent, the sensor will capture a frame rate of “between ten to sixty frames/sec.”
It is Not Flash LiDAR
Up until now, the sensor sounds a lot like flash LiDAR, which also captures frames of 3D data quickly enough to create 3D videos. But there’s a difference.
Flash LiDAR uses complex silicon architecture to emit a laser and steer it to different parts of the scanner’s field of view. It is built on microchip and uses no moving parts.
Apple’s LiDAR, on the other hand, uses moving parts much like the LiDAR sensors you’re used to. It includes “a mirror that is motorized to pivot, rotate or tilt so as to quickly and accurately sweep the outgoing pulses that are reflected by it, so that the light beam pulses sweep the entirety of a desired 2D space.”
See the mirror, item #6, in the diagram below.
Let’s have another look: Apple’s LiDAR diagram with mirrors highlighted
It is an Unexpected Approach to Consumer LiDAR
I was most fascinated by Apple’s plan to use a fixed mirror (item #8 in the diagram above) that allows them to extend the LiDAR’s field of view when quarters are tight… like when the sensor is packaged “within a consumer electronics product.”
To my knowledge, this is the first time a company has changed a classic LiDAR setup with the intention of including it in a consumer electronics form factor. Up until today, I thought that the only sensors small enough to fit were classic RGB cameras, infrared sensors, and solid-state LiDAR.
This patent doesn’t mean that Apple has no plans to use solid-state LiDAR. The patent application in question was filed a full two years ago–it’s entirely possible that Apple has explored other avenues of 3D scanning since then. Either way, now that Apple has joined Google in developing its own LiDAR, it seems like its only a matter of time before we have real, powerful 3D-scanning sensors in our consumer devices.
