Today, CARTO’s cloud native platform is used in retail, telecom and for decision making based on environmental data such as large-scale climate datasets and satellite imagery. CARTO opted for creating a cloud native platform to be able to take full advantage of the cloud computing model, which does away with infrastructure, scale or connectivity issues of traditional IT.
CARTO’s cloud native platform realizes a connectivity between different data warehouses and databases, and includes a spatial analytics toolbox that brings spatial functions into the data warehouse to perform tasks such as geocoding, spatial data analysis or create map tiles. The spatial SQL language connects the different data warehouses offered by different cloud storage providers. However, interoperability between cloud data warehouses is not a given which causes challenges to work with data warehouses from different vendors.
To solve this problem, CARTO is doing some work towards open geospatial interoperability, so that CARTO users can eventually work with geospatial data from different data warehouses in the cloud. It also builds its stack on open source technology: CARTO’s visualization library for example is built on open source technology and manages deck.gl, a WebGL-powered visualization framework that provides a variety of easy-to-use 2D and 3D data visualizations with support for large data sets.
Under the OGC flag, CARTO is currently working on native support for geospatial data types between different data clouds. GeoParquet is the standard storage layer for geospatial data that will provide interoperability between different data warehouses, so users can connect and query from different data providers.
Geospatial interoperability will also be improved by using spatial indices, to improve query visualization and performance using CARTO’s own platform. Quadkey, Uber’s H3 and S2 are examples of different spatial indexing systems that refer to different geographic hierarchical indexes to partition the space on earth into discrete grids. The benefit of H3 is that it deals with projection and area, so the same shapes are preserved anywhere on earth. CARTO users benefit of spatial indices that allow for faster spatial joins than complete geometry joins, resulting in shorter query times at scale. Additionally, spatial indices makes it easier to join spatial data with third party data as they use a common geographic support system.
A recent CARTO webinar previewed an upcoming CARTO map feature: spatial indices using raster data. Traditionally, raster files are very large and have performance issues when displaying them and zooming in and out. Using a spatial index layer of raster data, the same information as a raster layer is presented to the user, but aggregated at different resolutions depending on the zoom level. In this particular demo (starting around 29:20), the highest resolution used has a total cell area of around .85 square km, with all aggregation is happening on the fly at the database level where only the tiles shown on the map are calculated. This functionality allows for massive data visualization with all of the benefits of vector data such as interactivity.
In the same webinar, another upcoming feature was shown: a no-code workflow called Workflows that is meant as an alternative to writing spatial SQL queries when performing data analysis. This new feature will look familiar to anyone that has worked with Safe Software’s FME or Alteryx, where a user can interact with an intuitive user interface instead of typing code.
Resources:
The Rise of Cloud Native Spatial Analysis - Big Data Analytics Summit Canada
