Esri has released a new web application for users that want to integrate deep learning into their imagery workflows. Deep Learning Studio, available with the release of ArcGIS Enterprise 11, offers a collaborative environment where multiple users can work together on a image-based project that includes deep learning. With the app, multiple users can work on a single project and perform deep learning tasks, such as collecting training samples, train deep learning models and run inferencing at scale.
The app combines multiple things at once: a frontend experience to deep learning tasks that are part of backend raster analytics, a collaborative environment that divides otherwise tedious deep learning tasks over multiple users and a complete end-to-end deep learning workflow, that is offered through a user-friendly project wizard (figure 3). Additionally, users can customize their own workflow if necessary.
Although the app requires no local software installations, it requires both ArcGIS Enterprise and ArcGIS Image Server, as these provide the data and analytics tools that are accessed through the app. ArcGIS Enterprise is Esri’s full-featured mapping and analytics platform which is available on-premises, in the cloud or a mixture of both. ArcGIS Image Server is part of ArcGIS Enterprise, and provides a distributed computing and storage system that powers the analytical processing and serving of large collections of imagery, elevation data, rasters, and other remotely sensed data.
How the app works
The app is available through the App Launcher in the ArcGIS Enterprise portal website. After starting the app, users can create a new project, name it and select which type of task that you want this deep learning model to perform. This can be either object detection, pixel classification or object classification. There are two steps in creating the model: preparing the data and creating the model. After this, you are able to use it on other imagery data, which is called running inferencing. Esri published a video that describes how the app works in detail.
Data preparation and model creating workflows
The data preparation process is a four-step workflow. First, you select an imagery layer from a configured raster data store. Second, you define the labels you want to collect, which is the process of augmenting the data with informative tags. Third, you invite other members of your organization to label the data and define geographical work areas for each of these members so that an imagery set is split into multiple areas where each member gets a part of. All these steps use a simple project wizard where you can simply click on a set of buttons indicating the different options for a new project.
After finishing the data preparation workflow, you can start collecting training samples, which uses digitizing tools to collect representative training samples of the objects of interest. The labels are created manually in the imagery data. Because this process is rather tedious and repetitive, it makes sense to use this app to split up a dataset into multiple areas and divide the work over multiple users. After labeling, the app provides a dashboard to the project manager where he or she can review project samples and either approve or reject these. After the labeling process is completed, the training data is exported, after which the deep learning model can be trained.
Training and running a deep learning model at scale
First, you select the training samples (called image chips) and specify the model type you want to training. The model train parameters are defined in the app documentation. Next, start the training process is started, which takes a few minutes. The resulting deep learning model can be published to an ArcGIS Portal as a deep learning package, which includes the trained model along with a model definition file. This model is now ready to be used on other imagery data at scale (using the running inferencing tool inside the app).
