Research Challenges

To summarize, the goal of this project is to leverage image repositories, specifically high-resolution aerial imagery, and non-image repositories, specifically gazetteers, to improve automated image analysis and the improve fidelty and coverage of the non-image repositories.

Gazetteers, such as the USGS Geographic Names Information System (GNIS), serve as a weakly-supervised source of traning data. They index millions of instances of a range of geographic features. We focus on spatially compact and visually distinct geospatial objects such as schools and use the GNIS API as well as the USGS National Map to identify and download high-resolution aerial images as training examples.

However, current gazetteers typically only specify the spatial footprint of the objects using a single point (a longitude/latitude pair) and frequently this point does not even fall inside the object. Thus, a major challenge is how to use this weakly-supervised training data to learn geospatial object appearance models. In earlier work titled Integrating Gazetteers and Remote Sensed Imagery [ACM SIGSPATIAL GIS 2008], we showed that this is possible. A major goal of this NSF funded project is to improve the weakly-supervised training by developing more complex object models and better ways to accomodate the missing spatial footprints in the sample images.

The object models will in turn be used to improve the gazeetteers. Specifically, they will be used to estimate bounding-box spatial footprints of the indexed objects. Gazeetteer curators have noted that having even bounding-box footprints will support a wider range of spatial queries. This project will build on earlier work by the PI on Object Localization Using Texture Motifs and Markov Random Fields [ICIP 2003]. Shown below are two examples of using an object appearance model to iteratively grow a bounding box for a mobile home park and a golf course starting from a gazetteer provided starting point.

We will investigate how the object models can be used to estimate the bounding box either by iteratively growing it from a seed point as above or through other techniques, how to detect and overcome the case when the gazetteer point does not fall within the object, and other technical issues. We will also start to investigate the more difficult problem of using the object models to detect new object instances.