What is GIE?

Geospatial Impact Evaluation (GIE) is a quasi-experimental statistical approach that aims to estimate the effect of wide range of policy interventions on land cover at different scales. Because many types of policy interventions, such as forestry, agriculture, water resource management, urban development, and pollution control, are associated with land cover values, GIE can shed light on results across time and space.

To establish the impact of an intervention, we compare land cover values within the area targeted and in comparable areas that were not targeted by an intervention. We construct a set of control areas that have observed characteristics similar to the areas that receive interventions. Treatment effects of the intervention are estimated based on statistical comparisons of these matched locations.

Policy Relevance

The developing world has a pressing need to understand what types of aid activities are most successful at promoting sustainable development. Better evidence is needed about why some programs work and others do not. Evidence is limited because impact evaluations are typically expensive and results are difficult to measure across time. GIE uses high-resolution, high-frequency data that is available globally across many years.

Development organizations are spending hundreds of millions of dollars each year on programs to achieve environmental and development goals. GIE has the potential to rapidly increase the amount of evidence available about a wide range of intervention and can be linked to other types of impact evaluations.

The Geospatial Component

To carry out GIE, we build a high-resolution geospatial and remote sensing database that allows for statistical matching and the construction of control areas across a suite of demographic, environmental, and other geographic attributes.

Demographic measures typically include the full range of census variables available such as race, income, family, size, economic activity, education level, etc. from official data. The environmental suite of data typically includes the land cover measures, forest measures, slope, soils, land disturbance measures, deforestation rates, biome data, ecoregion information, climate, etc. The other geographic attributes include rivers, roads, utility infrastructure, and the distance to each of these locations. This grouping also includes indigenous boundaries, protected lands, managed communities, and other municipal boundaries.

The goal is to build as complete a geographic dataset for the regions as possible and then match based on the variables that predict land cover change.

The Statistical Principle

Disentangling the direct effects of land management interventions from the background factors that drive both land cover change and the targeting of interventions is a major challenge. Both observable and unobservable differences between treatment areas that receive an intervention and control areas that do not receive an intervention can confound estimates of treatment effects.

We employ a matching approach to construct a set of control plots that have similar pre-treatment characteristics as the plots that were assigned to treatment under the interventions. As noted earlier, we compile spatial data on other variables that have strong associations with land cover change. We then iteratively search through sets of control observations outside of intervention areas to select a set of control units that have observed distributions for each covariate that are not observationally different from the distributions of the same covariates in the treatment plots, —ρ(Xi | Ti = 1) ≈ ρ(Xi | Ti = 0)— where ρ is the observed values in the selected set of treatment and control groups, X is a matrix of covariate values, and T is the treatment state. This is geospatial matching.

We carry out statistical matching using a genetic algorithm that both weights and discards control plots that do not serve as comparable units to the treatment plots. We apply a genetic algorithm that iteratively searches through many sets of potential control plots to maximize balance on the observed covariates. Good matched sets are those that have a large minimum p-value on a paired t-test for differences of means between treatment and control observations across all covariates. These sets are passed onto the next generation, along with mutated sets to ensure the full space of combinations of observations for the control set is explored.