Geospatial Impact Evaluation (GIE) is a quasi-experimental statistical approach that aims to estimate the effect of wide range of policy interventions on land cover at different scales. Because many types of policy interventions, such as forestry, agriculture, water resource management, urban development, and pollution control, are associated with land cover values, GIE can shed light on results across time and space.
To establish the impact of an intervention, we compare land cover values within the area targeted and in comparable areas that were not targeted by an intervention. We construct a set of control areas that have observed characteristics similar to the areas that receive interventions. Treatment effects of the intervention are estimated based on statistical comparisons of these matched locations.
The developing world has a pressing need to understand what types of aid activities are most successful at promoting sustainable development. Better evidence is needed about why some programs work and others do not. Evidence is limited because impact evaluations are typically expensive and results are difficult to measure across time. GIE uses high-resolution, high-frequency data that is available globally across many years.
Development organizations are spending hundreds of millions of dollars each year on programs to achieve environmental and development goals. GIE has the potential to rapidly increase the amount of evidence available about a wide range of intervention and can be linked to other types of impact evaluations.
To carry out GIE, we build a high-resolution geospatial and remote sensing database that allows for statistical matching and the construction of control areas across a suite of demographic, environmental, and other geographic attributes.
Demographic measures typically include the full range of census variables available such as race, income, family, size, economic activity, education level, etc. from official data. The environmental suite of data typically includes the land cover measures, forest measures, slope, soils, land disturbance measures, deforestation rates, biome data, ecoregion information, climate, etc. The other geographic attributes include rivers, roads, utility infrastructure, and the distance to each of these locations. This grouping also includes indigenous boundaries, protected lands, managed communities, and other municipal boundaries.
The goal is to build as complete a geographic dataset for the regions as possible and then match based on the variables that predict land cover change.
Disentangling the direct effects of land management interventions from the background factors that drive both land cover change and the targeting of interventions is a major challenge. Both observable and unobservable differences between treatment areas that receive an intervention and control areas that do not receive an intervention can confound estimates of treatment effects.
We employ a matching approach to construct a set of control plots that have similar pre-treatment characteristics as the plots that were assigned to treatment under the interventions. As noted earlier, we compile spatial data on other variables that have strong associations with land cover change. We then iteratively search through sets of control observations outside of intervention areas to select a set of control units that have observed distributions for each covariate that are not observationally different from the distributions of the same covariates in the treatment plots, —ρ(Xi | Ti = 1) ≈ ρ(Xi | Ti = 0)— where ρ is the observed values in the selected set of treatment and control groups, X is a matrix of covariate values, and T is the treatment state. This is geospatial matching.
We carry out statistical matching using a genetic algorithm that both weights and discards control plots that do not serve as comparable units to the treatment plots. We apply a genetic algorithm that iteratively searches through many sets of potential control plots to maximize balance on the observed covariates. Good matched sets are those that have a large minimum p-value on a paired t-test for differences of means between treatment and control observations across all covariates. These sets are passed onto the next generation, along with mutated sets to ensure the full space of combinations of observations for the control set is explored.
Stuart Hamilton is Assistant Professor of Geography and Geosciences at Salisbury University. He has published research in Land Use Science on international aid and mangrove deforestation, in Global Environmental Change on land tenure and tropical forests, and in Bulletin of Marine Science and BioScience on the role of aquaculture in driving deforestation. He is currently serving as a Prometeo Fellow within the Ministry of Environment in Ecuador and helped establish the GIS lab at the National Fisheries Resources Research Institute in Uganda.
Mark Buntaine is Assistant Professor of Environmental Policy at the University of California, Santa Barbara. He has published research on the management and impacts of foreign aid in a variety of leading journals, including Global Environmental Change, International Studies Quarterly, World Development, and Climate Policy. He is currently working with a number of implementing partners in Uganda, India, and China to design and execute impact evaluations related to environmental management and governance.
Marco Millones is an Adjunct Assistant Professor of Geospatial Information Sciences at the University of Dallas School of Economic, Political and Policy Sciences. He has 10 years of academic and professional experience applying spatial analysis and remote sensing to build evidence on the drivers of land cover change for governments, non-profit organizations, and research institutes in several Latin American countries. He has published research in the International Journal of Remote Sensing, Transactions in GIS, Applied Geography and GEC.