Skip to Content

The limitations of data-driven policymaking

My work on opportunity mapping at NHC has uncovered some important issues regarding the use of data in both describing social problems and in prescribing the solutions. For example, one question that came up during the webinar I recently hosted referred to the limitations of data-based methods like opportunity mapping. In the interest of expanding the discussion out from simple description, I wanted to provide a more nuanced discussion of what opportunity mapping and other forms of data-driven policymaking can mean for the participants on both sides of the process.

Opportunity maps have an unavoidable limitation: they are static representations of very non-static processes and relationships. Maps are a very powerful and important tool for translating facts and figures into a more relatable format. However, like a travel guide, they are outdated from the moment they are printed. Neighborhoods in many of the cities embarking on the great Geographic Data Journey (most recently prompted by the updated AFFH rule) are changing rapidly. On top of that, there are a lot of things that can factor into “opportunity” that are not easily mapped, such as social relationships. The ability of households to rely on neighbor and family members can mitigate some of the negative consequences that come from living in “low opportunity” areas.
Besides the inherent limitations of using a static map to represent active social processes, another issue to consider is how the data informing the maps are chosen. Many of the data used for making opportunity maps are “publicly available,” but this does not necessarily translate to “publicly accessible.” Equity in data access can be difficult to achieve given the technical knowledge required to find, analyze and visualize data. Not only can the simple process of gathering and packaging data for analysis be time-consuming, but effective analysis and visualization of that data often requires some sort of training in various software packages and analytic theories. Data can be difficult to wrangle even for seasoned researchers; community members that haven’t had the privilege of learning the same skills are likely to struggle even more.
One potential answer for both of these issues may come in the form of community engagement. Including community members at various steps in the mapping process helps build equity into the final product. First, community members can be engaged with the selection of indicators, as well as during the data collection stage. This can help address the first concern mentioned above. Second, the actual analysis of the data and the creation of the map can benefit greatly from including community members whose voices are less likely to be heard in such technical policymaking processes. Training these community members might be beyond the scope of any data analysis program (in terms of agenda and in terms of simple capacity). However, the simple act of incorporating the importance of community perspective increases the possibility of policies that are inclusive rather than divisive.
The use of large datasets in analyzing social problems is an increasingly popular topic. Researchers at the Urban Institute recently published an essay on the use of data in predicting neighborhood change. The authors also cite the work of the Bloomberg Philanthropies What Works Cities project, which has funded an expansion of data-driven policymaking in 39 cities across the United States. Another recent data-driven project is the Silicon Valley Triage Tool, an algorithm developed to efficiently distribute resources for homeless people. The use of an algorithm to solve social problems created by a local economy whose success is based on monetizing algorithms may seem a bit too on-the-nose. Mother Jones has a balanced discussion of the tool and its implications. The federal government has also been participating in this expansion of data-driven policymaking. In addition to the broader Open Government Initiative, the White House recently launched the Opportunity Project, which collects open data sources from federal agencies and a dozen cities around the country.
Clearly, data-driven policymaking is becoming part of common practice. The potential for new insights is too great to ignore. Like other data-driven methods, opportunity mapping is an attempt to redress inequities that have built up over time. It’s important to remember that maps contributed to these inequities through the practice of redlining, and the seeming objectivity of data has been used in the past to justify oppression. As cities begin to grapple with the flood of data that is washing through policymaking, we need to make sure that the most vulnerable communities are not swept away, but are instead lifted up.
Refine Topics