Data, especially cause and effect data, are very hard to come by in the nonprofit sector. We’ve all been asking a lot of impact questions for a very, very long time. How many lives have we touched? How many lives have actually improved? Do we know which social programs really work? What specifically about a program model made the difference? For whom? These are the questions we all really want answered! But – and this is a huge BUT – we simply don’t have enough data to answer them!
The Nonprofit Sector Has Been Foraging for Impact Data for Far Too Long
The nonprofit sector has a decent amount of data answering only one of the above questions — How many lives have been touched? But, these data are NOT measuring actual beneficiary improvements, nor are they capturing their program experiences and stories. These are staff-entered data that document numbers served, as well as dosage. So, no matter how much foraging anyone does through these “output” datasets, you won’t find much true and reliable outcome data. And, each organization’s data are almost always disconnected from other similar organizations. As a result, there may be more of these “counts” and “dosages” data than anything else, but they are still not that big, in part because they sit within organizational silos. Even if this output data were connected, without reliable and valid outcome data sitting right alongside, we cannot answer whether a program succeeded, with whom and how.
How Bigger, Better Data Can Advance Nonprofit Effectiveness Right Now from Algorhythm on Vimeo.
The best data foraging comes from objective studies conducted by researchers and program evaluators. There are some key studies within a few fields of practice – e.g., youth development programs, home visitation programs, etc. – where rigorously gathered datasets are beginning to tell us if a program makes a difference. However, these datasets often contain very few cases; most with less than a few hundred cases. With so few cases, it is very hard to study anything more than whether a program, versus no program, made a statistically significant difference. To determine which specific program experiences, practices and/or components work for different segments of a population, we need much, much more data. And, these studies rarely, if ever, have data representing a random sample of the entire population that is in need; a program may work with a specific group that is studied, but we cannot say it will work with everyone unless a high enough number of study participants come from each and every possible type of group, community, setting and environment. So, we can and should forage from these studies to pull out key insights, learn more about if a program can succeed with those who are the same as those who were studied. But, without a lot more data we cannot to conduct the kind of analyses we need in order to figure out what works, for everyone, and under what conditions.
It’s Time to Form Data Farming Collectives!
It’s time to move from a foraging stage of data development, to data farming. In fact, if we play out this very apropos analogy, every nonprofit, funder and researcher within specific fields of practice (e.g., youth development, nonprofit capacity building, etc.) should be forming or joining data farming “co-ops,” or “collectives.” Why? Wikipedia explains it best in its description of actual agricultural cooperatives, which I alter here to highlight the point: “A practical motivation for the creation of agricultural [data] cooperatives is related to the ability of farmers [nonprofits, researchers, intermediaries, and funders] to pool [data] production and/or resources. In many situations within agriculture [the nonprofit sector], it is simply too expensive for farmers [nonprofits, researchers, intermediaries and funders] to manufacture products [data] or undertake a service [an evaluation study]. Cooperatives provide a method for farmers[nonprofits, researchers, intermediaries and funders] to join together in an ‘association’, through which a group…can acquire a better outcome [data]…than by going alone.” Wow! Quite the analogy!
There are many fields of practice within the nonprofit sector, each of which has a group of associated funders, researchers, nonprofits and other intermediaries. For example, there are groups working in “fields” (another farming reference that works, yuk, yuk!) like youth development, nonprofit capacity building, child welfare, juvenile justice, employment, senior services, health, mental health, environmental sustainability, human rights, and the list goes on. The potential for growing bigger, better and much cheaper data for impact assessment, planning and evaluation would be huge if everyone within a field were to join a data farming collective.
At the center of a data farming collective for a field of practice has to be a shared set of reliable and valid data collection “tools,” measuring and producing standardized data on organizational capacity, program delivery and beneficiary outcomes, all sitting alongside one another. With these three types of connected data, everyone in a field could finally learn about the cause-and-effect relationship between organizational performance, program delivery and beneficiary outcomes. Here’s how a data farming collective helps everyone:
- Researchers would access a much bigger dataset for conducting deeper and more rigorous research studies, including finding and studying comparison groups within a very large and growing dataset of beneficiaries, everywhere;
- Funders can access their grantees’ standardized results data and, for the first time ever, see apples-to-apples how a portfolio of organizations are doing, share findings with their boards and stakeholders, and plan for their next grant strategy, initiative and/or cycle;
- Nonprofit leaders and managers can use the data for program design and evaluation, as well as provide front line staff with better needs assessment and program planning insights, even before a program begins; and
- All of the intermediaries that support and advance the development of those within a field of practice can identify, share and build the capacity of everyone to apply the lessons and “how-to” strategies of the shining examples of success.
Algorhythm is an impact measurement technology business that partners with researchers and intermediaries within fields of practice to build and sustain real-time shared assessment, planning and evaluation tools. Algorhythm’s technology platform has drastically reduced the cost of rigorous impact measurement, while making it possible for everyone, everywhere, including those on the front lines, to learn from true outcome data, in real time. If you would like to plug your impact measurement tools and/or build new impact tools into our real-time, rigorous assessment, planning and evaluation platform, reach out to at info@algorhythm.io.