Are social programs making a measurable difference in the lives of those being served? Nonprofit leaders and stakeholders have been asking this question for a long time. Yet, there have been few who have ever received an answer! And, here’s why…
- Only the largest nonprofits and funders can afford the extremely costly price tags of rigorous outcome evaluation.
- Nonprofits can’t afford to hire the necessary staff with research and evaluation expertise to sustainably measure their impact.
- Complex data gathering systems often take a tremendous amount of time and resources to be purchased, installed, maintained and updated.
- All staff engaged in the effort require a lot of technical experience, training and support to be able to compile, clean, analyze and report data.
- It takes a long time to gather enough data (cases) to draw clear cause-and-effect conclusions.
- Most nonprofits don’t have the data science capacity needed to run cause-and-effect analyses.
- Most nonprofits don’t have the design and communication capacity to translate data findings into usable, real time decision-making insights, tools and reports.
As a result, few nonprofits can learn from their efforts to improve lives and communities. The best that the most well-resourced nonprofits can do is spend a heck of a lot of time and money (about $20 billion dollars each year) simply tracking numbers served – or, “outputs.” Yup, you heard right, just numbers served!
These realities are causing some in the sector to abandon the idea that nonprofits will ever be able to measure beneficiary outcomes (meaningful positive change in people’s lives).
Don’t Give Up!
For the sake of the beneficiaries of every program everywhere, it would be a huge social injustice to give up on measuring outcomes! Mammoth!!! It is NOT OKAYthat the only measurable benefits we can prove to beneficiaries are that “you’ve been served,” and “here’s how much you got.” Don’t we all agree that the sector’s purpose shouldn’t be reduced to the same metric as a fast food restaurant?
The first problem with only tracking outputs (i.e., number served) and dosage (i.e., how much) is that the data are likely biased by who is doing the collecting. Numbers served has understandably been established by public and private funders as an accountability metric to ensure that their resources are being used for their intended charitable purposes. But, for the nonprofit, it has become a costly compliance task that gets pushed down to the program implementers, whereby they document who showed up, for how long, and what they delivered. The quality and validity of the data depend on the subjective self-reporting of program staff, documenting their rates of recruitment, attendance, and retention, as well as the amount of services they delivered. Social scientists have consistently proven that bias increases when a data collector’s job security and promotion depends on the data they collect.
The second problem with only tracking numbers served is that we don’t listen to the beneficiary’s voice at all. In the world of accountability for numbers served, the only required beneficiary “voice” comes in the form of showing up, raising their hand, calling out, “I am here,” or swiping their ID card. Numbers and dosages – outputs – are worthless pieces of information if there are no associated beneficiary outcome data from which to draw cause-and-effect conclusions. Being served is not the same as being helped. If beneficiaries had a voice in all this they would very likely say, “don’t just count me to get paid and think I am being helped based on your gut; help me in a measurable and meaningful way!”
The greatest injustice to beneficiaries is that when funding and/or reimbursement are based on numbers served, many of the most needy don’t get served. When organizations are incentivized to serve as many as possible at the lowest cost, it makes sense to focus on serving those with the fewest challenges. It is a fact of life that individuals with more challenges in their life take more time and resources to help! This is happening much, much more than we like to admit.
In all fairness, the sector simply hasn’t had widespread, affordable access to the tools of outcome measurement. Therefore they haven’t had the capacity to move beyond outputs. That is, until now…
It’s a New Day!
We are truly at an innovation crossroads! Cloud computing, big data, predictive analytics, social networking and online learning all make it possible to create, measure and learn from social impact. This is being achieved in ways we’ve never dreamed possible before, including making it affordable and feasible for every nonprofit.
So, here’s how 21st Century social impact measurement and learning can transform the way our sector gets down to the business of improving lives. Imagine you’re a nonprofit leader running an organization that provides out-of-school/after-school programs for youth, with the goal of improving their socio-emotional learning (SEL) skills and behaviors. The long-term vision is to prepare them for success in today’s job market. All research shows that youth who have these critical SEL skills and behaviors do better in school, are less likely to engage in risky behaviors, and are more successful in the 21st Century workplace.
Your youth development organization becomes a member of a group of other similar youth development organizations from all over the country that are participating in an online impact learning experience. The purpose of this experience is for every staff member and leader to experience feasible, sustainable and ongoing “impact learning by doing.” Like the online learning platforms of Khan Academy and Coursera, content and how-to instruction is delivered through brief pre-recorded videos and webinars. These offerings teach about logic models, the research base for youth development, evidence-based practices, outcome measurement, data collection, and other field-specific topics related to social impact measurement and learning.
Also like these new online learning platforms, this experience includes a “learning-by-doing” opportunity. Each organization can use what they are learning with the help of the technology, tools, supports and technical assistance they need. With all of these resources at their fingertips they are guided through a feasible and sustainable process of conducting rigorous impact studies of one or more groups of youth; “no experience necessary!” At the center of this impact learning experience is an innovative technology solution. It standardizes and automates all of the rigor, data collection, outcome measurement, data analyses and real-time reporting that typically impede most organizations from learning about their impact. For the first time ever, your organization experiences:
- Real-time baseline outcomes – a.k.a., needs assessment findings – for all of the youth you serve, before a program begins;
- Recommendations before a program begins to help your staff maximize youth outcomes;
- After the program ends, statistically valid and reliable outcome findings, including seeing the percent of youth who your programs have significantly improved, alongside the cost per outcome;
- Data-driven findings about what worked, including insights and recommendations found within the rapidly-growing dataset of programs, practitioners and youth represented in the larger learning network; and
- Program managers and practitioners viewing the results of their efforts, as well as findings on their perception of the delivery of programs, with comparisons to what the youth perceived that they got.
Now, imagine that these 21st Century social impact learning experiences and tools are accessible to any youth development organization at a cost that is1/10th to 1/20th of what it would have cost in the past!
It’s a New Day for Those Who Support Nonprofits, Too!
For all of the funders, evaluators, capacity builders and intermediaries that want to invest in, advocate for and support individual and collective groups of nonprofit organizations, imagine being able to grow your business with the tools of social impact measurement and learning that used to be available only to the biggest of the bigs. It’s similar to how Uber and AirBnB have developed a disruptive technology application that makes it possible for individuals to run their own transportation or bed & breakfast business. This social impact measurement “application” would allow nonprofit supporters to run, enhance and grow their social impact “businesses” using the measurement and learning tools that, to date, were only accessible to the top 1% of the philanthropic and nonprofit sector.
Algorhythm has Cracked the Code
Our team of social scientists, data scientists, computer engineers, human-centered designers and capacity builders have created a technology platform that democratizes social impact measurement and learning for the nonprofit sector. The platform makes it possible for leaders to affordably build and provide widespread access to real-time, rigorous, shared social impact measurement tools and processes.
Think of Algorhythm’s platform like a resource for groups wanting to build the analog of a software application to put into an app store. Our platform provides access to all of the expertise, supports and technology needed to develop, standardize, automate, and provide affordable ongoing access to shared social impact measurement applications (called iLearning Systems) for fields of practice – e.g., youth development, senior services, juvenile justice, nonprofit capacity building, etc.
However, the value of our platform to you doesn’t stop there. Our team has learned through decades of experience that social impact learning doesn’t happen if all you do is deliver a report. The true learning and innovation happen through meaningful educational experiences and people-to-people exchanges, where all stakeholders have an opportunity to “gather” and “make meaning,” together! We have incorporated our iLearning System into a larger “learning by doing” experience. Everyone gets to learn about social impact measurement through online educational offerings, webinars and gatherings, and then apply what they’ve learned by using their field’s iLearning System. We call this capacity building experience the iLearning Institute.