Skip to main navigation

Catalogue Blog

Rethinking the “Impact Question”: Evaluating the (Nonprofit) Evaluators, Part IV

The following blog, written by Catalogue for Philanthropy President and Editor, Barbara Harman, was published in the Huffington Post on Monday, February 24th. It is the final post in a series on the “evaluation problem.”

In my previous post, I argued that metrics measure something, but not everything. Let’s take a look at what a basic, metrics-based “logic model” looks like (though note that Charity Navigator’s new model is much more extensive and challenging than the more streamlined model I am suggesting here):

Inputs (what you bring to the table as resources: staff, funds, expertise)
Activities (programs and services; what you do)
Outputs (things that can be measured — numbers of people you serve, units of housing you build, meals you provide, numbers of classes you conduct)
Outcomes (results — impact you have in the short, medium and long-term)

Good, but not good enough. For the model to be complete, it needs to begin with a description and analysis of the community in which you work and the specific challenges you face. If you want to know, at the end of the process, what impact really means, you first have to know, and state, what the conditions are in which your work takes place and out of which it emerges. Describing these is a complex task — sometimes even a moving target — that doesn’t easily lend itself to metrics.

In addition, how you assess your results will depend on what you value. If, at the end of the line, you are measuring something intangible like the resiliency or grit of vulnerable children who have grown up in poverty, you will have a greater challenge before you than will an organization seeking, say, to measure an increase in the rate of employment for job-seeking adults, where numbers are their friends. (This is not to say that the work is harder, only that the task of assessing the work is.) You have to make sure that you have identified grit and resiliency, and any other critical life skills, as core values, and you have to explain why they are.

As citizens and donors, we should do what we can to make sure that those organizations are working to build more creative communities, and to devise programs to deal with extremely challenging (if not, thus far, intractable) social problems, are not excluded because their outcomes are not as easy to measure as others. If I am visiting a community center in Washington, DC’s Ward 8 where the average family income is $9100 a year, I should not be looking at outcomes the same way I would if I were visiting a community where the somewhat better-off youngsters need a smaller boost in order to be successful. The hill is steeper in some places than it is in others, and we have to take that into account.

At the Catalogue for Philanthropy: Greater Washington, we have approached these questions in what is, given the direction that evaluation appears to be taking, a rather unusual way. We have gathered the community of professionals in the field — from foundations, corporate giving programs, peer nonprofits, government agencies and the philanthropic advisory community — and asked them to evaluate applicant nonprofits. Our review process has three stages: programmatic review (the conditions you address, the programs you have created, the impact you have); financial review (reasonable projections of income and expenses; diversified funding; transparency); and site visits (reviewers are asked to share their experience of previous visits, not to visit anew).

Some 120 individuals participate annually, sharing their expertise and direct knowledge. Communities have this knowledge, but it is rarely aggregated or shared with the public at large. We share it in our annual print catalogues and, of course, online, and we are able to do what the rating entities cannot do: actually evaluate need, program quality, and impact — without overburdening community-based nonprofits that, by and large, lack the resources to perform extensive evaluations themselves.

Creating communities of knowledge — actually pooling the know-how of people who have expertise in the field — seems like an obvious thing to do in the service of philanthropy, especially in an era in which knowledge-sharing has become so much easier. It means, too, that we can ask questions that don’t lend themselves to easy answers because we can use the brainpower of the community to identify the nonprofits that are doing the best work. There is no reason why this model could not be shared, and why there could not be a Catalogue for Philanthropy in every region of the country — something we hope to make happen in the not-too-distant future. (A note: the Catalogue focuses on community-based nonprofits with budgets below three million. These are not, by and large, the ones reviewed by Charity Navigator, though this is a category into which the great majority of all nonprofits falls.)

For the moment, though, nonprofits need to remember that — unless they are primarily reliant on the U.S. Government, in which case they had better pay attention to its model — most individual donors are not themselves professional givers. Many are driven more by their desire to give back, their personal passions, and their wish to make a difference and than they are by evidence-based impact assessments.

This does not mean that data and measurement do not matter or that a reasonable approach to evaluating impact should not be part of what foundations are funding and even teaching. But charities also need to find a way to assess their work in a manner that does justice to its complexity, and then translate what they learn into an account that will have meaning and power for individual donors whose contributions make up nearly three quarters of all donations. We should keep in mind that it is not just the good work we do that matters, but also the speaking and writing about it — the sharing of it — that counts. We need to train ourselves and teach others how to be agents of the imagination, ready and willing and equipped to tell compelling stories about the differences for the better that philanthropic work makes.

The task is a challenging, but essential, one. It needs more attention than it has received and I intend to address it in future posts.

Telling a Powerful Story

Individual donors make up approximately 75% of the donating public, a pretty important fact for nonprofits to keep in mind when they are telling their stories to others – to the individual donor him or herself (women make up more of that 75% than do men), to the website visitor, to the newsletter reader, to the thank you note recipient, to the reporter. But as demonstrating one’s impact, proving one’s financial transparency, and clarifying one’s ROI become ever more important, it’s easy to let the plain business of telling the story get lost.

That would be a shame. While donors do indeed want to know that nonprofits are doing important work, and doing it with excellence and impact, and while they absolutely want to know that charities are financially sustainable and sound (this is why the Catalogue’s vetting process is one of our most valuable assets), the majority of donors is still, I would argue, waiting to be moved, waiting to hear or read something that resonates personally, waiting to learn where the need is and where it is best being met.

This is why – perhaps more impulsively than is good for us – we are willing to text away our bank accounts in the face of a disaster, sometimes without even knowing where our money is going. (Personally, I don’t think this is a good thing, though I am sure the impulse behind it is good; we should all know to what use our funds will be put before we give them away.) We see the need – often in powerful images, as we did last month in the aftermath of the Boston Marathon bombings and as we have in recent days in the wake of the terrible tornadoes in the Midwest. (For those who want to do their research and decide where best to give, take a look at this list of disaster relief organizations working in Oklahoma.) The stunning images of devastation, whole towns wiped out in seconds, one house standing here while its neighbor is gone there, and the accounts told by survivors – all these speak powerfully to the painful loss of those whose friends and family members died, and the needs of those who must remake their lives in the aftermath of this terrible disaster.

Stories of real need move us, and there are many such stories to be told – some more immediate and dramatic than others, and…many more than there should be. The problem is, we aren’t always very good tellers. We get bogged down in our own internal languages – jargon of the trade, insider talk that only our colleagues understand, a too-numerical view of what “impact” means. We need to speak to each other in a human voice, help the reader understand what the real need is that we are meeting and why it deserves the reader’s attention. We need to describe what we are doing to meet the need in a way that conveys important information that is still compelling and coherent (not a list of seemingly unrelated programs). We need to talk about impact – through powerful metrics if we have them, but in narratives if we don’t. We need to convey our vision of the future in a way that is inspirational and aspirational. And we need to communicate to donors, directly or indirectly, how a contribution to our cause will make a genuine difference.

Above all, we need to speak in a human language, a human voice – individual to individual, person to person, as members of one human community. In fact, helping readers to see that we are indeed members of a shared community is perhaps the best way to help them see the power and importance of joining the cause.

Barbara Harman gave a version of this talk at the America’s Charities Members’ Meeting on May 21, 2013.

The New Face of Philanthropy

Yesterday, the Meyer Foundation hosted a workshop about a recent report on next generation philanthropists: #NextGenDonors: Respecting Legacy, Revolutionizing Philanthropy. The report, a project of 21/64 and the Johnson Center for Philanthropy, surveyed thousands of high-capacity Millennial and Gen X donors (ages 21-40) in the United States. High level findings from the report show that next gen donors:

1. Conduct due diligence and do research before deciding whom to support.
2. Decide philanthropic goals or ideal solutions first, and then search for potential recipients who fit those
3. Fund efforts that address root causes and attempt systemic solutions
4. Prefer to have information about an organization’s proven effectiveness or measurable impact before deciding whether to support it
5. Often recommend a cause or organization to others

Many of the other trends that emerged from yesterday’s conversation include the importance of technology in engaging and cultivating donors (like the preference for email or online communications over printed mail) and the importance of demonstrating impact and outcomes. However, I find that these are the same trends that are currently discussed in conversations about “today’s donor” and not just the “30 year old, high capacity donor.”

At the Catalogue, we try to look at all emerging trends in philanthropy from the lens of a small nonprofit. One question at the event yesterday hit the nail on the head, in terms of recognizing the impact of a new donor profile for the small nonprofits with restricted resources and capacity – what does this all mean for us? How do we balance our current donor outreach with the type of specific engagement that is suggested for next generation donors…and continue offering the services and programs that our clients need? There are only so many hours in the day and dollars in the budget.

Not many donors are directly asked to reflect on the constraints that small nonprofits face and consider how this might impact their donor outreach. But for donors who admittedly have a preference or inclination towards the start-ups or little guys, perhaps they should be. It’s a tough question to consider for someone who isn’t immersed in the day-to-day operations of a nonprofit with a budget under $2 or $3 million (or even under $500,000). Obviously certain aspects, like a well-structured and aesthetically pleasing website, are must-have’s for today’s tech savvy world, but other types of donor personalization might be out of reach for an organization with a limited development team. This is gap that the Catalogue attempts to fill – addressing the information asymmetry in the philanthropy marketplace by allowing small nonprofits’ best versions to shine and connecting them to donors with whom those stories resonate.

How does your nonprofit attempt to engage millennial or next gen donors? Do you have specific outreach for this demographic? Let us know what you think!

In The News …

DC-Area Apartment Rents Rise, Vacancy Second Lowest in the Country (Urban Turf via DCentric): “… rents in the DC area for Class A and B apartments have risen 3.6 percent over the past twelve months while vacancy rates sit at 2.8 percent, the lowest for any metro area in the country except for New York City.” For Class A apartments (primarily large buildings constructed after 1991), the vacancy rate is just 1.6% and rental rates “averaged $2,582/month, up from $2,448/month in September 2010.” So not only is affordable rental housing difficult to locate in the city, any rental housing is not easy to come by — and the search alone can require both funds and time.

Continue reading

The Magic Number?

Should all non-profits aim for that magical 75% / 25% ratio of program costs to administrative costs? What makes a “high-impact” non-profit high-impact? Should we assign a number to something like “impact” or should we define it differently?

Washington Life Magazine asked Catalogue President, Barbara Harman (me), Catalogue Director of Partnerships Kathy Jankowski, and the Nonprofit Roundtable’s Chuck Bean, to think about these questions. Check out our answers in this piece and let us know what you think!