Tools for social impact assessment

The Foundation Center’s Tools and Resources for Assessing Social Impact – TRASI - community had an online launch today. It was a chance to hear from the TRASI’s creators about how the project came about, what’s next for the project, and how to make sense of such a long and varied list of tools and methods.

Bottom line: Each situation is unique, and you need to pick the right tool according to your unique assessment needs. If you need a tool but you’re not sure which one, TRASI is one stop shopping:

Tools: Global Impact Investing Ratings System (GIIRS), HIP Framework, Impact Reporting and Investment Standards (IRIS), B Impacts Rating System, Charity Navigator, LEED

Methods: Acumen Fund Scorecard, Balanced Scorecard, GRI Framework

Best Practices: Theory of Change (Keystone Accountability), Impact Framework (Independent Sector), Community of Learners (TCC Group)

Here are the main themes from the discussion (with selected quotes from experts):

How the project started

Laura Callanan: “TRASI was originally envisioned by the Learning for Social Impact team at McKinsey. We saw the need for a repository which brought together tools, methods and best practices from across the social sector in one place. There are repositories for microfinance assessment tools, and social service assesment tools — but no opportunity to look across sectors and borrow and tweak tools.”

How tools and methods were gathered

Callanan: “We identified the tools, resources and best practices which we thought were great examples of how to do assessment well. We also tried to ensure that familiar methods like balanced score card and SROI were included.”

A fragmented field

Steve Godeke:  “I see the great diversity of tools rather than lots of redundancy. This may have implications for those seeking to simplify the field.“

Callanan: ”One observation — it wasn’t that hard to accumulate a critical mass of 150 tools! We all know the sector is fragmented, and here is just another example. During the beta test, the Foundation Center which know manages and leads the TRASI database, added more resources… we intentionally tried to showcase the vairty of tools that are out there.”

Godeke: “I think it is important to first understand what you are trying to achieve with the assessment (key decisions, strategic options) and find the appropriate tool to aid the decision-making.”

Tom Lumpkin: “It would be nice to zero-in on a few great tools so we could accumulate reliable and comparable information. But the aims of nonprofits and other orgs are often so diverse that there is clearly a need for a diverse set of instruments.”

Callanan: “There are expert ratings provided for the tools and resources which appear in TRASI. The experts provide guidance on which tools are best for which purposes.”

Next steps

Elena: “Has TRASI explored ways of increasing the comparability of different assessment methodologies – in your opinion, where is that standarasation supposed to come – from the methodologies applied, the tools used or a combination of both?”

Lumpkin: “It seems that actually using the different tools and sharing what works well and not so well is needed… For there to be standardization, the tools would need to be used in the same fashion across all users. Then results can be compared and “flaws” in the instruments, if there are any, can be worked out. That will take a community if users sharing their findings and experiences.”

Elena: “That could be an ambitious but hugelu useful contribution for TRASI to make!”

Making capital more patient

For a while I’ve been thinking of how to answer this question: Is venture capital too selfish?

My initial reaction is a qualified yes. The world is not short of profit-minded investors who wring out the maximum (financial) value from their investments. But self-interest of this kind isn’t necessarily the same as selfishness, and whatever negative baggage is attached to the latter.

To help answer the question, two stories I heard recently shed light on how some insiders are supporting socially responsible ventures in ways that traditional VC does not.

One is the story of Guayaki Yerba Mate, and the role played by patient capital in achieving their remarkable impact. In particular, ‘mission-aligned growth capital’ from RSF Social Finance allowed Guayaki ‘to grow without requiring a liquidity event such as a company sale or an initial public offering.’

Second, a new initiative called Presumed Abundance confronts the often adversarial nature of investor exits. At the time of the scheduled exit, this program encourages angel investors to plow their investments back into their growing ventures, and use that equity (once it has appreciated in value) to fund the next group of promising new ideas.

This is a unique breed of investor indeed. Though I imagine this is not far off from what angel investors plan to do with their payouts anyway, though working together in this way will build everyone’s experience in building a networked community of interdependent ventures.

I don’t mean to blame investors who don’t come close to these supportive investment models, and who are, in fact, selfish. Some amount of aggressive profit-seeking behavior pushes things forward.

In fact, profit-seeking that actually impedes innovation should not be condemned out of hand. Some selfishness is rooted simply in industry norms. Many traditional investors look for a set of signals – including a physical office and, yes, large financial upside – in order to separate good investments from bad.

But these norms must change over time, and what we need are some well-timed nudges like the ones mentioned here.

‘Pleading for help’ with data and evaluation

To the pessimists who think ecological change is too complex and too slow to affect environmental policy-making, the federal government thinks you’re just not trying hard enough.

This is the message I heard Shelley Metzenbaum deliver to a group of environmental evaluators earlier this week. It was interesting to hear this and similar perspectives from the many agency people (mostly NOAA and EPA) who were at the gathering.

The message I came away with was that the OMB – and the federal government more generically – wants (and needs) help to meet the Obama Administration’s goals for transparency and accountability. There can never be too many people working to collect and analyze high-quality information.

The complexity and pace of ecological change are stumbling blocks for many – the government is not unique in this respect. Earnest, numbers-minded people hesitate to overly simplify the cause-and-effect relationships that drive ecosystem functioning (and degradation).

However, the fact that change often occurs more slowly than the pace of our policy-making decisions is not an excuse for giving up on measuring performance.

Metzenbaum, who reports to the nation’s first Chief Performance Officer, said there is plenty to measure and analyze (and learn) even very early on in any effort to address an environmental problem. One example: Take an early success, try to replicate it in another place with different circumstances and contextual factors, and see how the outcomes compare. 

Some worthwhile analyses, of course, are simply beyond anyone’s ability to do. But the government should not be alone in confronting our society’s pervasive water and air quality problems.

The private sector, including private philanthropy, has a role to play. Even analyses that are within the government’s scope won’t be relevant without the credibility of non-government participation.

As an outsider looking in, I hope the government helps us help them by sharing the environmental problems they see, and the data that show them.

In the quest to prove the worth of efforts to protect and restore the environment, data collection, use, and analysis are shared tasks. After all, we’re all working to realize the same (shared) environmental benefits.

When to invest in measurement

Early in this year’s Environmental Evaluators Networking Forum, Nick Salafsky made a convincing argument for investing in the quality of information in an ‘era of results-based performance.’ Nick highlighted the Conservation Measures Partnership’s work to design, manage, and measure conservation impacts.

Such efforts, however, cost money and other valuable organizational resources. So we also heard about when such investments should be made.

Programs should invest in measures when:

1. Stakes are high
2. Potential exists to leverage learning
3. Costs are low relative to actions

A foundation making grants – or any other organization deciding how to allocate scarce resources – must make choices about the number and types of investments to make, and how much focus and attention to pay those investments once they’re made.

The Packard Foundation was cited as an example of a foundation that makes a relatively small number of (relatively large) grants. (Marisla was the example of the opposite case.)

Given the small number of grants, it was implied that Packard is better able to support grantees with the tender loving care of impact measurement and evaluation. An organization making lots of smaller investments necessarily will be more passive in their support and monitoring.

Indeed, in Packard’s case, the rationale for spending money on measuring impacts is that although grants are made strategically, and often to established and proven organizations, there is a risk associated with using only a handful of baskets to hold all the eggs.

The argument follows, then, that we should be especially deliberate in the monitoring and evaluation of these ‘big bets.’

At first, I was very aware of, and concerned about, of the trade-off of spending resources on evaluation, at the expense of investments in the program’s objectives.

After several months, however, I have come to appreciate the need for digging deep into each grant decision. Evaluation simply cannot be separated from a thoughtful grantmaking strategy.

Since the stakes are high (#1), there is much learning to be shared with the community (#2), and evaluation costs, in the grand scheme of things, still are relatively low (#3), investments in evaluation seem entirely justified. In fact, they are critical to success.

What I would do with $2 million

Thanks to the success of Sophie’s World, a novel about philosophy which sold 30 million copies, Jostein Gaarder, a Norwegian, is a rich man.

When I heard about the $100,000 annual prize (Andy Revkin wrote about it) he created with this newfound wealth, I immediately started to do the math, to see just how rich he must be to fund such a thing.

The answer: an annual prize of this size might be easier (or at least cheaper) to do than I initially thought. In the U.S., a private foundation retains its tax-free status by paying out 5 percent of its total assets each year.

Now, if the endowment is invested (which it should be), then the amount of assets will be a moving target. And there is also the small matter of the management and legal costs to actually do the foundation’s business.

But leaving these two details aside for a moment, the quick math is that a $100,000 annual prize would require an endowment of:

$ 2 million

Not pocket change, but well within reach of many people in this country.

There are certainly alternatives to this approach, and we could argue whether something like Gaarder’s prize has an impact on par with those other options. But it certainly is an opportunity for people who seek to have that impact on the world in a tangible and visible way.

The virtual ecosystem of a new venture

In the past couple weeks, two rising Bay Area entrepreneurs separately told me the same bizarre story about a new obstacle to raising capital: ‘traditional’ investors are reluctant to support a venture lacking a physical office.

Those investors must think a physical address is a status symbol of sorts, an essential tool to foster the teamwork and hard work required to launch a new business.

The entrepreneurs couldn’t disagree more.

They argued – rather convincingly – that a physical office was expensive, inefficient, and wasteful: in terms of electricity, commuting, printer paper and bad coffee, to name a few.

What a sharp contrast to the streamlined virtual offices they run instead. The entrepreneurs argue that the start-up phase of the businesses is exactly the wrong time to take on avoidable expenses.

Fast forward to a somewhat better known story from Harvard Business Review’s blog about YES Bank, which has achieved startling impact by being successful in a previously unprofitable market: microfinance in India.

How does YES Bank do it? In short, they do more with less by minimizing overhead:

“Rather than managing its technology in-house it has outsourced it all; rather than reinventing the wheel, it taps a vast network of partners who offer specialized services… By leveraging an ecosystem of partners and the power of connectivity, Rana Kapoor believes his bank can cost-effectively deliver more financial value for even less cost to a much larger number of customers”

As a result, the bank earns 2% over its cost of lending, a 3 to 3.5% improvement over competing banks (which, by implication, typically operate at a loss).

This focus on network efficiencies got the bank over a small but persistent hurdle – offering competitive and attractive interest rates while still reaching financial sustainability – in order to deliver immense social value.

I believe this may be a future opportunity here in the U.S. as well as in developing economies: finding ways to launch previously unprofitable business models by tapping into this kind of networked, virtual business ecosystem.

Numbers that matter

One of my main jobs at the Packard Foundation is to identify meaningful metrics to help us assess and evaluate the impacts of our investments. It is a challenging and interesting job, and harder than I thought it would be.

As with previous projects, the challenge is not coming up with the metrics per se. The bigger task is cutting through the clutter to enable a few key numbers to the surface in order to tell a clear and succinct story. Very often the hardest part is deciding which numbers not to include.

In this context, I really liked this recent HBR post about numbers that surprise. It chronicles a group of advertising creatives at DraftFCB in their hunt for surprising and ‘sticky’ numbers.

It’s an interesting story mostly because these users (and creators) of numbers are not quants. Instead, they are left-brainers looking to pull out the tastiest nuggets of data fit for mass consumption.

Their chosen numbers are able to tell a story because they are:

1. Simple and easy to understand
2. Human and easily relatable
3. Surprising – capture the gap between intentions and actions

Here are two anecdotes to illustrate how a single number can reveal an insight (and in their case, inspire an ad campaign):

1. For Robert Harris Coffee Roasters: The average coffee drinker spends the equivalent of 11 days a year on coffee breaks. Robert Harris believes this is valuable fresh thinking time.
2. For Dockers: Men’s testosterone levels have dropped 17 percent in the last 20 years. 82 percent of jobs lost in the last year were by men. The new Dockers campaign is a call to inspire the masculinity in all men.

Hopefully I’ll have another post soon with some examples of my own! Also on the agenda of bringing meaning to numbers: we need to find new and creative ways to represent information and data visually.

Measuring harm

Scientists weigh in on the important role of measurement in evaluating BP’s various attempts to plug the oil leak in the Gulf:

It is our view that accurate, continuously updated measurements are not only possible, but absolutely essential if we are to respond effectively to this and future disasters… Without knowing the flow’s true magnitude, how can anyone judge the success of any approach? Without determining how much oil is beneath the ocean’s surface and how much is floating toward land, how can we best direct response efforts?

Between the authors’ remote sensing images of the surface sheen, the dispersal of oil in the water column, and BP’s live video of the leak itself, it seems there can’t be too much data collected.

The soft-spoken annual growth rate

Tell me if this sounds alarming to you:

“The total harvest across species has increased, on average, by about 5% per year over the past 50 years.”

This rate of growth – quoted from a recent paper about tuna overharvesting in Conservation Letters – is equivalent to a 500 percent increase in annual harvest over this time period. The year-to-year change in harvest just doesn’t do justice to the dramatic changes in the scale of these fisheries, not to mention the fact that year-to-year change is not as important as, say, the absolute quanitity of fish harvested and, in turn, the total estimated fish left in the ocean. It seems as if the authors are under-estimating the impact of this fishing.

In terms of communicating the magnitude of the growing human footprint on an important biological resource, the authors could have picked been more dramatic than they chose to be. But maybe they didn’t want to take away from their $3 billion headline.

(I recently summarized the paper here and wrote about the $3 billion opportunity here.)

Discount rates for the future

Most people know about the magic of compound interest and how savings can increase dramatically over time. Well, now put the interest rate in the denominator – now the value of something in the future is discounted to its present value – and you get equally dramatic results:

…the Obama administration’s recommendation of a social cost of carbon based on a 3 to 5 percent discount rate, for an analysis that stretches hundreds of years into the future, puts a surprisingly low value on the next generations’ welfare. Here’s an example: An event with the magnitude of material damages of Katrina (which some estimates put at $300 billion) occurring 500 years from now would be worth just $110,000 today at a 3 percent discount rate, or $8 at a 5 percent discount rate. Personally, I think the future is worth more than that. 

Not surprisingly, the author of this quote favors a discount rate of zero percent. The Stern Review used a 1.4 percent discount rate, the ‘risk-free’ rate is usually around 4 percent, and businesses use anywhere from 7 to 30 percent for their discounting.  

The rationale for using a discount rate is typically two-fold: 1) we’re uncertain about the future, so let’s not forego too much now in preparation for the rainy day that may never come; and, 2) we’ll be richer in the future, so we’ll be able to cover future needs with no problems.

Both arguments may turn out to be true, but the math that gets us to $8 for a Katrina-scale disaster is a little unreasonable. Population growth and the expansion of the human footprint will only worsen the impacts of catastrophic events.

The Obama administration is currently deciding on a social cost of carbon for use in cost-benefit analyses throughout the federal government. In setting the social cost of carbon, I believe they plan to use a 3 to 5 percent discount rate for the social cost of carbon. I think this is a little high but definitely in the ballpark of being reasonable.