Welcome to Research Friday! As part of a continuing weekly series, each Friday we invite a nonprofit expert from our academic faculty to highlight a research report or study and discuss how it can inform and improve day-to-day nonprofit practice. We welcome your comments and feedback.

A couple months ago, the ASU Lodestar Center released its 2010 report on Arizona Giving & Volunteering. The data were collected in the summer of 2009 by asking people to reflect on their volunteering during all of 2008. On one of the pages, amid all the charts on who volunteers and what they do, is a big banner depicting the following result: "33 percent of Arizona adults volunteered in 2008." One in three. The number seems high to some people and low to others. But is it right?

The main point of comparison is information on volunteering from the Current Population Survey (CPS), conducted by the U.S. Census Bureau and the Bureau of Labor Statistics. It provides the basis for regular reporting on volunteerism by the Corporation for National and Community Service. For 2008, the CPS put the Arizona volunteering rate at 25 percent. One in four.

That’s a pretty big difference, between 33 and 25. The ASU Morrison Institute presents both numbers and asserts that both survey methods have strengths and weaknesses, likely leaving the true rate somewhere “between these two estimates.”

That’s probably a fair guess. We might think that figuring out how many people volunteer over the course of a year would be a pretty easy task, but it isn’t. Through the 1980s and 1990s, such outfits as Gallup, Westat, and Princeton Survey Research Associates reported national volunteering estimates north of 40 percent and sometimes higher than 50 percent. Nobody thinks that volunteering behavior has really changed that much over the past couple of decades. Rather, the differences are mostly due to how we collect the information. What we ask and how we ask it makes for a big difference in estimates.

The ASU Lodestar Center study was conducted as a dedicated wave of the Arizona Indicators Panel Study. These data are collected periodically through Knowledge Networks, a reputable firm that specializes in panels of Americans who respond to surveys online. Approximately 1,000 Arizonans sit on their KnowledgePanel, and 687 of them took the time to respond to the Giving and Volunteering Survey questions.

In contrast, the Current Population Survey is a government-sponsored telephone survey that interviewed 57,000 households in 2008. The size and the money behind it make a big difference in generating a sample that the Census Bureau can really stand behind. Though volunteerism questions come late in a very lengthy, multi-purpose survey, hang-ups are fairly rare. The CPS knows who it doesn’t reach and who doesn’t answer certain questions, and makes sophisticated adjustments based on this information. On the other hand, the Arizona residents on the Knowledge Panel could clearly see that the topic of the ASU Lodestar Center survey was “Giving and Volunteering,” and many may have never started the survey when they realized it did not apply to them. This would have the effect of inflating the number of volunteers who opted into the study. So, the size, sponsorship, adjustments, and lower potential for response bias are clear advantages for the Current Population Survey study.

On the other hand, the ASU Lodestar Center study has a couple of other things going for it. One is the use of prompts to help people remember what they may have done over the past year. After all, a year is a long time to remember back on. The CPS takes a pretty simple approach. The first question in the September volunteer supplement asks “Since September 1st of last year, have you done any volunteer activity through or for an organization?” Because there has been little orientation to what is meant by volunteering, it asks those who say “no” one more question: “Sometimes people don’t think of activities they do infrequently or activities they do for children’s schools or youth organizations as volunteer activities. Over the past year, have you done any of these types of activities?” This simple prompt turned 2.4 percent of those who answered “no” to a “yes” response. Prompts really matter.

So, the ASU Lodestar Center survey, modeled on the Indiana University Center on Philanthropy Panel Study, emphasizes several major prompts. First, respondents are lead through a series of questions on giving to specific types of organizations (like religious, education, health, and neighborhood), which serves to orient people to the kinds of formal organizations that both studies are concerned with. Second, the survey includes written prompts of potential volunteer tasks: “Volunteering could include activities such as coaching, helping at school, serving on committees or boards, building and repairing, providing health care or emotional support, delivering food, doing office work, organizing activities, fundraising, and other kinds of work done for no pay.” We believe both lists greatly help to orient and remind respondents of their voluntary action over the past year. We also suspect that the online survey allows people more opportunity for reflection and recall than a live telephone survey does. These prompts and opportunities for recall are clear advantages for the ASU Lodestar Center study.

Another controversial issue to consider is the use of proxy respondents. In the case of the CPS survey, respondents are asked to respond on behalf of others in their household when those others are not available to get on the phone themselves. The typical case is that an adult respondent answers for his or her spouse, as well as children aged 15 or older. Comparisons of proxy with non-proxy responses show that proxies volunteer less, probably meaning that respondents mistakenly err on the side of saying that their spouse and children have not volunteered when they actually have. This has the effect of driving down the volunteerism estimate. The ASU Lodestar Center does not use proxy respondents, only asking the adult respondent (18 and older) to answer for him- or herself. As far as generating a true estimate of volunteerism, this approach is an advantage to the ASU Lodestar Center study.

Really, considering that the two surveys are trying to study the same thing, they are amazingly different. We should not be surprised when they return fairly different results. Which one is right? Well, neither: both have strengths and weaknesses. Which one is better? Well, it is hard to overlook the strengths of the Current Population Survey. However, the ASU Lodestar Center study has its definite strengths as well — which lends credence to the argument that the true estimate lies somewhere squarely between the two estimates.

Mark Hager is the lead author of the ASU Lodestar Center's "2010 Arizona Giving and Volunteering Report," as well as the "2010 Nonprofit Compensation and Benefits Report for Maricopa and Pima County Organizations."

• Research Friday: "Is Religious Giving Increasing or Decreasing?"
• Research Friday: State of the Sector 2011: All quiet on the nonprofit front