Illustration of writing an article on a laptop

ASU Lodestar Center Blog

Telling the story in a new way: Should arts leaders use impact evaluation?


To borrow from the song: “Art is a many splendored thing.” Difficult to objectify and quantify. Art is also very subjective. Again borrowing: “One man’s trash is another man’s art.” So, what is the value of art?

Recently, I attended a conference on arts education. At the plenary panel discussion, a woman from the audience, an arts teacher, asked, plaintively, “Why do we have to justify the arts in school? Math doesn’t have to be justified. Science doesn’t.” No one on the panel had a decent answer for her. Her question stayed with me for a long time. I think we have been telling the wrong story. Or more accurately, we have been telling the story wrong.

Impact evaluation in the arts, and its broader use for leaders of any nonprofit, can drive results. Qualitative yet empirically-based impact evaluation bridges the gap left by other evaluative methods providing the context of mission fulfillment for a nonprofit organization. Armed with such data describing the value of the arts for its participants, arts leaders can change the perception that art is merely a luxury to show that, instead, it is a vital necessity to human beings. Only within the last decade has research on the efficacy of evaluating the effects of art on audiences been realized (Brown & Novak, 2013). 

Telling the wrong story

For the past forty or so years, to ‘prove’ the value of the arts, arts leaders, funders and subsequently policy makers have focused on measuring the instrumental effects of art on society and the economy, neglecting the intrinsic value of art (McCarthy, Ondaatje, Zakaras, & Brooks, 2004). These instrumental effects are such quantifiable elements as: number of participants, academic success, economic impacts, general demographics of participants, etc. Intrinsic impact refers to those benefits of the arts that enrich or transform participants (McCarthy et al., 2004). In today’s climate of examination and justification of the nonprofit sector as a whole, isn’t it essential for leaders in the arts to embrace this kind of evaluative methodology to empirically quantify the qualitative value of art? With such evaluation data, arts leaders can promulgate the value of the organization’s art to all stakeholders.

Using different terms to mean the same thing confuses the story

Impact, outcome, performance assessment are often all used to mean the same thing. Adding to the confusion of terminology, we, in the arts, use terms like participation or engagement to also mean impact. But essentially these terms are reaching towards the same end: understanding the value of your art to its intended audience. We all have very lofty words in our mission statements, but how do you know your work is being “provocative”, “challenging” or “innovative”? How do you measure that?

The knight on a white horse arrives

In 2004, Gifts of the Muse: Reframing the debate about the benefits of the arts arrived. In it, Kevin McCarthy called for a new way for the arts to describe itself by evaluating the “intrinsic impacts” of art. In the intervening years, several research efforts (Brown & Novak, 2007; Radbourne, Glow & Johanson, 2010; Brown & Ratzkin 2012) proved it was possible to quantifiably measure those intrinsic impacts of art.

The ogre under the bridge isn’t as scary as you might think

As artists, we can be very insecure about our work. After all, our art is a part of our soul put on display for all the world to see. Many arts leaders are reticent about any prospect of their audience being allowed to judge or even criticize their work. They may be willing to stand in the lobby and chat informally with audience members after the show, but to survey an entire audience to measure how impactful the work is can be anathema to them. However, the studies in measuring intrinsic impact were intended for “internal assessment and critical reflection purposes” (Brown & Novak, 2013). They exist not to be able to say, “My art is better than your art,” but to say, “My art impacted its audience in these intrinsic ways, let me show you a data chart on that.”

And they all lived happily ever after

After decades of economic impact studies, academic success studies, counting the number of tickets sold, and maybe a handful of anecdotal stories, we are still left with shrinking government funding, declining attendance, and an inability to accurately describe the real value of art in a meaningful way for policy makers, government officials, donors, artists, organizations, and participants. That is a wide continuum of stakeholders each with their own agenda and interests. Evaluation can be painful. It’s expensive, time-consuming and a downright burden. Added to that, intrinsic impact evaluation is new, not well understood and a little scary—just like art. We, in the arts, must tell our story in a new way. Yes, unfortunately, we have to justify art when math and science don’t. We have to embrace the intrinsic impact of our work and find a way to express it. Our very souls depend on it.

 

Sources:

Brown, A. S. & Novak, J. L. (July, 2013). Measuring the intrinsic impacts of arts attendance. Cultural Trends Vol. 22 , Iss. 3-4, Ppg 223-233.

Brown, A. S., & Novak, J. L. (2007). Assessing the intrinsic impacts of a live performance. WolfBrown. Retrieved from: <">http://www.wolfbrown.com/mups_downloads/Impact_Study_Final_Version_full....

Brown, A. S., & Ratzkin, R. (2012). Understanding the intrinsic impact of live theatre: Patterns of audience feedback across 18 theatres and 58 productions. In C. Lord (Ed.), Counting new beans: Intrinsic impact and the value of art (pp. 65–164). San Francisco, CA: Theatre Bay Area.

McCarthy, K. F., Ondaatje, E. H., Zakaras, L., & Brooks, A. (2004). Gifts of the muse: Reframing the debate about the benefits of the arts. Santa Monica, CA: RAND Research in the Arts. Retrieved from <">http://www.rand.org/content/dam/rand/pubs/monographs/2005/RAND_MG218.pdf>

Radbourne, J., Glow, H. and Johanson, K. (2010). Measuring the intrinsic benefits of arts attendance. Cultural Trends, 19: 4, 307 — 324.

 

Christopher Haines is the founder and Artistic Director of iTheatre Collaborative, a 501(c)(3) nonprofit theater company.  Under his leadership, iTheatre has performed across the country, as well as across the Phoenix area. Mr. Haines received his BA from Duke University (’91) and his Master’s in Nonprofit Leadership & Management from Arizona State University (’16).


Tags

ASU Lodestar Center Blog