US demographics are changing, so an organization welcoming more diverse audiences than it did a decade ago is an expectation, not necessarily an achievement.

A goal is to welcome diverse audiences at a representative rate relative to the population.

The Metropolitan Museum of Art’s percentage of non-white, American visitors has increased from 21% in year 2007 to 27% in year 2017. (Growth!) Some other organizations might (and do) see similar growth and tout it to reporters as a major success. Not the Met. As the CEO, Daniel Weiss, said eloquently on a recent segment on Marketplace on NPR, “The ultimate goal is that the people inside of the building reflect the people outside of the building.”

I’m also interviewed in the segment. In an unaired portion of the communication, the reporter wanted to know if this growth at the Met was on par with other museums. The answer? It’s on par.

IMPACTS data suggests that the Met’s 28.6% observed increase in non-white visitors (i.e. from 21% to 27%) is comparable with reporting from other organizations – it is neither appreciably “better,” nor meaningfully “worse.” And a significant portion of this increase is likely attributable to the increasing diversification of the overall population.

But this article is not about the Metropolitan Museum of Art.

In general, there is increasingly diverse participation at most visitor-serving cultural organizations (i.e. museums, zoos, aquariums, historic sites, performing arts, etc.), but that doesn’t necessarily mean that inclusion programs are working.

Cultural organizations are not successfully reaching new visitors.

Not yet, at least.

Moving the “diverse audience” needle is a big deal – and an extremely important one for the long-term sustainability of cultural organizations. Articles, initiatives, papers, and conference presentations coming from these organizations suggest that cultural entities are hard at work trying to move this needle!

Here’s a problem, though: Making diverse audiences into regular attendees is an ultra- marathon, not a 5k. There seem to be a number of organizations claiming to reporters or board members that they’ve already run the race in record speed or are ahead of the pack, when actual data suggest that they have barely moved from the starting line.

Here’s why some organizations unknowingly believe that they are doing a data-informed great job at engaging diverse audiences…when this is not necessarily true.

These problems may be related to neglecting to put data into context, thus skewing the key takeaways.

As we often say at IMPACTS, “Uninterpreted data is misinterpreted data.”

1) Organizations tend not to adjust their findings for population growth and demographic evolution.

While there is some increasing non-white participation at most visitor-serving organizations, a large part of this observed increase in diversity likely relates to evolving demographic trends. For example, the percentage of the US white non-Hispanic population has been in steady decline since year 2000.

In year 2007, non-white persons comprised 34% of the population. In year 2016 (the last available year for national reporting), non-white persons comprised 39% of the population. This trend toward diversification is expected to continue in the coming years.

Let’s go back to the example of the Met to illustrate what this means for museums: The Met increased its non-white participation by 28.6% from year 2007 through year 2017. (Excellent!) However, the overall US population became more diverse (i.e. non-white) by 14.7% during approximately the same duration. This means that in year 2007, the Met underserved the representative non-white audience by a factor of 1.62 (34% of the US population vs. 21% of visitation). More recently, the Met underserved the representative non-white audience by a factor of 1.44 (39% of the US population, 27% of visitation). So, indeed, while the Met’s numbers indicate improved engagement of non-white audiences, they fall short of being representative of the overall population…and the majority of the improvement may be attributable to evolving demographic trends.

As stated by the CEO himself, the Met is not ignoring the makeup of the US population when interpreting its own numbers. And again, the Met’s growth in diversity (or lack thereof) is generally in-line with what we are seeing among visitor-serving organizations overall. The Met is a market leader and (as is the burden of a marker leader) it makes for an excellent example to illustrate the point.

Unlike the Met, however, recent reporting coming from cultural leaders may suggest that other organizations are far less aware of the context of their numbers.

When organizations do not consider the changing demographics of the US population alongside those of their attendees, they risk thinking that any increase in diverse audiences represents a job well done.

In reality, an increase in reaching non-white audiences but still not engaging diverse audiences at representative levels is arguably as much a cause for concern as celebration.

The US population has increased meaningfully in non-white persons in the last ten years. Thus, a certain increase in audience diversity is expected. If we didn’t reach any additional diverse audiences, then, well, we’d have much bigger problems than those facing the industry right now.

2) Survey mechanisms have generally evolved to be more inclusive, skewing early numbers and making growth look more impressive.

Another factor worth contemplating is the means and methods by which audience research has been conducted over the years. Data is generally more of a thing now than it was ten years ago, and methods for representatively surveying audiences have improved dramatically.

Unless your organization has not evolved its methods at all in the last ten years to better adapt to reach ESL audiences, your numbers may be showing inflated growth.

For example, in year 2007, few organizations regularly conducted audience research so as to reliably contemplate English as Second Language (ESL) visitors. This may suppress the research response rates from non-English speaking visitors. It is a fair question for cultural organizations comparing years 2007 and 2017 (or any other timeframe) to ask if diverse participation has actually increased…or if an entity has become more adept at soliciting research feedback from ESL households.

This is even true at IMPACTS! Since year 2011, IMPACTS has deployed audience research in multiple languages so as to be more contemplative and inclusive of ESL households. This methodological improvement resulted in increased participation rates for non-white audiences (who were more likely to come from ESL households). IMPACTS is a company with 87 FTEs, the overwhelming majority of whom obsessively work with data. My colleagues live and breathe data collection, analysis, and consumer psychology. Many of these people have doctorates in this stuff, and I’m totally not one of them. We at IMPACTS, even, are better now at reaching ESL folks than we were in the past because being better at reaching them – and doing so representatively – is the reality of the world we live in today. It’s critical.

But the world was only starting to move this way around 2007. Also, in general, organizations didn’t care as much about data back then as they do today (and that is an ongoing struggle). Capturing these audiences at representative rates often requires both meaningful funding and a concerted, prioritized effort. For many, that may be added effort, expertise, and funding than were allocated to the cause ten years ago.

I bring this up because I am guessing that there are some folks reading this and thinking, “Oh, but we prioritized data and put adequate additional funding into capturing representative ESL visitors in 2007…because we had Karen! Karen spoke Spanish and helped with our education programs, too.”

Shout out to hypothetical 2007 Karen! But hopefully today you have a bit more dedicated, concentrated, and trained expertise on the scene. Hopefully 2018 Karen isn’t carrying this alone. And perhaps some organizations really were far ahead of this curve and had survey methods that were just as advanced ten years ago as they are today! For the rest, it’s helpful to keep in mind that changing methodologies can skew the data.

These considerations do not negate the data that an organization has collected! They simply add important context for data interpretation. Making sure that organizations are representative in their data collection is a big deal. If your organization was similarly and equally representative in 2007 as it is in 2017, power to your institution!

I know, I know. It’s a bummer when data that makes us look good needs to have its context adjusted in order to be interpreted accurately over time. But we probably don’t get better at things by misleading ourselves about our successes. At the end of the day, this may be good news for organizations: Data collection methodologies are evolving to be smarter and more inclusive. That’s great!

While cultural organizations have a ways to go to engage more diverse audiences and – specifically – to be seen as welcoming for these folks, there’s no doubt that this important conversation is taking place.

It can be tempting to see increasing numbers in the data and celebrate them without taking a moment to dig deeper. That said, misinterpreted data isn’t only unhelpful. It can be harmful. This may especially be the case when the misinterpreted data relates to something as important to the long-term survival of cultural organizations as diversity and inclusion.

Ours is an industry needing to keep up with the Joneses and evolve from an “I think” industry into an “I know” industry. Evolving to data-driven from opinion-driven doesn’t mean that strategic design should be replaced by computers! It means learning how to leverage data and fact-based realities as tools to aid the human beings who make strategic decisions.

Data is a necessary tool for success, but it need not – and should not – fully replace the thoughtful work of a talented executive’s brain.

Data may need more power within our industry – but it also needs context.

The post Why Some Cultural Organizations Overestimate Success In Welcoming Diverse Visitors appeared first on Colleen Dilenschneider.