On 3/15/2015, the Pittsburgh Post-Gazette published an article by Elizabeth Bloom reporting on a $100,000 marketing study that uncovered audience development woes at the Pittsburgh Symphony Orchestra (PSO). On 3/19/2015, the chairs from the orchestra’s board and musicians’ committee (Dick Simmons and Micah Howard, respectively) co-authored an op-ed piece that takes issue with the article and, rather surprisingly, their own marketing study.
If you find yourself reading through the post a few times, don’t be alarmed as it is just about the most puzzling response to public criticism I’ve seen in the past 20 years. Granted, the colloquialism “shooting the messenger” exists for good reason but the despatcher isn’t supposed to be the one holding the gun.
To that end, here are some curiosities that stand out:
- The PSO reportedly spent $100,000 on the marketing study then spent most of their op-ed poking holes in the study’s methodology and results.
- Simmonds and Howard criticize the report’s failure to form focus groups outside of non-ticket buyers comprised of their predominant ticket buying demographic.
This is a worthwhile complaint but again, this was the PSO’s study and it would be interesting to know how and/or why they were so uninformed about the research firm’s process. Moreover, having the client and provider work together when crafting parameters for elements such as focus groups isn’t unusual; and like all good third party projects, both sides draw from the other’s expertise and understanding to identify considerations and refine methods in order to maximize results. - Simmonds and Howard never really address the PSO’s grave ticket revenue and paid attendance issues referenced in the report beyond referencing gains in contributed revenue; but even then, no hard figures are offered.
Ultimately, it is difficult to ignore the 800-pound gorilla that is the $100,000 they spent on a study they are now publicly discrediting. That act doesn’t exactly lend itself to developing a great deal of overall governance credibility.
It is one thing if the report generated results the PSO simply wasn’t expecting or found uncomfortable but were the results from a solid research and analysis process. But based on the information released in the 3/15/2015 Post-Gazette article, it seems clear that the process was far from ideal and fell equally short of being capable of producing useful outcomes.
If nothing else, it will be interesting to see if the report’s firm will fire back since it isn’t unusual for work agreements to include one or more clauses about how both parties deal with disputes.
In the meantime, the PSO has yet to reply to my initial inquiry about whether or not the study examined the PSO’s box office capacity; specifically, whether or not they have the ability to sell tickets the way we examined in an article here from 1/30/2015 titled To Subscribe, Or Not To Subscribe, That Is NOT The Question. I’ll post a follow-up article when/if they do.
Well, yes, it should be very worrying that the chairs from the orchestra’s board and musicians’ committee are now publicly discrediting a study the orchestra itself commissioned.
If, that is, you think that the two are writing sincerely and really believe the study is bogus – rather than that they are simply trying to cover their (and the orchestra’s) asses.
Seems to me more likely that, once the article about the study ran in the Post-Gazette, some folks on the board basically started saying Aaaaaack!! What are you doing?? You’re telling the whole world that nobody wants our product!! Aaaaaack!! Do something!! Aaaaaack!!!!!
And the op-ed that the two chairs wrote is the ass-covering they threw together on short notice to shut those folks up.
Ah well…not all $100,000 studies prove actually useful. But you often don’t find out until the report is delivered, unless you monitor the survey process and assumptions behind it. The PSO would do better by reading Holly’s blog for free. In one post I found her notion that attracting new audiences is sometimes hampered by your existing audience profound. (All that “schussing” at the end of Tchaikovsky 6th third movement comes to mind.)
…and for those readers unfamiliar with the reference material in Chris’ comment, Holly’s blog is Neo Classical, http://www.insidethearts.com/neoclassical
“The portion of the W-5 study from which many of the quotes were taken surveyed only a very narrow slice of our community: a small group of residents who do not go to symphony concerts but whose demographics are similar to those who do. It does not — and was not intended to — reflect the views of our audience or the community as a whole.” But this is the purpose of doing focus groups of non-attendees…to see what the ‘community’ of those not attending think about your organization. These are still valid results. While I agree they aren’t representative of the current audience, these small focus groups are intended to get feedback from the community. The results can’t be discounted. I can see how, however, they find it unfortunate that the Post-Gazette chose to zero in on those particular statements! Maybe it’s best if they just publish the results of the research themselves on their website.
It sounds to me like the PSO op-ed isn’t upset so much at the study – but at the way it was presented by the Post-Gazette – which I would agree with. The PSO was simply pointing out the intricacies that are involved with any small research study. When you’re only looking at a focus group of 45 participants, their few quotes bound to be are taken out of context and not necessarily representative of a large majority. Was the op-ed the best way to get at that point? Maybe not, but it doesn’t mean it’s incorrect either.
I agree with Martha and Samantha: the op-ed piece is not so much criticizing the survey as it is protesting the way it was covered in the newspaper. As all of us in this business know, “if it bleeds, it leads.”
That was one of the scenarios I considered during the re-reading process but if that were the case, the PSO would have pulled data from the report to support any notion of a deliberate misrepresentation. Nonetheless, the op-ed didn’t include anything along those lines.
Moreover, I’m not entirely certain the original Post-Gazette report was misrepresenting the report. The author presented the study’s data and parameters on its focus group and the op-ed seems to confirm exactly what they reported.
The paper contacted the PSO so unless they deliberately edited out any material that would have altered the tone of the article, there isn’t any indication of crafting a hack job.
Ultimately, if the study included information that was intentionally left out of the initial Post-Gazette article for the purpose of sensationalism, one would think that the PSO would have pointed that out in the op-ed.