The Hill
Tuesday, December 02, 2008
SEARCH
Home
HillTube
Mobile
White Papers Portal
New Member Guide
BLOGS
Pundits Blog
Congress Blog
Blog Briefing Room
NEWS
Leading The News
Business & Lobbying
K Street Insiders
John Breaux
John Engler
Vin Weber
Dave Wenhold
The Executive
Campaign 2008
Endorsements '08
COLUMNISTS
Dick Morris
A.B. Stoddard
Brent Budowsky
Ben Goddard
David Hill
David Keene
Josh Marshall
Mark Mellman
Jim Mills
Markos Moulitsas (Kos)
Byron York
COMMENT
Editorial
Letters
Op-eds
Weyant's World
CAPITAL LIVING
Today's Stories
50 Most Beautiful 2008
Other Features
In The Know
Bookshelf
Food & Drink
Onward and Upward
Hillscape
RESOURCES
Classifieds
Subscribe
Order Reprints
Last Six Issues
Useful Links
RSS


Home arrow David Hill arrow Mellman should refocus
David Hill PDF Print E-mail
Mellman should refocus
Posted: 09/27/06 12:00 AM [ET]

Mark Mellman, my Democrat pollster foil here in the pages of The Hill, is an accomplished and deservedly respected researcher. But he took a wrong turn last week in attacking focus groups. Recall his statements about the use of focus groups for testing of political TV ads:

“Focus groups are about the worst technique ever devised for this purpose.”

“People are very poor reporters of their own decision making process.”

“People’s well-documented tendency to conform to group pressure also threatens the validity of focus group results.  . . . We see the impact of conformity in focus groups all the time.”

Re-reading his remarks, I came to the conclusion that it’s not focus groups that Mark Mellman loathes; it’s the focus group participants that bug him. Somehow voters aren’t insightful enough to test the very ads that are designed to influence them.

Anyone who has moderated or observed a focus group sympathizes with Melman’s thinking. Sometimes the people who participate in groups seem to be the moral equivalent of down-and-out drifters who donate blood for money. “Here, take a pint and give me $75.” They force us to reexamine whether we want their opinions, much less their plasma.

Yet there are ways to have better focus groups that get around issues raised by Mark. Political focus groups can be done better.

First, we can do a better job recruiting focus group participants. Rather than simply being satisfied with a panel of dull participants that roughly conforms to the demographic profile of the target market, do more. Also screen participants in advance for their conversation and analytical skills. Few political researchers do this, but it’s not uncommon for skilled market researchers to do so.

Second, we can moderate better. Good moderators can ask probing questions that reach deeply into the values and beliefs that underlie and give form to opinions about a particular ad. Rather than simply asking a person to rate an ad, try to go beneath the surface of their first (or even last) opinion about an ad to see what values are driving their ratings.

Third, we can use questioning techniques that get around the self-reporting problems Mark makes reference to. Voters are sometimes incapable of discussing their own views, but I have found that they are more articulate when I ask them how their neighbor will respond to an ad. Once they get clear of the fear of revealing too much about themselves, they will dig deep to tell you about opinions that are likely to surround a particular ad.

What troubled me most about Mark’s attack on focus groups is his denigration of the effects of conformity. He’s doubtless seen focus groups where voters start agreeing with the more vocal or charismatic participants in the group. Well, instead of seeing this as a weakness of focus groups, it must be recognized as a strength. Public opinion formation is a social process. We do talk about ads over the water cooler with co-workers or over the backyard fence with neighbors. These conversations influence our opinions. It’s inconvenient for the researcher that assumes that all opinion formation is private, but it’s reality. If voters care so little about their opinion of an ad that they will change their views to conform to those of others, shouldn’t we want to know that?

Mark insinuates that there are better ways to test ads, but he fails to mention any weaknesses of the alternatives. I suppose he’s recommending Internet-based testing, where broadband PC users sit all alone at their desks and “rate” ads they see in a small 2 or 3 inch window. This is altogether better? So much better that we should dismiss focus groups to the trash bin? Let’s not we jump to rash conclusions.

Hill is director of Hill Research Consultants, a Texas-based firm that has polled for GOP candidates and causes since 1988.

 
 
 
BLOGS
ADVERTISER
Home | Privacy Policy | Terms And Conditions
The Hill
1625 K Street, NW Suite 900
Washington, DC 20006
202-628-8500 tel | 202-628-8503 fax

The contents of this site are © 2008 Capitol Hill Publishing Corp., a subsidiary of News Communications, Inc.