At 63 I have been in mall based data collection for 35 years with offices all over the country in which I have spent plenty of time rubbing elbows with shoppers we recruited into our offices. I am decidedly old school when it comes to the thought that this is a people business and since we are a service business, the way we deliver our service to the people we interact with has a lot to do with their satisfaction.

To that end, we spend a lot of money building professional looking storefronts that become our brand as shoppers pass by. The offices are large by mall office standards, well appointed and all designated reception and interviewing areas are large enough and well enough furnished to be comfortable.

All our interviewers dress professionally (a moving target in our casual world), wear name tags, and have a long list of rules for conduct in the mall and in our office before and after the interview.

We observe all client requirements regarding past participation, and other generally recognized standards for study inclusion. We make every attempt to move respondents through the interview process as quickly as we can while properly administering the questionnaire.

So, in the day-to-day world, we are doing everything we can that is within our power to put our best foot forward and represent the industry to the best of our ability. But not all of my colleagues are making the same choices.

Emiel mentioned Offline being just too expensive, but that Online has over promised or been under controlled by end users and full service research suppliers who hire these services. Whichever is correct, the pricing pressure placed upon mall based interviewing services has forced some operators to cut costs by not doing the things we believe are important-professional looking offices that are clean, professional conduct by interviewers, proper study administration. These operations undercut our industry, and I’m afraid many clients look the other way as long as these interviewing services can produce interviews at a low price and reasonably close to deadline.

MRA has never stopped talking about Quality in my 35 years. The problem is that Quality means different things to different people and I dare say, there is probably not a single MRA member who does not believe they do quality work.

We have had ongoing efforts at educating respondents since Frank Walker coined the Your Opinion Counts slogan in the late 60’s/early 70’s. I’m proud to say that my mother Pat Heakin was the first Chair of the National Marketing Research Week campaign launched in 1982 which was the first time all our major research associations/organizations came together on a common cause.

Thanks to organizations like the Burke Institute, MRII, the cooperative education courses offered by institutions like the University of Georgia, and now the MRA’s PRC Program which I’m proud to say we helped get started, there is no lack of educational opportunities for research professionals to develop their skills and careers.

And with regard to public relations, the research industry has sponsored other efforts like CMOR. So I do not believe the problem lies in education.

The problem is our commitment to Quality, and our willingness as an industry to pay for Quality, and to do the right thing by demanding facility operators to run professional facilities and stop supporting those who don’t simply because the offer huge discounts. How, pray tell, do you think they are able to affect these cost savings when other operators are not able to?

Likewise with Online. These people need to be subjected to the same level of scrutiny that we Offline providers are. Everyday, we go through past participation and critical industry screens, are validated by our clients and our respondents are filtered by Sigma. In addition to having to satisfy all requirements, clients are timing our Internet interviews and evaluating open ends to see if they believe everything is on the up and up. But my best friend, who belongs to one of the best known national Online panels tells me he is solicited to do an interview every single week.

A couple of years back, a respected Field Director from a Honomichl 50 research supplier told a MRA audience that “you cannot validate Online as it is an invasion of privacy.” As long as there is no one guarding the hen house, I’m afraid there is no hope that the abuses that are being created by the need to be ever cheaper will soon end.

What is needed is that end users and research suppliers need to reward the small and mid-size interviewing services who compete based on Quality and service, rather than price as seems to be the preferred method of the larger companies.

Finally, I do not read a single article in the MRA Alert that does not hype technology as the answer to every problem and the only hope for the future of research. Technology is only a tool. When placed in the hands of people who are not dedicated to Quality and service, we will get the same results we always have in the past and the problems we have will not be solved.

John Heakin, President
North American Insights

I echo your comments - they are right on target. The industry will clearly make much more progress working together.

Gregg,

Thanks for your nice comments and the presentation. Client education could help alot. I don’t believe it is the end all but a nice start fro sure. I would love to hear from some clients and associations. I hope they find the time to add their thoughts.

Merrill

In my opinion any effort that will succeed in turning it around must include all sectors of our industry. Clients, Research Companies and Data Collectors must band together.

What needs to be done ? First it’s important that we acknowledge the severity of the problem and clearly address the causes of the problem. Long and tedious questionnaires, interviewing at intrusive times and venues are just some of the causes. I believe that educating the public about the meaningfulness of Research is critical and has to be part of the process. However , this will not be accomplished on a shoe string budget. It will take major dollars and the involvement of influential people in our industry working on it..

Someone should step up and recruit members from all sectors to make a major commitment of time and dollars to address the issue. All of these forums are not getting it done. There needs to be a Champion.

I would be happy to volunteer some of my time to explore ways to move the marble a lot faster than it has been moved.

Joe

I do agree that we must always strive to improve data quality, shorten our lengths, be more engaging and respondent friendly, and improve response rates. This effort is not method or mode related… it should be part of everything we do.

Many of us with ample gray hair grew up in this industry learning that all data collection methods have their place, along with their inherent limitations and biases. As I’ve said in previous industry presentations.. the principles of statistics are akin to the laws of physics, and cannot be waved away because they have become more expensive, time-consuming, or simply too hard.

The Project is the key level of evaluation, in my opinion. Every need for data exists on a continuum of cost, time, allowable variation and bias.. that springs from the business need, the nature of the decision to be made, and the acceptability of risk.

I believe, however… we have, in many cases, lost our project level focus and adopted a business-model based focus …. understandable given the tremendous investment required to build our various data collection infrastructures … along with the resulting need for an ROI. When you only have a hammer, every problem is a nail…

Frankly, this gives me doubts about industry level solutions to the “data quality problem”… because we are, afterall, a collection of different types of players … and the business-model fragmentation is exceedingly difficult to overcome. In many of the DQ discussions, I tend to hear more defensiveness than intraspection…

I think that Client education is the key, even if the news isn’t always better-cheaper-faster … and I’m encouraged of late to see more of them approach us with “a study”… rather than a phone, or web, or mail, or whatever project. While we should always strive to improve our quality… all of our tools have their limits.. and the client’s needs should be paramount in selecting the proper one(s).

You asked the right questions at the CASRO Data collection conference and there was a lively, healthy and constructive exchange from the participants. I think it needed to be done and opened up some frustrations and issues that we’ve all been thinking about.

To me one of the key take-aways from that event was that the sponsors of research and the providers of it attend different conferences. It is not just that we’re talking past each other, we’re talking among ourselves as two independent groups! We can’t address the broader issues that way. My sense is that we’re pushing the issues back and forth and it will be really hard to make any substantial progess with this approach.

I would like to see the professional associations organize an event that getsboth sponsors and providers involved. Not in an adversarial or confrontational sort of way –but in a constructive way that reflects the involvement everyone needs to have in order to make a difference.

In the meantime, I made a comment at the conference I stand by. “let no challenge go unanswered”. The next time someone asks you to be involved in something that doesn’t make sense from the perspective of quality outcome…..engage them in a dialog, educate them with whatever you know about the topic that causes you to have reservations and work with them to create change…..one survey at a time. I see some of these
posts making the types of suggestions and sharing the kinds of information that can help us make a difference.

First, regarding education. This is a great idea, but one that is simply impractical. We can educate people all we want about the importance of research, but that is not going to make people less bored with long surveys.

It strikes me as very odd in the day of YouTube, LinkedIn, MySpace, FaceBook, and the hundreds of thousands of blogs, that professional market researchers are having trouble finding people to give their opinions. Consumers do not need incentives to be encouraged to tell us what they think. They are more than willing. We just have to find ways to meet them half way.

I designed an online customer feedback system that has gotten a 70% response rate. In designing the approach to sampling I went back to R.A. Fisher’s 1920s papers on sampling theory. The first question I asked was, “Why do we sample the way we do?” The answer, from the ’20s, was that we sample because we cannot practically ask everyone to tell us what they think. Imagine the phone costs or the labor of personally interviewing every customer.

But, online you can ask everyone what they think. The limiting factor is their willingness to take the time to respond. We have found that if you ask only a few questions, say, three at a time, most people have no problem taking a few seconds to reply. But, you have to place the survey somewhere that people are already interacting. You can’t just put up a blind link or send an email invitation. They have to actually see the questions.

We developed a process we call ASK3 that takes a full length survey and rotates through the questions three at a time. The process is similar in some ways to Conjoint, but actually much simpler. Questions are grouped into three classes and combined so that key crosstabs and correlations can be conducted.

When we placed ASK3 onto a client online checkout page, over 70% of their customers provided 3 answers without a single negative comment.

ASK3 methodology isn’t a panacea, but it is the kind of solution that we need to be thinking about. The statistical underpinnings are sound (in my opinion), even if it does appear on the surface to violate some of our traditional assumptions.

There are plenty of ideas out there. The question is whether the MR industry is ready to take a serious look at them. So far, most of what we have done is simply to take phone survey methods and put them online. That has served us for a decade, but it won’t hold up in the long run.

If anyone is interested in discussing these methods in greater detail, please let me know.

Steve at SchwartzResearch.com

Thanks for your comments. At the CASRO presentation we discussed having some best practices and actually one idea we kicked around was having 4-5 slides that we could all agree on to include in our company capability presentation to clients. the thought was that if we had one message that included best practices we could make a little progress.

Thanks for contributing.

Merrill

I agree with alot of your comments. My belief is we have a tremendous amount of intelligence in this industry. These same people are very passionate and can be very solutions-oriented. The client summit that Bob Lederer had a few months ago ONLY included clients, the CASRO Data Collection conference a few weeks ago ONLY included suppliers and data collection companies. UNTIL EVERYONE IS TALKING AND IN THE SAME ROOM PROGRESS WILL NOT BE AT THE SPEED THAT THE INDUSTRY NEEDS.

I am sure that the client summit was great as was the CASRO session BUT wouldn’t it had been better if there was a task force made up of 4-5 leaders from suppliers in the room with them and wouldn’t it been fantastic if there were a client task force that went to CASRO? I say yes - more progress would be made as opposed to having barriers. I would also go as far as saying let the task force go to the events for free.

This is just one thought., I hope we can continue to hear from many more people.

Merrill

Thanks for your comments. Sounds like a very interesting paper - I will review it.

Merrill