M/A/R/C Research
Search the blog:

Client Point of View

We all know that clients are very important. In some ways they set the stage for the direction of the research industry and whether the latest methodology is accepted. I really enjoy meeting and speaking with clients. It is probably the best part of my job. The seven clients that contributed to this blog all have different backgrounds and represent different industries. What they all have in common is they are professional researchers, have been in the industry for many years and represent large research budgets.

Please sit back and enjoy getting to know seven special people:

What is your most memorable research project you have ever worked on?

Jon Last – Vice President, Corporate Marketing, Research and Brand Development – Golf Digest
(To see all of Jon Last’s responses, click here)

What’s that adage about always remembering your first time? My first exposure to MR was where we conducted a study of top level executives for a conference center client about the effectiveness of meetings. The goal of the project was to generate publicity for the client, and the "shout it from the rooftops" headline that we derived was that these execs felt that over half the time spent in meetings was wasted. We quantified the dollar value that was wasted and garnered incredible amounts of press for our client. It was this project that flicked on the lightbulb for me that marketing research could be incredibly powerful on multiple levels.

On a lighter note, I will never forget some focus groups we were doing when I was in the cruise industry. On a check-in, I went to the back room to find our CEO, incredibly enthused by the input of a particular respondent. He begged me to let him come into the focus room and address the group. I tactfully explained why this was a bad idea and then pulled the rest of my team aside, encouraging them to watch him and hold him at bay. Of course, it was to no avail. Just as I was wrapping up the group, he came bursting into the room and started "selling".

Regina Lewis, PhD VP, Consumer Insights InterContinental Hotels Group
(To see all of Regina Lewis’ responses, click here)

1997 extensive in-home ethnographic study of how women were using the internet. I did this in conjunction with P&G, and it really changed the way we all (I was at women.com Networks at the time) viewed chat, info seeking, and much more!

Jill Donahue, Brand Insights Manager Nestle Purina
(To see all of Jill Donahue’s responses, click here)

I was working for a footwear retailer and we were conducting ethnographies to see how woman organize (or not) their shoes. One woman not only had a separate closet for her shoes, she had religated her husband to the basement storage area for his everyday clothes. Amazing!

Dan Womack, Manager InsightsAflac
(To see all of Dan Womack’s responses, click here)

For me, there is one project I’ll never forget and the reason has nothing to do with the research I was conducting. On the evening of September 10, 2001, I flew to Austin, TX and prepared to moderate focus groups early the next morning. That day, a few of the last respondents to arrive for the first group were talking about hearing vague reports of a plane crashing into a building in NY City. That was all any of us knew until after the group was finished.

My second group of the day was mid-morning. The show rate was surprisingly good, but the research wasn’t. We spent very little time discussing the products I came to study. Instead, we used the time to discuss something we all knew was far more important.

I spent much of the next two days waiting, worrying and talking with the two flight crews that were also stranded in the same hotel. I was fortunate to have a rental car and, on the third day, I decided to drive the 1500 miles back to Virginia. I’ve never loved business travel, but I have never been happier to return safely from a trip and hug my wife and kids.

Jami Guthrie, Senior Director, Global Consumer & Market Intelligence Wm. Wrigley Jr. Company
(To see all of Jami Guthrie’s responses, click here)

It was a segmentation study in an adjacent category that we (as a company) knew very little about. The reason that it is my most memorable is because the learning was completely new and so insightful. It allowed us to look at our brands and the opportunities through a completely different lens. It was great to see how engaged everyone in the company was with the learning and more importantly, how quickly we took action against it.

Bill Tanner Director, Strategic Research & Audience Development Dallas Morning News
(To see all of Bill Tanner’s responses, click here)

I was in Hangzhou China, in January, many years ago, doing parent- child dyads in a school with a TV monitor set up. A window was broken in the observation room, a cold winter wind blowing into the room. We were bundled up as best we could but it was impossible to stay warm. The monitor flickered and there was a military training event going on outside making it hard to hear. I asked that we move to the hotel the next day where we set up in an empty dining room. Unfortunately, the next day’s sessions at the hotel were stilted. Parents did not open up as much as they had in earlier sessions. It turned out that the hotel once was a major Communist Party Leadership hangout and still a place often visited by the hierarchy, so the next day, it was back to school. Fortunately, the window and monitor were repaired.

Bob Graham, Consumer Research Manager Energizer
(To see all of Bob Graham’s responses, click here)

Hard to say. At Micrografx, I did an international project with focus groups in London, Paris and Munich. It was fascinating to observe the cultural differences among respondents in different countries. I’m prouder of some of the other projects I’ve worked on (where I used a new method or analytical tool), but I think the international one was the most memorable.

How would you categorize the state of online research?

Jon Last

It has become "the great equalizer"…both friend and foe. For those that maintain sound methodology, sampling discipline and a focus on action generating analysis, it can bring speed and cost efficiency to a client and enable you to accomplish more with less, thus growing the stature of MR within the organization. However, there are so many possibilities for abuse and misinformation, that it’s incumbent upon both suppliers and clients to know what they are doing. We all hear the cries and questions about data and respondent quality. I believe that these concerns have legitimacy, though I’m fortunate in that we typically get around that because of the size and quality of our house file.

Regina Lewis

Online research is well on its way to full acceptance. I know that at this point, I only use phone for specific market questions that require sample beyond what current online panels can provide.

Jill Donahue

Online research plays a big role for us. We currently have safeguards in place that we feel protects our data quality (as best as it is able to be protected) to ensure the right people are being contacted. Obviously there is always room for improvement. It would be great to incorporate some of the safeguards that the more classic methodologies lend themselves to (i.e., audit responses)

Dan Womack

I would say online research is a bit like a young toddler. Growing quickly, showing tons of potential but certainly stumbling and breaking a few things as it starts to mature. I won’t go on with this metaphor, but I think we are just starting to tap the potential of online methodologies. There is still much to learn, but much to look forward to as well.

Jami Guthrie

I would call it "developing". At Wrigley we manage the Consumer Insight function on a global basis, so depending on what region we are operating in, we use on-line research very differently. For instance in Russia and China we are still using it on more of experimental basis given our consumer targets and the internet penetration of those countries. Our challenges there are quite different vs. North America and Western Europe where our use of the internet is our dominant data collection methodology.

Bill Tanner

We may kill the golden goose. The ability to do a wide variety of online research quickly at low cost is a tremendous advantage. However, I and many others are concerned that progress has been made at the expense of respondent quality and co-operation. The number, quality and style [repetitive, uninteresting, etc.] of online surveys is creating a growing problem. Alas, I am just as guilty as the next researcher but am on the 10 step program to better, shorter studies.

Bob Graham

It’s gone beyond "evolving" into "maturing". I think everyone accepts it as a valid quantitative option (I still have too many reservations about online qualitative), but something needs to happen to make it seem groundbreaking.

Is Ethnography playing a bigger role in your research plans? If so, how?

Jon Last

Conceptually, it remains appealing, and I’ve dabbled with it. But the costs are still too restrictive to deploy ethnography with any regularity.

Regina Lewis

I have been a fan and user of ethnography for over 10 years now; however, I do see colleagues using it more extensively — even internationally! I, personally, will embark on in-home work in China this winter

Jill Donahue

Absolutely, it helps us get a deeper understanding into our quantitative results.

Dan Womack

Yes. I’m a fan of anything that can provide a richer, deeper understanding of customers, and I find some of the ethnographic-like approaches very promising. Regardless of the product or service category, it continues to be more and more difficult to differentiate based on traditional things such as features or price. The kinds of insight you can get from ethnographic work may provide an edge you wouldn’t get with any other kind of work. I expect to continue exploring ethnographic approaches and using them where they make sense.

Jami Guthrie

It sure is. We are continuing to put much more emphasis on including consumer input early on in a new marketing/product initiative to make sure everything we do is rooted in consumer and category insight. We talk about it as one of our many "feed the gut" initiatives where the sole objective is to improve our consumer understanding/instincts so we rely less on tactical research evaluation later in the development cycle.

Bill Tanner

We have done ethnographic studies and learned a great deal from them. A study we did several years ago showed that young adults were forming social networks using the technology available at the time. It helped us realize the potential for online social networking tools.

Whether we use ethnography more depends on the business problems we need to solve. That said, basic observation of consumer behavior is something we will continue.

Bob Graham

Since I just started a new job, it’s too early to tell here. But we were doing a lot of exciting ethnographic work at RadioShack.

What do you think is the greatest need in the market research industry?

Jon Last

As I’ve said before, and I dedicated a large part of my presentation at IIR to this, I’m upset that many in the profession are too complacent and fail to embrace the true opportunity that we have to be strategists and consultants rather than just passive users or dumpers of data…often like a drunk uses a lamp post… for support rather than illumination. Of course, the fact that not enough people understand and practice this, can be a great source of competitive advantage for us, but the fact that this situation continues, hurts the overall credibility of the profession.

Regina Lewis

I say this throughout my press interviews … but I will repeat it. The greatest need is for enhanced supplier quality control.

Jill Donahue

A voice. The common consumer understands this industry exists, but not really what it does. I think the more we educate the better results we will get as consumers will know they have a direct link to the product and services they purchase.

Dan Womack

My greatest concern relates to respondents. How do we reach them and keep them? I think there is a great need for innovation in the ways we interact with respondents. Too much of what we do seems irrelevant to customers, and we deliver this seemingly irrelevant material using some of the most boring tools imaginable. And I’m not just pointing fingers at others here—I pull from the same basic tool box as everyone else.

End users or client-side researchers like me can’t dismiss this as just a supplier issue. Yes, we should expect our research providers to be working on the problem, but we have to take more responsibility for this and other quality issues and work with research providers to find solutions.

Jami Guthrie

Wow, that’s a great question. I guess my answer would be the industry’s need to attract a constant flow of great young talent into the field. When I recruit MBA’s on-campus there still is a low level awareness of what a career in Marketing Research looks like. Furthermore, a lot of the great young talent that I meet on the supplier and client side tended to "stumble" across the field in one way or another (and are quite happy that they did so). Hence, I would like to see the industry get a lot more intentional about promoting itself and attracting the best talent into the field.

Bill Tanner

Researchers on an aggressive personal growth trajectory. Researchers that are committed to professional development have genuine intellectual curiosity, the desire to lead and who are dissatisfied with the status quo. There is as much opportunity to raise research ROI by including an understanding of business issues, creative solutions that go beyond the research at the project back end as there is by smart design at the front end. Researchers add value when they develop better solutions than clients can develop for themselves.

Bob Graham

A better understanding of what we do by the public at large.

What type of research are you most frustrated with regarding it’s ROI?

Jon Last

I’m not a big fan of a lot of the syndicated audience measurement currency. I’m particularly appalled by some of the newer product that purports to measure advertising ROI and effectiveness by asking consumers directly whether a particular advertisement prompted them to think differently about a brand or take a specific action. Respondents’ minds don’t work that way. We push our clients who insist on trying to measure this, towards experimental design—specifically blind pre-post perception and awareness studies with verified samples of control and exposed populations. At least in this fashion, you can look at changing attitudes over time and begin to assess the overall impact of an entire marketing campaign.

Regina Lewis

I can’t truly answer this, Merrill. I am not a believer that research — in any form — lends itself to true ROI analysis. The potential costs of *not* conducting insights work are so high that they typically are immeasurable. At the most extreme, in my view, we should be pushing internal clients to think in terms of outcome / scenario planning versus research ROI.

Dan Womack

I work hard to match needs and methodologies and honestly don’t have a lot of frustration around projects not doing what they are supposed to do. Having said that, I do find that very large, complex and –therefore—expensive studies often fail to deliver as expected. I think this is true for a variety of reasons.

Studies often get very large and complex because objectives are unclear or someone is trying to "save" by squeezing everything into one project. Long, complex questionnaires abuse respondents and greatly diminish quality. This leads to internal questioning of the data and, accordingly, no action. This list of reasons could go on, but the conclusion would be the same—we need to simplify wherever possible.

Jami Guthrie

The research that we conduct when we really already know the answer!! :)

Bill Tanner

If a method fits the problem, it can deliver good ROI. My frustration is with weak research application and execution e.g. questions people cannot answer or that include an inherent bias [e.g. "What type of advertising first made you aware of this product?", etc.], research that is so tedious or cognitively overwhelming that it begs respondents to check any box or quit, ‘lazy’ analysis of results which contribute facts but no insight or solutions, use of inappropriate techniques, etc.

Bob Graham

Trying to judge the effectiveness of event sponsorship research. Everybody wants to know the ROI from sponsorship, but I’ve yet to see a model I feel fully confident in.

How would you characterize the quality of the deliverables in 2007 vs. previous years?

Jon Last

Realize that we’re a full service provider moreso than a research buyer….so our deliverables were among the best work we’ve ever done and our data collection suppliers continued to perform at the highest quality levels. As for syndicated product or third party work that we purchased, I’d say that quality continues to wane both on the customer service side and the efficacy of the research.

Regina Lewis

I have seen a decrease in the quality of supplier deliverables. The year 2007, for my team, was wrought with data quality issues, poor judgment regarding everything from design to weighting, and powerpoint decks that needed to be reworked. I am confident that 2008 will be more solid. :)

Jill Donahue

Our reports are much more actionable than they have been in the past, and that is great. There seems to be an increase in strategic thinking of the results, not just a book report of what the results are.

Dan Womack

Unfortunately, I’ve seen a decline in quality over the past 4-5 years. I think some of this can be attributed to a variety of well-known respondent issues, but many of the problems I’ve seen are talent related. I’ve seen questionnaires that could have originated from an elementary school class project, and data tables that were just flat wrong. I’ve had to redo reports and correct very basic kinds of things—mistakes that experienced, thoughtful researchers wouldn’t make (unless they were overwhelmed, which is quite possible and a potential topic for next time).

To be fair, I’ve seen some real bright spots in the past few years too. I’ve worked with a few research providers, mostly smaller to mid-sized companies, that really seem to value quality and work hard to attract and retain quality talent. There is a lot of good work being conducted, but it’s harder to find than it should be.

Jami Guthrie

For us, the quality of the deliverables from our research partners has definitely gotten stronger over the years. As with most companies, we simply don’t have enough resources to do everything we want to do, so one of the initiatives we employ is to bring our key research partners as far "upstream" as possible. In doing so, these partners gain a deeper knowledge about our business, learn the internal processes/quirks of the system, and know how to influence key constituents. That certainly helps drive the quality of the deliverable and lightens the load on my internal team.

Bill Tanner

New plateau. In my business, quality has generally improved but it is still not where it should be.

Bob Graham

Varies by supplier. The best ones deliver consistently year in, year out and are always looking for ways to improve. Some are so set in their ways that they keep on rehashing the same old report.

From your standpoint what is the hottest trend in market research?

Jon Last

A lot of the current "buzz" seems to be centering around word-of-mouth, scraping the blogosphere and utilization of dedicated online communities for insights. It has my interest given the social nature of our endemic categories (golf/leisure), and we’ve already lined up some work with our clients for 2008 that will delve into these modalities.

Regina Lewis

Probably, social media insights work. I personally have two calls in upcoming weeks to explore such possibilities.

Jill Donahue

The biggest trend I have been seeing is the ethno and alternative qualitative methodologies. Really getting to the understanding of why consumers react they way that they do.

Dan Womack

In general, I would say the hot trend is still the continued move to a variety of online methodologies. Ironically, I think some of the issues (perceived or real) associated with this move to online tools have also sparked another trend. While I don’t have data to back it up, conversations with some of my colleagues leads me to believe more than a few are moving back to telephone or even mail-based work.

Jami Guthrie

The idea of "co-creation" with consumers as it relates to communication and new product development. I’m seeing lots of proprietary consumer panels and "communities" being deployed in an effort to involve consumers early on in the process.

Bill Tanner

Neuroscience, hot but useful? There is a great deal of new learning about how people think and why they think that way. Some of it obvious and accessible at lower cost through other methods but some of it revealing. Some good sources: Any book by Robert M. Sapolsky of Stanford and the blog Neuromarketing [ http://www.neurosciencemarketing.com/blog/articles/cool-products-and-neuromarketing.htm ]

Other interesting trends include advances in data mining, new developments in tradeoff analysis, more interesting ways of asking questions online, and hopefully a return to getting the basics right. We live in exciting times.

Bob Graham

DK/NA

What research conferences do you think you will you be attending in 2008?

Jon Last

I’ll attend both national MRA conferences and probably go to IIR. If I am in town while ARF is here, I’m sure I’ll stop by there as well. This year, I spoke at TIA’s market outlook forum, and will also consider returning to that.

Regina Lewis

I would like to attend Conference Board Council on MR meetings, + an Innovation conference (this may or may not be produced by the MR industry), + a Forrester conference. We will see.

Jill Donahue

MRA Annual and Fall conferences.

Dan Womack

Is it time to talk about 2008 already? My hope is to attend the MRA Annual and Fall Conferences along with the AMA Marketing Research Conference—-though I couldn’t even guess when or where any of these are scheduled for ’08.

Jami Guthrie

The ARF for sure. Beyond that, I haven’t really given it any thought at all.

Bill Tanner

None. While I value research conferences, I have been attending conferences that leverage research but are not labeled research such as OMMA [Online Media & Marketing Association.] It is a good way to learn something different.

Bob Graham

I’d like to attend the AMA Conference. Haven’t been there in a few years.

I would like to personally thank Jon, Regina, Jill, Dan, Jami, Bill and Bob for taking the time to participate in this blog. I hope all of the readers found the information interesting and appropriate to their business.

Fee free to post any comments or questions for the seven researchers.

3 Responses to “Client Point of View” - Leave a Reply


Leave a Reply


Please notify me of new posts on Merrill's Blog. (New posts generally come out every Monday, Wednesday and Friday)

Please enter the text from the image to the left in order to submit your comment





Thank you for your comments. If you would like to sign up for your picture to be displayed next to your comments, click here to go to gravatar.com