So you wanted to be in the research industry. Your dream was to be a researcher? Well whether you are or not…now is your chance.
I was reading a recent survey column in the May 2007 Smart Business Dallas Magazine and thought I would share it with you.
The survey was conducted by Robert Half International and Careerbuilder.com and surveyed more than 3,000 U.S. workers.
Here is a table that the article referenced:
I am satisfied with the performance of my boss/supervisor.
| Strongly agree . . . . . . . . . . . . | 19% |
| Somewhat agree . . . . . . . . . . . . | 33% |
| Neither agree nor disagree . . . . . . | 20% |
| Somewhat disagree. . . . . . . . . . . | 17% |
| Strongly disagree. . . . . . . . . . . | 11% |
- What does the data mean to you?
- If you were the research analyst and had to write the headline what would it be?
The headline that was written in the article was:
A GOOD BOSS ISN’T HARD TO FIND
When I was reading the chart, I thought that if our team rated their supervisors this low we would have a problem. Less than one in five people strongly agreed with the statement. To me that was shockingly low. I would have thought it would have been higher than that. As an executive I would have wanted it to be much higher.
Am I missing something or does that data not support that headline?
I look forward to your “analysis” and what this data means to you.
Obviously it means something else to me.
Dan Krason says:
These numbers aren’t great. While there are plenty of poor managers out there, I think it may be unfair to point fingers solely at management… I’m sure many managers request candid feedback from their staff, but they don’t receive it because it is fundamentally difficult (and sometimes unwise) to criticize your boss directly.
GE had a great practice years ago… They’d have “one over one” meetings, where staff would provide candid feedback (in a safe setting) to their boss’ boss. Senior management would then synthesize and “de-personalize” this feedback so it could have a meaningful dialog with the appropriate manager involved.
This approach isn’t a cure-all, but it may be well-suited to managers that are sincere in their efforts to create a positive environment for all employees.
Stuart Drucker says:
On the surface, the numbers aren’t particularly encouraging. However, I think that this “finding” may be more an indication of poorly executed research than a representation of American workers.
In order:
The question as posed sounds like a ratification of the degree to which the respondent considers themselves satisfied with their superiors’ performance. Directly asking “I am satisfied…” on a five point Likert scale of agreement isn’t a standard research practice. Rating overall satisfaction (assuming performance is measurable) on 0 to 10 or 0 to 100 scale with anchored endpoints, and a “How satisfied are you….” lead-in is more so.
Beyond that, was this a random sample, or people that were self-directed to the website because they were either starting a career or looking to change positions? If so, this may be representative of those respondents (unknown without having more demographic comparisons) than representative of workers per se.
Finally, there’s the issue of heterogeneity. Do these respondents come from a single population or industry, or segments that might have particularly low levels of job satisfaction which mask more satisfied segments or industries? With the distribution from the Smart Money article, we could have four segments: a) a highly satisfied segment, b) a marginally satisfied segment, c) a strongly dissatisfied segment and d) people that are indifferent about the satisfaction they have with their superiors. We just don’t know from the total report.
And then there’s scale use….if this question were in the middle of a battery of questions about job performance, you might get a different answer – one more driven by fatigue or “yea-saying/nay-saying” than if it were separately after that section of the survey.
In short, as a representation of “3000 workers”, I think the answers may be biased, but without knowing the population, I’m not sure in which direction the bias moves. A “survey” does not imply an unbiased, well-executed piece of research.
Personally I think this research serves as an example of the need for certification more than a subject for serious analysis.
Ed Erickson says:
Ugh. Two major, but – sadly – all too common, problems.
1. A poorly worded question.
2. A huge mis-interpretation of the data
And that’s assuming the best about all of the sampling issues raised by Stuart.
A few weeks ago, there was a post about “faster, better, cheaper” which raised the issue of downward price pressure in research. Clearly, the sponsors of this research opted for faster and cheaper.
[...] In his blog, Merrill Dubrow, points out an all too typical example of survey research poorly executed. My opinion is that this was most likely an in-house DIY survey done by people who lack experience in research. Why would they opt not to hire a researcher to conduct the survey? Becuase their experience or perception (or both) told them that they wouldn’t get any more expertise than they already had for their money. [...]
Steve Runfeldt says:
I think that this example shows the problem with attempting to say something meaningful by picking one question out of a survey. What was the sample? Were these clerks at Wal Mart or engineers at Google? What were the responses to the other questions? Without demographic and other information, it is really impossible to tell what these, or any other, stats mean.
We also need to be careful to not reject the results simply because they do not fit into our expectations. It may be entirely accurate that nearly 1/3 of workers are not satisfied with their bosses.
That said, the question itself is problematic in that it combines two subjective ratings questions into one – How satisfied are you? and How much do you agree/disagree? This introduces two sources of variance – a level of satisfaction (or what the respondent understands by the word) and a level of agreement. If someone agrees 80% that they are 80% satisfied with their boss do they check “strongly agree”, “somewhat agree” or “neither”? Is strong agreement with strong satisfaction 100% or would it be a combined 64% or something in between? While the question may be fairly straightforward for any one respondent, different respondents are going to differ in their interpretations of the question.
Dino Fire says:
In isolation, there’s at least one bit of encouraging information in this data…Top-2 box performance ratings are almost twice that of Bottom-2 box ratings.
Personally, I would worry if Top-2 box scores were overly high…one sure path to failure for managers is trying to please everyone. To make an omelet, you have to break a few eggs.
Bob Grayson says:
Given all the above comments, most of which makes some sort of sense–after all, most any research can be better if you fiddle enough, the fact remains that the numbers pretty much reflect the real world.
Think about your own experience and those around you. There are those who are always at odds with the boss and those that are in concert. The results are skewed off the bell shaped curve in favor of a good boss. That’s most likely the way the business world is.