You Can Lead a Boss to Data but You Can’t Make Him Think

The most carefully planned data analysis may not survive the intervention of a boss (or a client or other reviewer), whether well intentioned or not. Your aim may be to generate sound data and conduct a thorough and valid analysis, but your boss may have different motives and concerns. He or she may have budget or schedule constraints, not to mention business vulnerabilities and office politics to contend with. So be prepared when the unthinkable happens … like just a few days before you planned to start collecting data your boss calls you into his office and says:

Change This Before You Start

Add a sample, reword a procedure, drop a measurement, and other requests that will defile the perfect data collection and analysis plan you spent so much time creating. What do you do?

Adding a sample or measurement may not be a problem so long as you have the equipment and sampling supplies available. Changing a survey shouldn’t be a big deal if you haven’t already printed the questionnaires. Do what the boss wants and don’t worry about it too much.

Changing sampling procedures or rewording survey questions may or may not be a problem. Keep an open mind. It’s when the validity of the sampling procedure or meaning of the survey question is changed that you have to be careful. Sometimes the changes may seem subtle. For example, changing the order in which samples are collected may seem inconsequential but it may have an impact on the results. Asking survey respondents if they do something once a week is not the same as asking if they do something often. Asking if something is good, is not the same as asking if something is better than expected.

Dropping a measurement or question is problematical. If you didn’t need it, you wouldn’t have included it in the first place, but your boss won’t see it that way. Metadata and supporting measurements seem to be a favorite target of bosses looking to put their stamp on your plan. They might not understand that you really need those survey questions that characterize the respondents. These omissions are tough to lose but it’s better to have something over nothing. Make sure the boss knows what he’ll be losing and go with what you can.

We Can’t Afford This

That’s more than I expected to spend is a mantra statisticians hear often. You may even have said it yourself to the dealer of that hot red convertible you want, or to the plumber who came to fix your broken water main, or to the tax preparer you finally cornered at 5:00 pm on April 15. Did you pay up, pass on the offer, or negotiate something in between? Likewise, there are three things that your boss can do in this situation:

• Relent and pay for the study
• Negotiate a reduced price
• Cancel the study (or get someone cheaper to do it).

You want the first thing to happen and can live with the second, but the third would be a disaster.

So put yourself in your boss’s position. Is the analysis something he has to do? If so, gently review the consequences of not doing it. What is it he doesn’t want but might get if the study isn’t done? Will his boss be upset? Will a regulatory agency come calling? Paint a picture of the cost of the consequences compared to the cost of the study. If the analysis is something he doesn’t have to do, you’ll have to convince him that it is something he wants to do it, or even better, he needs to do. Consider what the boss is looking to do with the results. Show him how your data analysis will add value to his operation. Give him a clear vision of what the payback will be.

If that doesn’t work, try working out a compromise. Just remember, a cut in price has to be balanced by a cut in scope, otherwise you’ll have no credibility. If the cuts will impair the analysis so much that little will be gained, it’s better to pass on the study and survive to analyze another day.

That’s Too Many Samples

Having your boss (or client) ask if you can get by with fewer samples is almost a given. Unless the boss understands statistics and variability, it’s unlikely he’ll see the need for as many samples as you planned to collect. Agreeing to reduce the number of samples is a self-inflicted but nonfatal wound. The margin of error will be bigger but quantifiable. Let the boss know what he’ll be getting. He should appreciate the concept of trade-offs since he has to deal with them all the time in his own work. This assumes, of course, that the boss is at least in the same ballpark as you. If you want 800 samples, and he was thinking of just asking a few customers some questions over lunch, you’re toast.

Here Are the Samples You Should Take

After spending a lot of time ensuring that your samples will be truly representative of your population, your boss gives you his list of samples to be collected. Will your boss’ list fairly represent the population or is there some perhaps unintentional bias? Does the boss want you to survey the biggest or oldest customers, or worse, the customers who will give the best reviews? Does he want you to sample only the processes or waste streams that he knows are already within specifications? What do you do?

This can be a deal breaker from your perspective. There’s no reason to collect data and do a statistical analysis with a judgment sample let alone a highly biased judgment sample. You’ll only be deluding your boss and yourself. Try to convince your boss that his directed samples will invalidate the study. If you can’t, take a list as a window onto your boss’s real reason for conducting the survey. He may just want something flattering to show his boss. If you still have to analyze the results even with the biased list, be sure to caveat your findings.

We’re Not Going to Do This After All

There are two common reasons the boss will pull the plug on a survey he asked for. The first is that he didn’t get approvals from his bosses. If you raise the level of attention of the survey in the organization early in the process, like when you go in search of data, this shouldn’t be a problem.

The second reason is funding. Some bosses relax during the first two quarters of the fiscal year then suddenly realize in the third quarter that they aren’t going to make their business plan unless they take drastic measures. Your survey will be the first thing to go, deferred if you’re lucky, canceled if you’re not. Try to schedule your survey for early in the fiscal year.

The Results Aren’t What We Expected

If your boss says he is surprised with your results, it could mean a couple of different things.

If the boss says either that the results were too general to tell anybody anything or that the results were too detailed for anybody to figure out, it’s a presentation problem. That’s actually a good thing. It can be fixed. Don’t be reluctant to get a communications specialist to help translate your work into a better presentation. You will benefit from more people being able to understand what you did.

If the boss is surprised by your results, and it’s not the presentation, it’s a virtual certainty that the results are unfavorable. There are at least four possible scenarios to consider:

• The boss is really surprised by the results because he is out of touch with his business or whatever it is you studied. In this case, watch how he reacts as you explain some of the intricacies of the findings. As long as he doesn’t become defensive, it could be a great opportunity for both of you. If he does become defensive, try stressing that the results don’t take into consideration all the constraints he is under. Protect his fragile ego. You need him to take the next step and do something positive.
• The boss is really surprised by the results because he thinks there may be a problem with the data or analysis. Double-check your work to be sure nothing is amiss statistically, and keep an open mind. The boss may be aware of some aspects of the data that may have skewed your results.
• The boss is not surprised by the results but doesn’t want to admit it because he was hoping for something different. He’s playing dumb. This is his way of trying to deflect some responsibility away from himself. Play along. Remember, the only important thing is that he takes some positive action based on your results.
• The boss was not surprised by the results but doesn’t want to admit it because the results don’t match what his boss wants to see. There’s not much you can do about this. The boss will probably take the results, and you’ll never hear of it again. Consider it a valuable life experience.

Just Give Me the Results; I’ll Take Care of the Rest

Everybody who has a boss has lived this alternative reality, whether you’re analyzing customer satisfaction data for the CEO or wrapping burgers for hungry customers. You give your work to the boss who becomes the face of the product. The boss gets all the accolades, and perhaps, an occasional complaint, while you get little if any recognition.

True, some bosses do this to try to claim credit for work they didn’t do, but this is usually transparent to anyone who matters. The CEO knows your boss neither conducted the analysis nor has the knowledge or the time to have done it. He is responsible, though, for the work you do. You know when you go to a restaurant that the host didn’t prepare the food. You know when the politician cuts the ribbon at the opening of the new library that he didn’t build the building or shelve the books. Those people represent the work of many.

Just give me the tuna. I’ll take care of the rest.

But a boss may want to be the spokesperson for your work for other reasons. He may want to bury unfavorable results. He may want to control access to the results because knowledge is power. Even more important, he may want to control access to you. Your demonstrated skills make your boss even more powerful. Finally, it’s possible that your boss doesn’t want you involved in decisions made as a result of your work. While this judgment is usually blamed on ego flexing, it may also be attributable to self-loathing over the decision-making process. If you think data analysis is messy, you should see what decision makers do. They are often illogical and inconsistent. They choose paths of least resistance over avenues of greatest effectiveness. They pay more attention to anecdotes than data. They worry about obscure possibilities and ignore likely consequences. They have a penchant for inaction and a desire to preserve the status quo. Unless you have a frustration deficit in your life, it might be better to let this one go.

One big issue with this response is that you don’t get any feedback. It’s nice to get recognition for your work but it’s absolutely essential that you get feedback. That’s how you grow as a professional and as an individual. If you can’t get any feedback from your boss, look elsewhere. Do any of your colleagues have opinions about the study? How did the high priest of the database like working with you? Do you ever run into the CEO or other managers in the hallway who might be familiar with what you did? Take whatever you can get (without making your boss paranoid) and put the knowledge to good use.

Read more about using statistics at the Stats with Cats blog. Join other fans at the Stats with Cats Facebook group and the Stats with Cats Facebook page. Order Stats with Cats: The Domesticated Guide to Statistics, Models, Graphs, and Other Breeds of Data Analysis at Wheatmark, amazon.com, barnesandnoble.com, or other online booksellers.