Rule #1: Surveys are about information, not justification

By Ann Ray on January 20, 2009 | Last updated October 5, 2009
Permalink | Comments (0)

There are three possibilities when you have a theory (or better yet, your boss or client has a theory) about survey results and start reviewing it in the data:

• You were right! All is well in the universe, the sun continues to shine.
  They like your feature best, frequent buyers have higher satisfaction levels, and last year’s hybrid matrix re-org was the best thing since sliced bread.

• Chaos reigns! The opposite of what you expected is showing up in the data.
  Your feature is unpopular, frequent buyers have more complaints than those who rarely contact the company, and employees are reporting lower productivity since the re-org?

• Crickets. Nothing seems to be happening.
  Everything’s just, well, Good, and you can’t find much in the way of differences between customer segments or before and after the re-org. Huh.

We all love to be right, but the important thing is to remember that a survey is all about getting information. Because if we’re not open to getting accurate information—whatever it may be—we’re just putting blinders on and setting ourselves and others up to stumble around the business landscape.

This is setting aside the minority who make ethical choices to slant "research" through loaded questionnaires or outright falsification of data. Instead it's a common sense reminder to everyone who's trying to juggle their organization's performance metrics, group goals, survey data, personal theories, and any other elements, of what's important to the organization at the end of the day. Also, remember that survey data is always only part of your decision process, integrated with secondary research and the knowledge of key stakeholders—never taken by itself as Sole Truth, no matter how good your sample and methodology.

Happy Surveying!