Dangers in Blind Reliance Using Online Survey Tools
Posted by cemiller on April 19, 2008
Yes, Dangers. I opened this article with some seemingly twisted logic about survey software taking you blindly to a wrong destination without you realizing you were blinded. Let me explain…
Survey software tools make it easy to write a good survey. True. They also make it easy to write a lousy survey! And you’re probably better off doing no survey than doing a lousy survey. Why? Because a poorly designed survey can generate misleading data that provides you with delusions of knowledge. Researchers refer to this as “instrument validity,” which simply means that the survey instrument measures what you intend to measure. That may sound like a no-brainer. It’s not! Examples of invalid surveys abound, many done by professional market research organizations.
These online survey tools can not tell you how to create a good survey. They also can’t tell you that you have a bad survey! Only knowledge of good survey design practices will help prevent you from designing a bad survey. (Obviously, this is why I recommend you buy my Customer Survey Guidebook or attend one of my Survey Design Workshops!)
The Internet web form survey approach exacerbates this blind spot. With a paper-based survey, respondents will give you feedback on the quality of your survey questionnaire. You’ll see big question marks and comments in the margin. (I’ve been known to do this on occasion…) With telephone surveys, you can listen to respondents formulate questions or talk with the interviewers to find out where the survey instrument has shortcomings.
How does the respondent provide you this feedback about an online, web survey? Perhaps you’ll see something in a comment field. But I doubt it. This is an incredible danger zone for this surveying method — and one that the survey software tool vendors never mention.
Technical knowledge of how to use the survey software too does not equate
to knowledge of how to design a valid survey questionnaire !!
A twelve year old can push a car’s gas pedal and turn the steering wheel. Would you want them to drive a car? Yet, today many well-intentioned people are using SurveyMonkey and the like, creating survey questions that lack validity and then make business decisions on that bogus data.
Advertisement
