10 Steps to Creating More Effective Surveys

Surveying your customers, clients, staff and visitors can be incredibly useful in terms of providing information you need to further develop products, services and relationships. By asking the right questions, in the right format, you provide people the opportunity to give you feedback on a host of topics. You can then use the feedback to move forward. The difference between success and failure in survey creation is a matter of ten simple steps.

Step One – Know What You Need to Ask
If you’re going to create a survey, the first thing you need to establish is an objective. So, start by asking yourself what you need to find out – then work to frame a survey packed with questions designed to get your respondents to give you the information you need.

Step Two – Ask Effective Questions
Once you’ve set a course for your survey, you’ll need to choose your questions carefully to get the right responses. Choosing the wrong format for your questions can lead to inaccurate information, making your survey results less helpful than they could be. Ask questions that give your audience scope to respond adequately, but don’t leave so much leeway that it’s confusing. If it’s a yes/no question, only give yes and no as answers – on the other hand, where there’s room for maybe, be sure you include it.

Step Three – Choose The Right Respondents
Targeting the survey appropriately is an important step in gleaning the most from the exercise. You’ll need to be sure you choose the right method of survey deployment – whether it’s a survey link on your website, an email inviting past customers to participate in the survey or a random pop-up window asking visitors to take part. Choosing the right deployment for your survey can make a big difference to the quality of respondents you have to work with, as well as the data you receive.

Step Four – Screen The Audience, If You Need To
In addition to deciding the best method of deploying your survey, consider if it is worth screening the respondents at the beginning of your survey. For example, if you’re interested in learning what your shoppers think of a new shipping service, you don’t really need 50 responses from customers who only pick items up in a local depot.

Step Five – Estimate The Time Required
As a courtesy, tell your survey participants how long the survey will take to complete. Obviously, you don’t know how fast or slow people read and tick boxes, so you can only guess based on how long it takes you. But providing a rough estimate of the time, a number of questions, and even progress meter along the way can go far in terms of keeping customers engaged in the survey process. An engaged respondent is far more likely to provide good data.

Step Six – Keep Your Audience Interested
Whether you’re assessing your customer base for a new product or service rollout or trying to get a gauge of customer satisfaction, it’s important that you keep your audience engaged in the survey process. This is best done by asking a variety of distinct questions and using a number of different question types. If people think you’re just reusing the same question over and over, they’ll click away or just randomly respond to get done with the process rather than provide the data you need.

Step Seven -  Keep Your Survey Tidy
Another key point in keeping your audience engaged is avoiding a massive scrolling survey. Try to keep page breaks at logical intervals, so that like questions are grouped together. You should aim to keep questions, and answers, to the point – spell check these as well to avoid appearing unprofessional.

Step Eight – Try Your Survey
In addition to taking the survey yourself, get a few other staff members in your organization to run through it before it goes live. This can help you proofread the questions, to avoid typos or grammatical errors. More importantly, it will ensure someone else has read through it and that the questions make sense to a third party. Have a look at the results your survey generates based on the sample responses to determine whether you need to fine-tune your questions.

Step Nine -  Interpret The Results
Whether you use an online survey software provider or organize the survey and receive the results yourself, you’ll need to find a way to make sense of it all. Generally, your survey results can be displayed in all manner of formats – how you choose to do this is largely up to you. In any case, making sure the information you collect from your survey respondents is easy to interpret is a key to the final step in survey success – making use of the data.

Step Ten – Make Use of the Data
Perhaps the biggest hurdle for those using surveys is making use of the data they receive. It’s all well and good to design a great online survey, and it’s even better to work with an online survey software company that can collate the data and present you with an easy-to-understand battery of results. However, if you don’t actually have a plan for using the information received, the exercise is fruitless. So, when you’re planning your survey, include a mechanism for making use of the results.

The Decision-Making Process for Selecting an Internet Survey Software Tool

Finally, how should you go about selecting a web survey tool?

Determine Your Needs.  This should seem obvious, but it’s not.  I’ve worked with companies where senior managers have become enamored with Survey Monkey, Zoomerang, or some other product because they had experienced a survey as a respondent from some tool.  It shows great marketing!  But when I ask them if they know if SurveyMonkey will meet their requirements for their survey, the tone of the response told me to back off.  The basic tools, including Survey Monkey, have limitations.  

How to determine your needs?  Develop your survey questionnaire in a rigorous fashion, discussing the role of the survey in your organizational process, and being sure that the survey properly captures the true concerns of the respondent group and not what management thinks are the concerns of the respondent group.  The questionnaire along with various administration requirements will become your requirements statement.  

Cost Comparison.  Cost comparisons can be difficult since each survey software company may use its own pricing algorithm.  Pricing may be driven by the number of survey projects, the number of responses to a specific survey, the number of invitations sent for a survey, or the number of questions in a survey.  Also, the features you want may drive you to a higher-level package or customization.  Create your expected scenario of how you will use the survey software and look at the price.  

Take a Test Drive.  All of the tools — or all that you should consider — will allow you to download or use the software.  Some will let you execute a survey project, limiting the number of respondents’ data in the database or some other limitation.  Definitely take a test drive.  Two products may compare well on price and features, but check out their ease of use.  You’ll be driving this car.  Wouldn’t you like to know how comfortable the seats are for a long trip?

In summary, online survey software tools can make the job of conducting survey research much easier and less expensive.  When selecting a particular tool, you have many factors to consider.  The most important factor is your level of knowledge of good survey research practices.  Without that, you may well be driving blindly without knowing you’re blind.  (I’ll give a final plug for my Survey Design and Survey Data Analysis Workshops.)

Key Features of Online Survey Tools

The potential feature list for survey tools is quite extensive.  Many of these may be unimportant to you.  But many may be important to you — and you won’t know it until you create a survey and find out the survey tool you selected doesn’t support certain features.  Or they may become more important as you become more sophisticated in your survey practices.  All the survey tools will support the basic features.  

Comprehensiveness of Survey Question Types.  Look at the survey question types supported.  (Note: the survey software tools may use different names for these question types.)  All will support multiple choice, interval rating, and open-ended comment boxes.  But are there limits on the number of points on interval scales?  How large a text buffer can you have for comment boxes?  

You may also want to use semantic differential, forced ranking, or fixed sum survey question types.  Are these available?  Does the tool provide a running sum as the respondent completes the fixed sum question?

Advanced users may want to randomize the question order to control for question sequencing interaction, randomize the order of the answers within a question, reverse code selected questions, or control the number of responses in a multiple choice question a respondent can select.  

Design Flexibility.  Can you customize the formatting of the output that the respondent sees?  Shading of alternate questions is a nice feature.  Can you add a logo and custom HTML?  Can you require an answer for selected questions only?  Does the vendor force you to have their “ad” at the bottom of the screen?  

If you’re setting up a matrix or table for a series of interval rating questions, can you control the width of the table with the radio button response options?  If you are putting anchors — those words that describe the scale — only over the endpoints of the scale, can you control the distance between response options?  This is vital to maintaining some semblance of interval properties to the question.  

Conditional branching is a very useful feature to pass respondents by a series of questions that are not applicable.  Is this supported? Can you loop through a series of questions for multiple occurrences of some event, for example, feedback on a number of different classes taken?  Is text piping supported in the looping?  

Frequently, we want to branch based upon the response a person gives.  For example, for a low score we may want to branch to an open-ended comment box.  Is this supported?

Progress indicators are a nice feature as is the option for respondents to postpone completing the survey should they get interrupted — saving the previous responses of course.  How simple is it to add section headers or other descriptive text that is not part of a question?  Upon completion of the survey, can you redirect the respondent to a specific web page?  

Ease of Use.  How easy is it to create a question or apply a scale to a series of questions?  Can you create your own library of questions or scale types that you can readily call upon for future surveys?  

Respondent Friendliness.  Check for the experience of the respondent.  If the respondent makes an error in entering an answer — a particular problem with forced ranking and fixed sum question types – what are the error messages?  I’ve seen some tools that scold the respondent.  Not good for response rates!  Does the system support halo text or some other means of providing further explanation of a survey question?

Integration with Transactional System.  If you’re performing event or transactional surveys, does the system integrate with your transaction system to provide a seamless transfer of contract information?  Are there administrative controls to control for repeated surveying of an individual?  

Administration Support.  Will the survey tool create a random sample from a list of contacts?  A stratified random sample?  Will it tell you an approximate sample size you need for a certain level of statistical accuracy?  Can you restrict access to the survey only to those to whom you’ve sent an invitation?  Can you send out reminders to those who have not completed the survey?

As you can see, the capabilities of survey software tools can be quite extensive – and the above is not even a comprehensive list!  Do your homework and think about your true needs.  

Types of Survey Software Tools

Two major versions of online survey tools exist:

PC survey software.  With these tools you buy desktop or server-based software that resides on your computers.  The system will generate an HTML web form that you then post to a website.  When a respondent submits a completed survey, typically the data arrives to a designated email box.  The application will process these emails and load the results in a database.  

 

This article is from our Service Insights Newsletter.   Become a subscriber.  ~~  See abstracts of our other articles.

The key advantages of this approach are 1) you have a one-time purchase cost – though you may need to buy upgrades and support; 2) you’re working on your own computer or within your company’s network so the software will function faster; 3) you can post the survey to your own company’s web server.  Perseus is among the oldest and best known of these applications.  

Hosted survey software.  With this approach, you rent use of an application at the vendor’s web site.  SurveyMonkey is the best known of these tools, but there are literally hundreds of similar products.  Comparison shopping is advised.  You create a survey interactively on the web site and the HTML survey form is posted on the hosting company’s website.  

The key advantages of this approach are 1) no large up-front purchase cost and easier budgeting; 2) the hosted application handles all the IT issues, except for your having a working browser; 3) upgrades to the survey functionality occur continuously; 4) quick implementation — just sign up and get surveying.  With the PC-based tools, you become responsible for posting the form and its associated scripting files.  This can be troublesome.

The hosted approach is simpler, but in the very long run, it may cost more.  Though, when you include all the costs associated with the PC-based survey tool approach, hosted surveys could be less expensive.  Some companies for security and privacy reasons may require the survey to run on its own servers.  In this case, the hosted applications lose out.  

Dangers in Blind Reliance Using Online Survey Tools

Yes, Dangers.  I opened this article with some seemingly twisted logic about survey software taking you blindly to a wrong destination without you realizing you were blinded.  Let me explain…

Survey software tools make it easy to write a good survey.  True.  They also make it easy to write a lousy survey!  And you’re probably better off doing no survey than doing a lousy survey.  Why?  Because a poorly designed survey can generate misleading data that provides you with delusions of knowledge. Researchers refer to this as “instrument validity,” which simply means that the survey instrument measures what you intend to measure.  That may sound like a no-brainer.  It’s not!  Examples of invalid surveys abound, many done by professional market research organizations.  

These online survey tools can not tell you how to create a good survey.  They also can’t tell you that you have a bad survey!  Only knowledge of good survey design practices will help prevent you from designing a bad survey.  (Obviously, this is why I recommend you buy my Customer Survey Guidebook or attend one of my Survey Design Workshops!)

The Internet web form survey approach exacerbates this blind spot.  With a paper-based survey, respondents will give you feedback on the quality of your survey questionnaire.  You’ll see big question marks and comments in the margin.  (I’ve been known to do this on occasion…)  With telephone surveys, you can listen to respondents formulate questions or talk with the interviewers to find out where the survey instrument has shortcomings.  

How does the respondent provide you this feedback about an online, web survey?  Perhaps you’ll see something in a comment field.  But I doubt it.  This is an incredible danger zone for this surveying method — and one that the survey software tool vendors never mention.  

Technical knowledge of how to use the survey software too does not equate

to knowledge of how to design a valid survey questionnaire !!

A twelve year old can push a car’s gas pedal and turn the steering wheel.  Would you want them to drive a car?  Yet, today many well-intentioned people are using SurveyMonkey and the like, creating survey questions that lack validity and then make business decisions on that bogus data.  

What are Survey Software Tools?

Simply put, survey software tools provide the capability to create a survey where the painstaking survey administration tasks are automated.  Typically, we think of these automation tools as online survey tools that generate an HTML form to post to the internet, allowing your group of interest to submit their responses via the web.  That’s the focus of the remainder of this article.  However, there are other survey automation tools, such as

Computer Aided Telephone Interviewing (CATI) software that manages surveys delivered by telephone interviews.  

Interactive Voice Response (IVR) surveys, which are like telephone surveys, except that the scripts are recorded and the respondent enters answers using the keypad of their telephone.  These are frequently used in call centers at the conclusion of customer service calls.  

Optical Mark Recogniztion (OMR) or Optical Character Recognition (OCR) that allow you to scan a paper-based survey to capture the responses.  (Think back to your high school SAT exams…)

Benefits of Web Survey Tools

The key benefits that can be derived from internet survey software tools are:  

Simplicity.  You can create a survey quite quickly — once you learn how to use the tool.  Survey invitations are typically sent out electronically through an email.  That email will contain a link to an HTML form that is the survey questionnaire.  Responses are then captured electronically, either through incoming emails or by direct loading into a data base, depending on the type of survey tool used.  This data collection function is by far the key benefit of online survey tools.  If you’ve ever processed a hardcopy survey by postal mail and keyed in the data, you’ll see the benefit from survey automation tools very quickly!

Low Cost.  The out-of-pocket cost of these tools is quite low.  You can buy a PC-based tool for $500 or less or you can buy it as a service in a hosted application, typically paying a monthly or annual subscription fee.  The fees can be as low as $15 to $20 per month for a basic tool.  If you need more sophisticated features, the monthly cost could be much higher.  As noted above, the major cost is not the acquisition cost of the survey capabilities, but rather, the time to learn how to use the survey software tool.  

Scalability.  Unlike telephone or postal mail surveys, you can survey 10,000 people as easily as 10 people.  This scalability is part of what makes this a low cost method.  

Quick Data Capture Turnaround.  Since you may be soliciting feedback information about organizational performance, such as a customer satisfaction survey, how quickly you get the data back is important, especially if this is a transactional or event survey.  Quick data capture allows addressing a customer issue promptly, which is a key element in successful service recovery practices.

With email most of us practice a “do or delete” approach.  People will either take the survey or delete the invitation.  So, you’ll find with a web form survey that most of your responses will come back in one to two days.  Responses will quickly tail off to nothing.  Only IVR surveys can be faster since they are administered at the close of the transaction.  

Response Rate.  When web-form surveys were a novelty, response rates were quite high, perhaps above 50%.  Based upon my own survey project work and feedback from those in my Survey Design Workshop, response rates typically are in the 20% to 40% range.  However, the relationship you have with your group of interest is the single most important factor in driving the response rate, not the administration method.  

Questionnaire Complexity.  Internet web form surveys are fairly flexible.  (We can argue forever about which is more flexible: a printed page or a web form.)  You can create a fairly complex survey questionnaire with these tools, using extensive branching and other available survey features.  Do note that the survey tool you chose will set constraints on the questionnaire you design!  It’s better to determine what functionality you need and then select the tool, rather than the other way around.  

Survey Monkey

Intelligent survey software for primates of all species.  SurveyMonkey has a single purpose: to enable anyone to create professional online surveys quickly and easily.

Http://www.surveymonkey.com

 

Online Surveying