April 17, 2017

There are three dreaded L’s in research:

  • Low response rate
  • Lack of quality of responses
  • Little actionability from results

These problems can leave holes in your Customer Experience study that will reduce stakeholder buy-in, the reliability of the information you are reporting and acceptance of future CEX studies; it all can stem from one area — the design of your survey.  There is an art to survey design.  Just as a poorly designed venue can make or break your experience and likelihood to visit again, a poorly designed survey can result in the three dreaded L words in research.  In this article, we will detail some of the common pitfalls in survey design and share some of our best practices in order to ensure a good ‘L,’ lessening the likelihood of a failed CEX study.

The Scale.  We recommend the 1 to 10 scale over any other scale for several reasons.  It is universal, some individuals will inevitably be confused by a scale, but fewer are confused by the 1-10 scale due to the Olympics, it is metric, grading scales, etc.  Further, the 1 to 10 scale provides granularity in responses in order to do correlation analysis to uncover which functional areas are the key drivers of Satisfaction.  The reliability of such analysis is less with a 1-5 scale.   We recommend only defining the end points; ‘1’ is completely dissatisfied and ‘10’ is completely satisfied.  If you define each rating, it may not align with a respondent’s perception of that rating, causing confusion.   However, through our 25 years of research we have clearly defined and understand what an average of 6.0 means versus an 8.0.

Question List Development.  Try to keep the respondent in the forefront when developing a list of possible questions.  Often, questions are developed internally, because you know and understand your products and services as well as your customers better than anyone else, but verbiage or functional areas that you think are important, may not be important to the customer.  We want the respondent to be fully engaged throughout the entire survey; therefore, try to keep the survey to about 8 to 15 minutes in length with short, direct questions.  Respondents measure time not in how long they talk/write, but in the length of the actual survey.

Avoid questions that you should already know or are self-serving to the company and not the customer.  For example, don’t ask the question, ‘how long have you been a customer?’  That type of information should be known.  Further, questions such as referral, what additional products or services can we offer, how do we compare with our competitor are great questions, we recommend them in all our surveys; however, the results benefit the business not the customer.  Those types of questions should be asked later in a survey so the respondent is engaged in providing information on how you can do a better job for them, not how they can do a better job for you.

Question Type.  CSM recommends a pair methodological approach (a rating question followed by an open-ended question) enabling the customer to explain, from their perspective, what needs to be improved.  Avoid asking more than a dozen specific detailed questions.  We recommend a half dozen direct questions that will allow the customer to discuss the areas that are most important to them.  For example, you do not need to ask individual questions on quality of the product, accuracy of the product and consistency of the product; one overall question about product quality will provide the information needed to understand if there are problems in those various areas of product quality.

Question Order.  The final best practice in survey design is the placement of three key questions; Overall Satisfaction, Overall Strengths and Overall Improvements.  It is imperative to have these three questions at the beginning of a survey before you bias them with the functional areas that you deem important; i.e. Product Quality, Communication, Lead Time, etc.  This enables the most unbiased feedback in measuring your value delivery to your customers.

Designing a survey is critical to ensure you are exploring the right areas, gathering the most reliable insights and able to competitively differentiated yourself by leveraging your customer relationships.

Stay tuned for more research insights.

Author Bethany Gripp, M.S. in Research, Six Sigma Green Belt and Net Promoter Certified Associate – Vice President of Research at CSM Research, Inc. 

CSM Research, Inc. helps our clients understand the customer experience they deliver to increase profitability, cash flow and competitive differentiation.  CSM’s unique “Voice of the Customer” approach and solution of real-time, accurate and objective customer and market information, allows businesses to make better and faster operational and strategic business decisions. 

Please contact Christi Suchyna at 800.849.6788 or csuchyna@csmresearch.com to discuss how we can design a custom research plan to differentiate your business.