November 7, 2017
I read an interesting quote the other day from a gentleman who was extremely honest about the bias in research in which he essentially indicated that the margin of error for survey research was infinite, it made me stop and think. As a research professional, that really flew in the face of everything I believe when it comes to the reliability and validity of the research studies we consistently execute. It forced me to ponder the different types of bias and how it could permeate into survey data, and truthfully, I was surprised by the limitless list that ran through my mind. This seemingly daunting realization was quickly calmed by the antidote to bias–systematic and controlled execution of a research study. Although you cannot eliminate all bias, you can take strides to minimize it drastically.
Bias is an interesting construct and very subjective. In a human context, bias is typically unfairness or belief in one idea or person over another. In research, the term takes a much more real context and in most cases, can be quantified, documented and reduced.
Relationship Bias. If you execute your own research study, relationship bias is a real and problematic source of error in your data. Social desirability, the halo effect, wanting to be agreeable or simply having a negotiation mentality can influence some respondents’ answers if they are speaking to individuals with whom they have direct relations. This type of bias more than likely will creative a false-positive in your results. We have found that respondents provide more candid, open and detailed information when they are communicating with an outside third-party.
Sampling Bias. This type of bias occurs when your sample does not adequately represent the entire population. Proper customer segmentation at the beginning of the study can reduce this type of error by gathering contact data that represents an adequate number of accounts, functional roles, revenue levels, product and segment types. Using techniques such as stratified random sampling can ensure you reach statistical reliability for each group of respondents by setting goals and conducting surveys based on the pre-defined criteria.
Survey Methodology Bias. How you word questions, the sequent of questions and even how you ask questions can lead to unintentional bias. Questions that are leading, unclear, or phrased in a way that puts a respondent in a certain state of mind can create either a false-positive or a false-negative response. Therefore, it is important to ask neutrally worded questions and ask an overall question before functional area questions to reduce order bias. If administering the survey instrument verbally, it is important to ask the questions the same way every time. Call centers who use multiple people on the same project have a higher risk of decreasing inter-rater reliability. In that, if questions are asked differently by each person, it can have a true impact on responses.
Non-Response Bias. In any survey methodology, there will always be some portion of the sample that choose not to participate, this in known as non-response bias. Telephone surveys have lower non-response bias as compared to online surveys or mobile surveys. You can reduce non-response bias by properly notifying respondents, sending reminders and words of appreciation after they completed a survey to ensure future participation.
Relationship, sampling, survey methodology and non-response bias all deal with the research setup. Minimizing these key biases can have an impact on increasing the reliability and validity of the data collected. There are other types of bias that range from the respondent’s personality to analyzing the results, but, just as having a solid foundation is important for the structure of a building, so is a solid foundation for which a research study is based upon. Thus, reducing these key biases in the setup of the research study can have a profound impact on the actionability of the results.
Stay tuned for more research insights.
If you would like to learn more about customer research, please contact us at firstname.lastname@example.org
Author Bethany Gripp, M.S. in Research, Six Sigma Green Belt and Net Promoter Certified Associate
Vice President of Research at CSM Research, Inc.