[Home Page] [Previous Page]
[Old Roundtable Archives] [New Roundtable]
Posted by Mike Curtis on July 02, 1998 at 15:32:03:
In Reply to: Maximizing Response and Minimizing Bias posted by David Eagle on July 02, 1998 at 14:08:51:
David,
I'll discuss your first question here, along with some other questionnaire design issues. I'll leave the second topic to someone else.
Q1 You should be more specific here. Are you referring to fiscal year or calendar year? It's important that everyone interpret the question consistently, and in the way you mean.
Q2 Your categories do not appear to be mutually exclusive. For example, couldn't "spreadsheets" fall within the realm of "bank software" or "in-house software"? Also, guarantee that your categories are exhaustive by adding "other". Finally, design your software to prevent logical inconsistencies, like allowing someone to choose "don't perform function" and "spreadsheets" for the same topic.
Q5-7 Same as last issue above - don't allow "none" to be selected in conjunction with other categories.
Q9 This question could easily be subject to bias, particularly if respondents think a high numeric response will subject them to a sales call. The pricing question just below it will just add to the bias potential. Furthermore, the question doesn't really make sense to me. You asked up above whether I use software for these functions, now you ask whether I'm interested in software for these functions. Do you mean an all-inclusive package? If so, say so.
I would rethink the design and flow of questions 2 and 9 since they are somewhat related.
Q10 Price point is a throwaway. You can't handle product pricing like this and get meaningful output. Furthermore, if they answer "no" to this question, the open ended dollar amount will be highly biased. Don't believe me? Do a split sample experiment with 1/3 getting a price of $25K, 1/3 $15K and 1/3 $35K. Then analyze the open ended responses for each group.
Q11 Seems very vague. Do they have any familiarity with this product? If not, I don't know how the question could be answered any way but "it depends".
Q12 Your categories suggest that this would be better addressed as an open end response.
So to summarize the point that you were actually interested in, Q9 and Q10 are the most susceptible to response bias. Q9 is probably salvageable with some rework but I would drop Q10, even though willingness to pay is always an important piece of information to learn. However, bad data are not better than no data, particularly if you don't know the direction or degree of "badness".
Subject: Re: Re: Maximizing Response and Minimizing Bias