Often when Questionnaires are drafted, business executives presume that respondents are going to hand in the silver bullets/ magic pills to their business problems.
So if we want to run a channel impact/engagement study to help meet the business objective of increasing sales for Firm X, there is a temptation to just ask the channel partners – “what do you think Firm X should do to increase their channel sales”
If doing business were that simple, then companies would only need to hire low cost panels of respondents, not high cost marketing and business manager, and market research firms.
A good questionnaire needs the precision of science backed by the creativity of art
We need to keep it logical and simple, but at the same time infuse creativity and intuition in questioning methodology. Questionnaires should connect dots (input to insight) rather than just capture isolated dots (information)
In my nearly two decades in marketing research I have seen the emergence of many effective channels for questionnaire administration. However we have made little progress with respect to creatively eliciting better responses.
Most questionnaires are inward focussed– they meet requirements for client, and research agency, whereas they need to be focussed on the respondent.
So, how do we fix this ?
- Keep the questions and potential answers simple
Stay focussed on asking questions that are short and simple. Avoid the temptation to ask “everything.” No long questions with similar multiple choices – respondents shouldn’t feel like it’s an examination where they need to give correct answers.
If it’s even slightly difficult for us to comprehend in the first reading, it is going to be extremely difficult for the average respondent.
- Logic is critical – never sacrifice it at the altar of convenience
Questions need to have a logical flow, one leading to the other, a good questionnaire flow ensures lower respondent fatigue. Making a flowchart and splitting the questionnaire into sections and having a template in place is critical. As a respondent I have faced a survey where I have been asked a detailed set of questions about the brands I use and why. Subsequently I was asked about brand awareness. You can imagine how reliable that awareness data is – and to think that it is a metric driving a possible million dollar business decision.
- Question like a counsellor – stay as neutral as possible
Imagine this question – “What is your health and fitness regime on a day to day basis?”
It assumes first, that the respondent does something for health and fitness, second that it is regimented, and third that it is done daily – look at the amount of pressure we have put on the respondent to be “correct”. His/her answer is unlikely to be “nothing” as he/she doesn’t want to appear to fall short of a strangers’ expectations of him/her.
Lets make this neutral – “do you feel the need to do anything specific to improve fitness ” . This allows the respondent to state the option in his/her own view without fear of being judged
- Not all details we want are worth remembering for the respondent – think short time frame
Don’t challenge respondent memory and expect to get factually correct information. e.g. : “Can you tell us the number of times in the last one year you have turned on the air conditioning in your home” – very tough to recall correctly as the time frame is too long, it’s always better to ask for a shorter duration and extrapolate the number. Be intuitive. Ask yourself first.
- One question, one meaning for all – don’t leave the details to individual interpretation
“When was the last time you visited a fine dining restaurant?” – What could be wrong with this question?
Each individual respondent can have his/her own interpretation of a fine dining restaurant. As researchers we need to seek clarity on what defines fine dining – is it the star rating, the costs involved, Michelin star chefs – precisely what facts constitute this “fine dining experience.” Good questionnaires define every element that is subject to individual interpretation clearly. So a better question could read as
“When was the last time you visited a fine dining restaurant – when I say fine dining I mean one or more of the following – a restaurant with a Michelin star chef, attached to a hotel of 5 stars or higher, individual meal costs in excess of 50 USD per person minus drinks and dessert ?”
- Move from least intrusive to more intrusive – respect privacy
Intrusive questions can put off the most cooperative of respondents – if they are unavoidable to the research push them as close to the end of the questionnaire as possible – at least we lower the risks of having an irate respondent refuse or get upset early into the interview . Financial information is extremely intrusive – please handle asking with care. A simple way to assess – how comfortable are you sharing this information with a stranger? If you are even slightly uncomfortable, chances are that the respondent will be too.
Good questionnaire design involves continuous learning through experience.
“Through learning we re-create ourselves. Through learning we become able to do something we never were able to do. Through learning we reperceive the world and our relationship to it. Through learning we extend our capacity to create, to be part of the generative process of life” –Peter M. Senge-
Let every question be an output of some learning. Make it count as an input to a deeper insight.
