Designing my Questionnaire part 2

The following are thoughts and reflections from reading Patterson (2000) book on constructing effective questionnaires 

The purpose of the questionnaire is to collect data of respondents perceptions of atmosphere.  This is primarily exploratory research, hence the use of open ended questions.  I do not want to restrict the breath and variety of the responses, but I still want them to be meaningful.  

Construction questions 

Don’t use double negatives 

Be mindful of the language used in the questions and that people from different background may interpret words in different ways

I plan to only survey participants who reside in the United Kingdom, this will limit need to make to many considerations regarding the language to ensure that the questions are understood.  I must therefore accept that this research will only give me perspectives that from a small section of the English-speaking world.  However, from the results that I have received already have been not particularly focused to useable information.

Effective questions: brief, relevant, unambiguous, specific, and objective

Brief: no more than 20 words. No more than 3 commas.  Limit the number of multi syllabic words.

Relevant: only include questions you plan to analyse, otherwise you are wasting your own time creating them and respondents time is precious to them and they may not complete the questionnaire if there are too many questions. Study participant may also refuse to answer questions they deem irrelevant.

There were suggestions from my group that I should collect demographic data on my respondents. The argument being that I could analyse perceptions over different age ranges and genders. Despite this sounding like prudent data collection advice the intent of this study is to look at differences in perception I have no intention to do a full quantitative analysis as I want this to be a visual exploration not a data exploration. Therefore, I am only going to collect information that is relevant on image atmospherics.

Unambiguous: don’t use word that have multiple meanings or that are unfamiliar.  Ambiguous questions may be interpreted differently therefore affecting the reliability of the study. Closed questions tend to reduce ambiguity.

Considering the advice given in this section it is now apparent to me why my original 3 open ended questions that I tested with two of my friends resulted in very strange results. Trying to define a term in itself is quite ambiguous. I think by adding the preceding closed questions getting people to rate the atmosphere of images will help to reduce some of the ambiguity on defining this term. However, there is a possibility that the images in the preceding part of the questionnaire will fact the open-ended questions. My hope is that this effect will only be in the reduction of ambiguity rather than preloading their perception the term that I wish to define.

Be specific: Ensure that each question is only asking one thing. Don’t be too specific as to force all respondent to answer in the same way or too general to elicit results that cannot be compared.

Again , this confirms the structure of my questionnaire firstly by asking closed questions on the images I've selected and then following it up with open questions on the different areas that they may perceive contribute to the atmosphere.  I am hoping to get a number of adjectives in the open questions that will enable me to compare with the results and hopefully the close questions preceding that will allow four more useful answers in the open questions.

Be objective:  do not ask biased or leading questions 

The problem I find myself in as one of my peers suggested is that I am trying to define something that potentially is undefinable . At the very least it is very subjective, therefore in an attempt to objectify this I have resorted to participants rating a subjective term against concrete objective images.   I am content however that I am not asking any overtly biased or leading questions, hopefully the only effect that the preceding closed questions have is to concentrate the minds on the open questions making them more useful for me to analyse.  From the test that I ran with my class, there was no agreement on what was an atmospheric picture which would imply that I am not leading the participants 

Rating scales 

Looking at the advice on rating scales in “general most rating scales have fewer than twelve categories” with the advice suggesting that minimal additional information can be gained having more 10 or 11.  The advice suggests that any scale between 5:00 have nine categories should suit most research questions . I originally have 5 when I pick my pre testing with my peers I found there was not very much agreement in what was committed atmospheric and a single jump between the five categories represented a bigger shift from the advice in the book I have now opted to go for the higher rating 9 in my scales so that each increment represents let's have a shift therefore if there is any agreement this will become more apparent would be great number of steps in the scale.  I'm happy to have a Newt response that is why I'm okay having nine points so that respondents can be on the fence if they choose.

Rating Anchors

For the closed questions I have opted for bi-polar rating scales going from not atmospheric very atmospheric these are antonyms and therefore making scales uni dimensional. I have elected not to label any of the into meaning points as I do not want these words to affect the way that the images are rated I simply want users to consider the atmospherics and nothing else 

The ratings scale that I will be using will go from left to right with the negative on left and the positive on the right this is due to Google only giving me one option to layout the scale.  The poles have been clearly labelled so hopefully not causing any confusion.  I would have preferred to remove the numbers from the scale entirely so that it would not impact on the writing, but I do not seem to have that option when building the scale. A number a numbered scale may make the participants to start ranking images against one another rather than in the initial statement to write them against their own perception of atmospherics.

After taking into account these considerations I feel the survey is now ready to be sent out to my target audience hopefully giving me some valuable results