Principles of Survey Research
If any of you are planning to do a survey, I highly recommend you check out a series of six papers (six in all) from the ACM SIGSOFT Software Engineering Notes:
Let's talk about something specific: Likert Scales. For those of you who are not familiar with them, here's an example:
Indicate to what degree you agree or disagree with the below statement :
1.) I prefer using Linux to Windows.
|Strongly Disagree|Disagree|Neither Agree nor Disagree|Agree|Strongly Agree|
Kitchenham and Pfleegar in part 6 of the series call answers to the above questions ordinal data. Can we treat orindal data as nominal data? In other words, can I simply convert the above (e.g., strongly disagree = 0, strongly agree = 0) example into a number scale and perform the standard ANOVA? ANOVA is a staple of HCI research. It will tell you if you can, with confidence, state that differences between two means are a result of some treatment and not just random effects.
From the paper: "In general, if our data are single peaked and approximately Normal, our risks of misanalysis are low if we convert to numerical values."
So, I guess using F-test + post-hoc tests like Bonferonni are OK to use if the results seem to have some sort of bell shape. What do you do if you don't have such a shape? You can try converting values (multiple/divide by a factor, take the log, etc.) or if the shape is bimodal, trying to split the data further. Any other tips people have? In fact, I haven't really found a paper or book that will tell you--1) first run this test for normality, 2) if it meets this threshold, then you can safely use the F-test. Any pointers? I'm wondering, is a Chi-squared test appropriate for Likert scales as well?
- Principles of Survey Research: Turning Lemons into Lemonade
- Principles of Survey Research: Designing a Survey
- Principles of Survey Research: Constructing a Survey Instrument
- Principles of Survey Research: Questionnaire Evaluation
- Principles of Survey Research: Populations and Samples
- Principles of Survey Research: Data Analysis
Let's talk about something specific: Likert Scales. For those of you who are not familiar with them, here's an example:
Indicate to what degree you agree or disagree with the below statement :
1.) I prefer using Linux to Windows.
|Strongly Disagree|Disagree|Neither Agree nor Disagree|Agree|Strongly Agree|
Kitchenham and Pfleegar in part 6 of the series call answers to the above questions ordinal data. Can we treat orindal data as nominal data? In other words, can I simply convert the above (e.g., strongly disagree = 0, strongly agree = 0) example into a number scale and perform the standard ANOVA? ANOVA is a staple of HCI research. It will tell you if you can, with confidence, state that differences between two means are a result of some treatment and not just random effects.
From the paper: "In general, if our data are single peaked and approximately Normal, our risks of misanalysis are low if we convert to numerical values."
So, I guess using F-test + post-hoc tests like Bonferonni are OK to use if the results seem to have some sort of bell shape. What do you do if you don't have such a shape? You can try converting values (multiple/divide by a factor, take the log, etc.) or if the shape is bimodal, trying to split the data further. Any other tips people have? In fact, I haven't really found a paper or book that will tell you--1) first run this test for normality, 2) if it meets this threshold, then you can safely use the F-test. Any pointers? I'm wondering, is a Chi-squared test appropriate for Likert scales as well?
-*-
Some minor corrections to our blog were pointed out by moondial [japanese only]. We were inconsistenly spelling the word blog in Japanese. The Japanese language allows one to explictly specify a glottal stop (a sort of hesitation, or pause)...sometimes its hard for me to correctly find whether an English word should have a glottal stop at a certain place when it is translated to Japanese. burogu or burrogu (the double r indicates the glottal stop)? Correct answer: burogu
0 Comments:
Post a Comment
<< Home