Prospecting the Blogosphere

about the UCI blog survey.

all opinions express herein are only makko's and ocean's, and do not necessarily reflect opinions of any of the other UCI blog survey team members

Monday, August 09, 2004

Principles of Survey Research

If any of you are planning to do a survey, I highly recommend you check out a series of six papers (six in all) from the ACM SIGSOFT Software Engineering Notes:
  1. Principles of Survey Research: Turning Lemons into Lemonade
  2. Principles of Survey Research: Designing a Survey
  3. Principles of Survey Research: Constructing a Survey Instrument
  4. Principles of Survey Research: Questionnaire Evaluation
  5. Principles of Survey Research: Populations and Samples
  6. Principles of Survey Research: Data Analysis
I also find some class slides on the papers. I wish I had read this series before we started working on the survey--it would've saved us a load of headaches.

Let's talk about something specific: Likert Scales. For those of you who are not familiar with them, here's an example:

Indicate to what degree you agree or disagree with the below statement :
1.) I prefer using Linux to Windows.

|Strongly Disagree|Disagree|Neither Agree nor Disagree|Agree|Strongly Agree|

Kitchenham and Pfleegar in part 6 of the series call answers to the above questions ordinal data. Can we treat orindal data as nominal data? In other words, can I simply convert the above (e.g., strongly disagree = 0, strongly agree = 0) example into a number scale and perform the standard ANOVA? ANOVA is a staple of HCI research. It will tell you if you can, with confidence, state that differences between two means are a result of some treatment and not just random effects.

From the paper: "In general, if our data are single peaked and approximately Normal, our risks of misanalysis are low if we convert to numerical values."

So, I guess using F-test + post-hoc tests like Bonferonni are OK to use if the results seem to have some sort of bell shape. What do you do if you don't have such a shape? You can try converting values (multiple/divide by a factor, take the log, etc.) or if the shape is bimodal, trying to split the data further. Any other tips people have? In fact, I haven't really found a paper or book that will tell you--1) first run this test for normality, 2) if it meets this threshold, then you can safely use the F-test. Any pointers? I'm wondering, is a Chi-squared test appropriate for Likert scales as well?


Some minor corrections to our blog were pointed out by moondial [japanese only]. We were inconsistenly spelling the word blog in Japanese. The Japanese language allows one to explictly specify a glottal stop (a sort of hesitation, or pause)...sometimes its hard for me to correctly find whether an English word should have a glottal stop at a certain place when it is translated to Japanese. burogu or burrogu (the double r indicates the glottal stop)? Correct answer: burogu


Blogger Janak India - Surveying Instrument said...

Looking surveying instruments in locality? Janak India offer surveying and positioning equipments in Delhi, Noida. Book order for surveying equipments at Janak India.

7:05 AM  

Post a Comment

<< Home