Goals

Here's the running count:

• Chinese (simplified): 94
• Chinese (traditional): 23
• Japanese: 58
• Korean: 21
• English (non-Flash version): 166

I've been contacting various Korean bloggers, but their numbers still seem low. Korea is one of the most wired countries in the world, so I must be doing something wrong. Any ideas? Or do they just play StarCraft with all that bandwidth?

On another note, it's really tough getting the popular bloggers to respond. This is understandable, since they are busy and probably get a whole of bunch of emails, but, nevertheless, it is frustrating. I'll keep trying. I've been posting to blogger forums as well--we'll see how that goes.

Someone asked me what our time frame is and how many participants we need. Well, ideally, we'd like 1,000 participants for each survey. At the very minimum, 500. Maybe I'm being too hopeful...

According to this article in Asia Times Online, "China has seen a significant rise in the number of web logs with an estimated 10,000 active bloggers and more than 600 web logs". Let's use statistics. Suppose I want to be have a 95% confidence level and a confidence interval of 10. Assuming a normal distribution, independent sampling, we need about 100 people to answer the survey. In other words, if 25% of people answered "yes" to a particular question in our survey, we can be 95% sure that 15 to 35% (25-10, 25+10, respectively) of the total population would've said "yes" as well. On the other hand, if I want to reduce that interval to 20 to 30% (an interval of 5), I would need about 400 participants. You can calculate this here and here's an easy to understand slide [powerpoint] about confidence intervals.

As for the time frame, we'd like to get results by the end of August, so that we can analyze them in September. As usual, comments and advice are welcome. Special thanks to Mengjuei for helping to clean up the traditional Chinese version of our blog survey. Zot! Zot!

A rose is a rose is a rose

In email correspondence, Isaac Mao told us:

Basically I'd like to take the survey, however, the Chinese name of "blogger" is not correct. So it will cause many confusion to take it...It never use the "Bo Ke" (your translation of "Blogger"). With this name, people even can't distinguish "Blog" and "Blogger" in context.

I'm not convinced yet that the word or concept of blog is fully mainstream, even in America. I just had a conversation with my mother (who is no slouch with the computer) a few days ago trying to explain what a blog is. If you use words like "online diary", people will start to understand what a blog is, yet blogs come in many forms ("diary" implies "personal", which not all blogs are). Sometimes, for those of us not familiar with blogs in other countries, it is not a trivial task trying to find what the de-facto word for blog is.

In the Chinese case, we looked on the Chinese Wikipedia and choose what seemed to be the word for blog, but we were wrong and Isaac graciously pointed out the error, which we corrected.

I asked Isaac if I could talk about our correspondence on my blog, and he gave me the a-okay. Yet, ironically enough, China has blocked access to *.blogspot.com since 1/2003, so he can't even read about himself. Now, I won't get political here and impose the America-is-all-great-democracy-rules view on other countries, but I humbly believe that such a blanket policy of blocking an entire domain name serves little purpose.

-*-
Our current tally is at:

Chinese: 78
Japanese: 47
Korean: 18

Thank you all for your tips and hints on spreading the survey; keep 'em coming. Please ask people to take the survey--tell them its the cool thing to do now and that if you don't take it your friends will mock you for not joining the bandwagon.

Anyone know who the bloggerati (popular bloggers) in Korea and Japan are?

Implementation

First of all, thanks for all your suggestions on how I can help disseminate the survey further. The advice I've received has been very helpful.

Let's talk about implementation. Our survey was split into two phases. Originally a class project for Quantitative Research Methods in Computer Science, the first phase consisted of creating an English version of the survey. The process of creating the questions is described in my previous entry. When it came down to implementation, due to time constraints, we opted to use a commercial program: Quask FormArtist. It has a nice WYSIWYG GUI for creating surveys. However, it lacks support for Asian languages and has some quirky interface problems (especially when creating macros). The later point, we could live with, but we needed to eventually write Asian versions of the survey.

For phase 2, I implemented the survey in the best language in the whole wide world. For a database, I used SQLite. SQLite is a really nice, lightweight database--if you need a quick and dirty way to store data with ACID transactions, I highly recommend it. There is a ruby binding for SQLite, whose usage is explained beautifully by why the lucky stiff. Basically, we have an Apache server with the modruby module installed that allows ruby scripts to be executed natively (reducing startup times). I use eruby to imbed ruby syntax into an html file which generates surveys from a specially formatted textfile (four versions, one for each language). So, for example, I might have:

I love to:
[] Eat sushi
[] Eat bulgogi
[] Eat Peking duck
[]_ Other:
[]* None of the above

The ruby script file will see that the question is []: in a checkbox format, _: should have a fill in field for "Other:" and *: should make "None of the above" an exclusive answer. A similar syntax is used for radio button formats and fill in questions. Javascript handles things like dynamically greying out the other choices when "None of the above" is chosen.

That in a nutshell is how we implemented the survey. I'm amazed that there isn't some freeware survey making software tool out there (though I did find some freeware services, that unfortunately are very much crippled without payment). Perhaps, when I have time, I'll try to develop a general framework for making surveys. Anyone else have experiences developing surveys?

Bias

Some people [1,2] have commented on the classical sociology problem of self-selection bias. By advertising our survey and relying on the goodwill of people for replies, we are essentially reducing our sample to represent only those people who are willing to do surveys. Does this sample represent bloggers on the whole? That is something we can never claim with absolute certainty. Things like comparing demographics, etc. of the target population with the sample can help support such an assertion.

Surveys have their disadvantages in that they are inflexible. Unstructured interviews and field studies are some alternatives. I recommend you all read Bonnie Nardi's paper on blogging. In that study, she focuses a select group of bloggers, noting their behavior, analyzing their blogs and interviewing them in person. Some of our survey questions are directly gleaned from results of that paper.

Another commenter stated that the survey doesn't take into account for the fact that some people primarily participate in a group blog (is that what they are called?). In other words, several people contribute to a single blog. We had to narrow our focus to just the opinions of bloggers; if we had wanted to find out how group dynamics play a role (how do bloggers of the same group blog differ/relate to each other?), then a different set of questions might've been necessary (like requiring participants to write what blogsite they write on). A separate study on group blogs would certainly be interesting; I suspect there are already studies that have been done on this (any pointers?).
-*-
We're nearing 1000 hits on our survey page. Obviously, we don't have 1000 participants, but I'm amazed at the spike in hits lately. Unfortunately, we are still in desperate need of people to fill out the Chinese, Japanese and Korean surveys. Right now, we have:

China: 7
Japan: 38
Korea: 17

(these don't include Asian language speakers who filled out the English survey)

How and where can I find more Asian bloggers? Do you know any forums, bulletin boards, or particular blogs I can post to? Perhaps I should try pursuing some of the Chinese blog links...

The Trials & Tribulations of Survey Making

Lisa, in a comment , mentioned that

on question #11 -- "does blogging connect me with more people like me" It does, but for me one of the important things about blogging is that it connects me with people *not* like me.

I think she's right. A lot of times bloggers will meet new people they perhaps otherwise would not have met through other means. Perhaps, we should've rephrased the question simply as "I have become more connected with people". That is a different kind of question (though interesting one) from #11.

But let me tell you--survey making is a tough process. And, if you want something interesting, shocking or provocative to come out of a survey and into a nice conference paper, you have to think up of questions that are conducive to interesting analysis. The process was roughly:

1.) We (the graduate students) came up with a myriad of hypotheses that we though were interesting.
2.) From these hypotheses, we developed questions and spoke with professors.

I think, we ended up with nearly 100 candidate questions! They all seemed so good and we each had our "favorite" questions. The most difficult part was in pruning the questions. This was done by:

1.) Each grad student marked out questions they thought could be done away.
2.) We met at a soup and sandwich joint, and went through each question that was marked. We argued and pleaded our case, and then a final vote was done to the question's fate.

Coordinating a project with a large number of participants is a difficult process. However, I think this is more than made up by the stimulation you get from so many different minds.

As with a software project, the design is the toughest part. Implementation is easier (though still tough, I'll talk about that in another post). I'd be interested to hear what other people's processes are when they create surveys. How do you come up with the format? How are initial questions formed? I found that previous surveys were a great help.

On another front, some people have graciously referenced our survey in their blogs:
yuuriblog [jp]
Joho the Blog [en]
Luca Pattaro #weblog# [it]

Anyhow, please do keep spreading the survey around. It'll make our data that much more effective.

Our Blog Survey

Hi, welcome to our blog. Do you write a blog? If so, please take our survey. It is available in English, Chinese, Japanese and Korean. Full details can be found at:

http://drzaius.ics.uci.edu/blogsurvey

It should take at most 15 minutes. Most people finish it sooner, and many of our participants commented that the questions we interesting and enlightening. Plus, since we will publically publish our results, you'll be contributing to a wider understanding of what blogging means to different cultures!

To maximize the validity of our survey results, we need as many responses as possible (ideally several hundred) from as wide range a demographic as possible. Most importantly, we need more results from different countries. Right now, we are lacking responses from non-English speakers. Additionally, we are very interested in responses from English speakers who don't reside in the United States. Of course, we very much welcome those in the US to still fill out the survey. =)

This blog is basically meant to help spread the word. Interestingly, when we asked people to fill out our blog survey, they often asked us back, "do you have a blog?" So here it is--we'll keep you updated soon on how many people have filled out each survey, as well as on our thoughts on the analysis of the data.

Please feel free to leave comments. We are really interested in hearing what you think of our survey. Help propagate the survey through the "blogosphere"--post it in your entries and forward it to your friends. Thanks!