Sunday, June 6, 2010

Systematically searching Usenet

When I first started looking at newsgroups for some evidence of VCoP traits, I just subscribed to different groups, guided by their name, downloaded threads and messages and examined how long threads were, how many participants were involved and how knowledgeable they seemed. Of course, this was far from systematic; and although I did pick up on a few groups that would later turn out to be true VCoPs, I needed a formal procedure to search Usenet and zero in on newsgroups which were good candidates for my research. 

Luckily, I came upon Marc Smith's (1999) excellent study of Usenet and read about Netscan, a newsgroup analyzer he built to map Usenet activity and to gather newsgroup interaction measures. Using Netscan, I was able to obtain interaction statistics from complete hierarchies, such as sci.*, comp.* and misc.*. This gave me an efficient means of comparing hundreds of newsgroups at a time and detecting those that were more likely to display CoP-like traits.

Specifically, I was searching for groups with a high volume of posting, low poster-to-post ratio, low thread-to-post ratio and low percentage of cross-posting. All of these were indicators that a newsgroup was active, with a small core of participants sustaining most of the discussion, and with relatively strong topical focus. Furthermore, I concentrated on newsgroups focused on professional topics, as opposed to hobbies or fan groups.

These selection criteria allowed me to discard a large majority of newsgroups, and focus my search on 41 very good candidates: active groups focused on a professional topic. To further narrow the field, I looked for what I called institutional documents; such as a newsgroup FAQ, a home page, formal posting guidelines, or a moderation policy (for moderated newsgroups). By focusing on newsgroups that had developed high-quality institutional documents, I was able to narrow the field of candidate newsgroups to 19. These I examined further applying a more intensive method, known as Social Network Analysis, which I'll explain in the next post. 

Smith, M. A. (1999). Invisible crowds in cyberspace: mapping the social structure of the Usenet. In M. A. Smith, & P. Kollock (Eds.), Communities in Cyberspace (pp. 195-219). New York: Routledge.

No comments:

Post a Comment