Using content analysis to examine online interactions

In addition to the survey we talked about last time, I wanted to take a close look at interactions in the finalist newsgroups to seek for evidence of the Wenger dimensions. For instance, I wanted to look for textual evidence of collective problem solving among various participants of the newsgroups, as this was a telltale sign of Mutual engagement.

Of course, having one year of messages from 11 newsgroups meant that I needed some sort of sampling strategy in order to build a manageable textual corpus. I did have the advantage of knowing, from the core periphery analysis, who were they key players in each newsgroup. So I decided to sample complete discussion threads with high participation from core members and that addressed professional topics. I analysed 44 threads, containing 1497 messages, using codes that reflected the Wenger dimensions.

I found considerable evidence of the Wenger dimensions, with the exception of Community, which I did not detect in three newsgroups, and Joint enterprise, which I did not detect three others.

In order to be more confident of my results, I triangulated the results of the content analysis with those of the survey. After doing this, it turned out that in four newsgroups, all the Wenger dimensions were present, and each dimension was independently detected by the survey and the content analysis. Hence I classified these four newsgroups as Usenet-based Communities of Practice, since they lacked none of the attributes Wenger (1998) had described for colocated CoPs.

This study comprised my doctoral research, of which I published a summary in the journal Information Research. In my next post I will talk about the current directions I see VCoPs taking.

Using a survey of newsgroup participants to detect CoP traits

It's been a while since my last post, but I took a much-needed summer break plus I was busy finishing a paper that was recently accepted. Last time we were down to 11 very good candidates for virtual communities exhibiting some CoP traits, namely a focus on a profession, a high volume of posting with low poster-to-post ratio, and a good fit of the core-periphery model.

The next step was to design a self-response online survey to ask participants about visible behaviors associated with Wenger's theory of CoPs. After piloting a first version of the questionnaire in a non-participating newsgroup, I published the revised instrument and sent 1706 invitations to distinct participants in the 11 newsgroups. I received 241 responses and was lucky to have 239 usable surveys.

Survey results revealed the presence in four of the newsgroups of all the dimensions Wenger describes for face-to-face CoPs: Mutual engagement, Joint enterprise, Shared repertoire, Community and Learning/Identity acquisition. Hence I concluded these four newsgroups could be considered Usenet-based CoPs.

The paper that came out of this study, co-authored with my doctoral supervisor Dr. David Spicer, was titled "Searching the Usenet network for virtual Communities of Practice". It was presented at the Academy of Management Meeting in 2007, and was selected for the best paper proceedings. It is available in Working Paper format from here.

Fitting core-periphery models to newsgroups

Last time I mentioned that Social Network Analysis (SNA) was a good tool for detecting cohesive subgroups within the online social networks created by the interactions of newsgroup participants over a period of time. What I didn't mention was that, once the data is imported into SNA software, there is a whole family of models that can be applied to this task. 

In my search for VCoPs, I decided to use Borgatti and Everett's (1999) continuous core-periphery model. This accomplished two related objectives. For one thing, I knew that CoPs often adopt a core-periphery structure (Wenger 1998), so I figured zeroing in on newsgroups already displaying such a structure would be a good idea. In addition, the core subgroup of such a structure is a cohesive subgroup in its own right, because core members have strong interaction between them. 

This is easier to see in the interaction plot shown below. It plots messages exchanged during one year between members of a newsgroup I'll call MEDTRAN. Furthermore, in the plot, participants have been sorted in descending order of coreness. Thus, the ones closest to the origin have the highest coreness scores. The plot shows quite vividly that core members, i.e. those with the highest coreness scores, exchange lots of messages among themselves. This makes the core, a high density region of the social network, in other words, a cohesive subgroup.

So in my search among newsgroups that were good VCoP candidates (last time we were down to 19) I included as a selection criterion that such newsgroups would display a good fit of the core-periphery model. This, coupled with the other criteria I mentioned before (good institutional documents, good posting volume, etc.) led me to 11 really good newsgroup candidates. On these groups I performed in-depth examination using a survey and a content analysis of member postings. But these I'll describe in my next post.


Borgatti, S., & Everett, M. (1999). Models of core/periphery structures. Social Networks, 21, 375-395.

Wenger, E. (1998). Communities of practice: learning, meaning, and identity. Cambridge: Cambridge University Press.

Detecting cohesive subgroups through SNA

Social Network Analysis (SNA) is a formal method, or rather a family of methods, that can be used to examine differentiated patterns of interaction between actors. Most social research methods work with attributional data. They measure the individual actor’s personal attributes (such as age, gender, socio-economic status, etc.), then try to group together individuals possessing similar profiles of attributes, and conclude that social behavior is the result of these common attributes. By contrast, SNA works with relational data. It adopts as its unit of analysis the ties or relations between actors, and tries to explain social behavior as a result of the patterns of strong and weak ties between actors, and the resulting constraints to social behavior. Once ties are defined and measured no assumption needs to be made about the spatial position or other characteristics of individual actors.

The fact that SNA only requires relational data made it well-suited to analyze newsgroups, where personal attributes of participants are invisible and not readily disclosed. But one thing that was easy to observe in newsgroups is who posted to who. Even after just following a newsgroup for a couple of days, patterns started to emerge: some people posted a lot, some people's posts were highly commented (for better or for worse), some people seemed to be ignored, etc. Since my research interest was to detect stable communities within newsgroups, I decided I would try to detect subsets of participants that displayed high levels of interaction between them. In other words, using SNA terminology, I was looking for cohesive subgroups, within the total population of the newsgroup.

To do this, I needed a complete listing (or at least a large sample) of messages posted to the newsgroup, and I needed to record all the combinations of one actor replying to another's post. Fortunately, Usenet messages have a standard format that made this task relatively easy. I used the From and References headers of each message. The first one told me who had posted the message; the second one told me to which message it was a follow-up. Using a simple BASIC routine, I imported these headers into an Access database, and then used a query to obtain a listing of all From-To combinations. In effect, this listing was the social network participants had created over a specific period of time through their online interactions. I then imported this into Ucinet, an SNA program, and was able to identify those people who formed the most cohesive subgroups in the 19 newsgroups I was focusing on: in effect who were the most active members of the virtual community in the newsgroup, and just how active they were.

I recently wrote a book chapter describing this technique and included as an appendix the BASIC routine I used. The book will come out later this year, here is the reference:

Murillo, E. (2010) Using social network analysis to guide theoretical sampling in an ethnographic study of a virtual community. In Ben Kei Daniel (Ed.), Handbook of Research on Methods and Techniques for Studying Virtual Communities: Paradigms and Phenomena. Hershey, PA: IGI Global.

Systematically searching Usenet

When I first started looking at newsgroups for some evidence of VCoP traits, I just subscribed to different groups, guided by their name, downloaded threads and messages and examined how long threads were, how many participants were involved and how knowledgeable they seemed. Of course, this was far from systematic; and although I did pick up on a few groups that would later turn out to be true VCoPs, I needed a formal procedure to search Usenet and zero in on newsgroups which were good candidates for my research. 

Luckily, I came upon Marc Smith's (1999) excellent study of Usenet and read about Netscan, a newsgroup analyzer he built to map Usenet activity and to gather newsgroup interaction measures. Using Netscan, I was able to obtain interaction statistics from complete hierarchies, such as sci.*, comp.* and misc.*. This gave me an efficient means of comparing hundreds of newsgroups at a time and detecting those that were more likely to display CoP-like traits.

Specifically, I was searching for groups with a high volume of posting, low poster-to-post ratio, low thread-to-post ratio and low percentage of cross-posting. All of these were indicators that a newsgroup was active, with a small core of participants sustaining most of the discussion, and with relatively strong topical focus. Furthermore, I concentrated on newsgroups focused on professional topics, as opposed to hobbies or fan groups.

These selection criteria allowed me to discard a large majority of newsgroups, and focus my search on 41 very good candidates: active groups focused on a professional topic. To further narrow the field, I looked for what I called institutional documents; such as a newsgroup FAQ, a home page, formal posting guidelines, or a moderation policy (for moderated newsgroups). By focusing on newsgroups that had developed high-quality institutional documents, I was able to narrow the field of candidate newsgroups to 19. These I examined further applying a more intensive method, known as Social Network Analysis, which I'll explain in the next post. 

Smith, M. A. (1999). Invisible crowds in cyberspace: mapping the social structure of the Usenet. In M. A. Smith, & P. Kollock (Eds.), Communities in Cyberspace (pp. 195-219). New York: Routledge.

How I started searching for VCoPs

I first started thinking about virtual or Internet-based Communities of Practice (CoPs) in late 2000. At the time I was finishing a master's degree in Information Technology Management at ITESM, in Mexico City, and I was about to embark on a six-year part-time distance-learning PhD in Management at the University of Bradford. I had a full time job, as the systems person of a small organization, and they had agreed to sponsor my PhD and allow me a couple of afternoons to work on the dissertation.

I was lucky to begin my master's in 1996, a time when you connected to the Internet by modem, and e-mail was just starting (in fact I introduced it in my organization). Furthermore, ITESM was at the time a pioneer of Internet-based degrees. The one I studied had a mixed mode of delivery: I attended a weekly lecture which was broadcast by satellite from Monterrey to all ITESM campuses, and did all other coursework and course interactions through the Internet. So this gave me an early feel for the possibilities and limitations online discussion groups and distributed teamwork.

At ITESM I took a couple of courses in Knowledge Management, and one topic that I found very interesting was CoPs. I first read about them in Tom Stewart's excellent book Intellectual Capital. It's a bit dated now, but it conveys, in a very engaging way, the excitement of those early days of the Information Revolution. Stewart did not invent the CoP concept, but was very enthusiastic about it. It was then that I first thought that the Internet would be a good place to search for like-minded colleagues to form a "virtual" peer group.

Later I read works by Etienne Wenger, who with Jean Lave first proposed the CoP concept back in 1991. The groups they described were all face-to-face, but I didn't see any reason why this couldn't take place online, at least for some kinds of knowledge work. So when I started thinking about doing a PhD, I thought that proving CoPs could function effectively over the Internet would be a good topic. On my first trip to Bradford (I went every year for a couple of weeks) I was lucky to connect with Dr. David Spicer, who believed in the topic from the start, and became my supervisor. And so, I began my research by trying to locate existing CoPs working over the Internet. This soon led me to the Usenet network, where discussion groups were named and hierarchically organized according to their topical focus. At the time, there were about 80 thousand newsgroups, and I felt sure there ought to be some successful Usenet-based CoPs among them. However, I had to devise a systematic procedure to evaluate newsgroups and narrow my search. But this I will describe in my next post.

Starting a research diary

The motivation to start this blog came from Andrew McAfee's excellent book Enterprise 2.0. Just this morning I read that blogs are well suited for converting what are merely potential ties into actual working ties (p. 111). So it occurred to me that writing a public research diary was a good way to put my thoughts into writing (and thus stop thinking about them after work), and might lead me to people thinking along the same lines. Another reason is that I needed more space to share news and ideas about my research, than what is provided in the status update at Academia.edu. So I will put a link from my profile at Academia.edu to this blog.

I work at ITAM Business School in Mexico City. My research is on Virtual Communities of Practice (VCoPs). This is a topic of great interest to people who don't have a local peer group that shares their very particular professional interest. This happens to be my case; seeking VCoPs has been a personal quest since I started my PhD in 2000, and continues to this day.

Currently I have three papers on the works, but I'll talk about those in forthcoming posts.