In our earlier paper, we laid out a detailed, theory-based, methodology for content analysis, in the hope that it would be a powerful way of studying the quality of learning in group learning situations. When applied to our small-scale experiment, it has produced some interesting findings:

  1. We found more statements indicating critical thinking than the opposite. So the cynical view that no critical thinking takes place in any kind of seminar was not confirmed.

  2. The next worry was that computer conferencing might reduce the critical thinking in seminars. But in fact, the computer conference discussions showed a significantly deeper overall critical thinking ratio than the face-to-face seminars. This was independent of group differences.

  3. However, the students said less in the computer-supported seminars. From thecontent analysis, we do not know why. But we found a negative factor in the factor analysis of student questionnaires, that appears to be related to difficulties of learning and using the computer conferencing technology (Webb).

  4. Is the difference in depth of CT ratio an effect of the different levels of participation? For example, one might expect more deviations and distractions in the longer or more energetic discussions found in the face-to-face seminars, leading to lower ratios. But, if anything, the CT ratios increased with participation.

  5. Apart from the overall depth of critical thinking, the content analysis technique allows us to study different aspects of critical thinking, through the different indicators. We found deeper CT ratios for bringing in outside material and experiences, linking ideas together, and making important points in the computer conference transcripts. But for some groups there were more new ideas in the face-to-face seminars. This reflects a somewhat "worthy" style of messages on the computer conferencing system, somewhat closer to points in an essay than oral conversation.

  6. It is possible to get an idea of the depth of critical thinking taking place in each of Garrison's stages of critical thinking by considering which skillsand indicators would be found in each stage. Doing this, we found deeper critical thinking at all stages in computer conferencing, but with the smallest difference in stage 3, problem exploration. This is in the stage where most creativity is required, including the generation of new ideas. It seems that the computer conference discussions did not stimulate the writing down of new ideas. This may be due to the self-censorship of new ideas before committing finger to keyboard, or less spontaneity at the slower pace of asynchronous computer conferencing.

  7. Stage 5, problem integration, was more affected by the subject discussed than the technology used to support the discussion. Privacy discussions were not brought to a successful conclusion integrating the solutions into the students knowledge.

  8. The content analysis technique allows us to study the effects of other things than just the technology used, such as teaching and learning techniques, and, most noticeably in this case, the subject matter studied.

Finally, where do we go next? Since this was a small sample, of 3 seminar groups of 10-20 students over one semester, similar studies need to be carried out on other classes. The content analysis technique seems to be a powerful way of studying critical thinking in group learning, in particular in the way it allows us to study different aspects of critical thinking and the stages of the critical thinking process. A particularly important issue to investigate is the effect of differing learning tasks upon critical thinking: does all group learning follow the problem-solving approach of Garrison's theory?

From our own results, we are now looking at new combinations of technology and learning task to draw on the strengths found for each technology. For example:

  1. Exploring more easily learned technologies, to try and reduce the negative factor mentioned above, such as later versions of PowWow, and World-Wide Web based discussion systems.

  2. Stimulating the generation of new ideas by bringing together participants with differing experiences who could surprise each other, like students in Northern Ireland and Brazil. This could be supported by synchronous technologies, like Internet Relay Chat, Maven or Cu-See Me, where they cannot be brought together in the same place.

  3. Drawing on the identified strength of computer conferencing in linking ideas together by setting up systems optimised for this, such as ideas mapping software, like CM/1, or group editing environments, such as WebShare, for use by student project groups.

  4. Implementing computer support for proven techniques of encouraging group learning face-to-face, including those mentioned by Gibbs and Jenkins (1992) and creativity techniques (Burnett 1994).