With this background therefore, MRS was asked, in Spring 2003, to assist with the formulation of a draft e-learning strategy for the West Yorkshire Learning And Skills Council. One tactic that the company wished to employ was to canvas the views of the movers and shakers within the sector, representing leading organisations and colleges throughout the region - all of whom had plenty to say about the subject, but had strict demands on their time.

Given the option of undertaking traditional research methodologies of either face to face or telephone interviews, or indeed physical focus groups in a central location, MRS decided to carry out the research using innovative eFocusgroup technology through the e-console. This resolved the issues of time and location by providing a convenient, virtual and remote solution, whilst encouraging the use of e-technology, itself at the very heart of the research topic.

Being heavily involved themselves in e-learning and therefore participants in the research, MRS chose to have the e-focus group moderated by an experienced eFocusgroup moderator to provide absolute objectivity. Lorraine Davidson's details are available here.

The e-group was conducted over a 9 day period in May 2003. Interest in the group was extremely high with some 23 people signed up to participate. Of these, 18 members were found to be active throughout the discussion, with participants being key personnel in colleges or organisations involved in e-learning. This high level of participation was judged to be excellent and it was recognised that it could not have been matched by a physical focus group.

The e-moderator developed a discussion guide from the client brief and this was divided into four topic areas that were phased in throughout the duration of the group. These topics asked for feedback on participants level of involvement with and definitions of e-learning; the type and success level of current e-learning provision in the region; gaps on the demand side amongst potential users of e-learning ; and, importantly, recommendations for the development of a regional e-learning strategy.

Overall, some 67 issues were raised throughout the duration of the group by participants and these were subsequently formed into an online questionnaire, itself customised to the MRS corporate colours. This online questionnaire facility, developed automatically from issues raised by the group itself, is a unique feature of the e-focus group technology and differentiates it from traditional research methodologies.

Encouraged by the e-moderator, some 16 members responded to this questionnaire and participants were asked to state their level of agreement with the issues based on a scale of Strongly Agree to Strongly Disagree. Then, a numerical analysis was used to aggregate responses and to place issues in descending order of strength of agreement. Detailed graphical analysis was further produced to understand individual responses behind each issue. For example, the following is one of the issues that was actually included in the questionnaire and highlights how individual responses are anonymised through a unique user ID. It further demonstrates how it is possible to gain a deeper understanding of all responses behind each issue, rather than an aggregated group score.

Feedback from MRS, who commissioned the work, suggests that the eFocusgroup was successful both in terms of the numbers of responses and secondly in providing useful indicators for contributing towards the preparation of a draft e-learning strategy for West Yorkshire.

Margaret Simmonds, Director of the MRS Consultancy has this to say about use of the eFocusgroup technology:

Reproduced below are anonymised details of another eFocusgroup conducted by independent market research firm Freelance Marketing.  These comments are unedited but there are some accompanying notes inserted in blue italics.

Project : Charity dealing with conservation (amongst other things),  with widely dispersed customer base. Pilot undertaken in July/ August 2002. (This was the first experimental eFocusgroup)

Given the large geographical area covered by the organisation, it was potentially prohibitive both in terms of cost and travel time to gather customers together for a physical focus group.

The format of the eFocusgroup therefore appeared to be ideal in presenting a forum for open discussion. Participants could contribute their views from the comfort of their own locations, as long as they had access to a PC and e-mail.

Key objectives were to explore how the agency could add value to the activities that it is involved in, covering feedback on performance on delivered projects, as well as helping to evaluate current effectiveness and future direction.

A number of "customers" were contacted, that had been involved in tangible live or completed projects, therefore able to speak from direct experience. This resulted in 12 participants being identified. The first problem, given the fact that this was run over the Summer, was trying to find a mutually convenient date for all interested parties. Two groups therefore had to be run. In total, there were 5 active participants in the first group and 2 in the second, despite 12 originally stating that they would be available.

Several key issues were raised and formed into an online questionnaire to which 6 people replied. This did however require some moderator chasing.

Feedback from the client suggests that :

Learning Points for eFocusgroups

As a pilot exercise with an actual client, key learning points for eFocusgroups were as follows:

The method of summarising questionnaire results numerically has since been revised to provide a more qualitative, rather than quantitative focus. Graphical analysis and percentages can sometimes be misleading given the small base of participants and an alternative visual presentation of results by issue is now being looked at (This has now been incorporated.  Also a 'drill-down' feature has been included for experienced researchers to take a closer look at individual responses to particular questions).

From the perspective of a trained moderator of physical focus groups, there are some interesting comparisons between the two types of forum. Effectively, they both have their niche and should be seen as complementary methods of undertaking research.

eFocusgroups are not intended to provide the statistical validity of large scale quantitative studies and are essentially qualitative in nature, with a small member base. Rather, they should be used to probe discussion points and then to illustrate strength of agreement or disagreement with particular issues as indicators of areas for further exploration or indeed for widespread quantitative research.

They effectively provide an alternative or a complement to other research methods. It is important to understand that the benefits of this type of e-forum need to be sold to potential clients, as a convenient, niche, alternative research methodology, rather than the features of the e-technology per se, which may actually be off-putting to some more traditional industries.

Moderator Tips

It must be remembered that since members of eFocusgroups are working remotely, with other distractions such as a busy office environment around them, they may need reminders to reply to discussion areas. In a physical focus group, the moderator would be sat in front of the participants and eye contact would be possible, but in an eFocusgroup, messages or possibly telephone (if you have permission) are the only option to motivate them to reply. However, the moderator must apply some skill and judgement here to strike a balance between providing sufficient incentive, whilst not hounding them. A sense of humour helps! (an area of the emoderator's console has now been built to provide tools for monitoring participant contribution and making this 'chasing' process much easier)

eFocusgroups should not therefore be seen as a panacea for obtaining views from group members who don’t really want to make comments! You can’t just plug them in and expect the conversation to flow! This can sometimes prove frustrating for the moderator when signed up group members remain quiet whereas in a face to face group they would be actively encouraged to contribute. Without a response, it is also difficult to tell if group members have actually received e-mails, although experience has shown that they have and are considering their responses.  (e-moderation is a skill in itself and our recommendation is that moderators who have not undertaken this kind of work before, should allow an experienced emoderator to shadow their first eFocusgroup)

Structure should be provided for the format of the group. It is recommended that the moderator prepares a discussion guide, as in a physical group and then divides this into a series of discussion points or topics which can be gradually introduced. This still allows for spontaneous discussion to take place but provides some direction for the group. It also enables the moderator to gain an understanding as to when each topic should be introduced over the scheduled duration of the group to ensure that there is adequate time to discuss each issue. Reminder notes can then be sent to participants to ask them to reply to particular topics before the next one is posted.

Lorraine Davidson

Freelance Marketing

[ Products ] [ Consultancy ] [ Psychology ] [ Security etc ] [ Latest news ] [ Comparison ] [ Case studies ] [ Prices ] [ Order ] [ About us ] [ Special offer ] [ eSurveys ]