Chapter 3 – Research design and method

Case studies focus on a phenomenon in a ‘real-life’ context (Cohen et al., 2007; Robson, 2011) and aim to gain a detailed understanding of the case, often using multiple methods of data collection (Gray, 2004; Cohen et al., 2007; Robson, 2011; Day Ashley, 2012; Thomas, 2013). Data can be gathered from different facets of the question being examined then “combined to tell [the] finished story” (Thomas, 2013: 150). While different themes and subjects can be examined, the focus of a case study is narrow, detailed and descriptive (Gray, 2004; Cohen et al., 2007; Thomas, 2013) potentially catching unique and illuminating features, which could prove essential for understanding a situation, that might otherwise have been “lost in larger scale data” (Cohen et al., 2007: 256).

It is possible for a single researcher to undertake a case study, and uncontrolled variables and unexpected events can often be incorporated into the study (Cohen et al., 2007).  Case studies are “particularly useful when the researcher is trying to uncover a relationship between a phenomenon and the context in which it is occurring” (Gray, 2004: 124); this seemed appropriate in order “to catch the close up reality” of the classroom situation for my study (Cohen et al., 2007: 254).

I adopted a quasi-experimental, case study as my design frame, as I believed it would be beneficial to make comparisons with a control group; it was hoped that if any changes were noted with the manipulation group, a control group would help indicate whether those changes might be due to the game elements or not.  I decided it would also be beneficial to have two manipulation groups (MG) and two control groups (CG) to increase the chances of finding corroborative or contradictive data.

Participants

Quasi-experiments often use ‘natural’ groupings, such as education groupings (Thomas, 2013): since rearranging my classes was not possible, I used the ‘natural’ class groupings. I tried to take a representative sample of Form 3 by selecting a higher and lower achieving class for both the CGs and MGs^[1]Based on my knowledge of their previous marks and level.. Although student aptitude was not being measured in this study, there could arguably be a connection between enthusiasm and ability; for example, students with a ‘gift’ for ICT might show more enthusiasm than those who struggle with ICT.

In an interpretivist study, there is no expectation that the researcher should be objective and an outsider (Thomas, 2013); as a teacher, I was a participant in my own research.  Mears (2012: 174) states that “subjectivity has its virtues” and a skilled researcher uses “their prior knowledge and experience to good advantage”.  I asked my Form 3 classes to participate in this study since I taught them last year in Form 2; knowing my students helped me identify subtle changes in their behaviour and understand the situation as an ‘insider’. I also believed it would be beneficial if the students had a personal frame of reference for how their ICT lessons were conducted last year to compare with the multiplayer classroom this year.  I had also asked my students last year if they played digital games, which games they preferred, and if they knew anything about MMORPGs.  The most popular games were ‘First Person Shooter’: this influenced my choice of the roles I created (Medic, Trooper and Agent) and the narrative I used to frame the multiplayer classroom (The Battle for Comino).

Ethics

Children have the right to voice their opinions in matters affecting them (BERA, 2011). I asked my students if they wished to participate in the study; although it is mandatory for students to attend lessons, the choice to ‘play’ and suspend disbelief during the lesson was theirs. Written consent was obtained from all legal guardians of the students involved. A phone number and email address were included with explanatory letters in case guardians had any questions; the option of making an appointment with me at school was also provided. Consent was also obtained from the Headmaster and the school’s managerial team. For the sake of anonymity, all nicknames have been changed. All identifiable data gathered will be deleted after the dissertation process.

My position as an authority figure was at the forefront of my mind when asking students if they wished to participate; I made it as clear as possible that there was no obligation to take part, explaining that the same tasks would be given in class whether they participated or not, and they would still have access to online study materials.  When asked, all classes unanimously agreed to participate.  Students were told that they could stop participating at any point, without any negative repercussions, and that since the study was to help me improve their lessons, they should feel at liberty to tell me if they were not enjoying it. Midway through the study (in December) they were asked if they wanted to stop the multiplayer classroom: all chose to continue.

Students were told that the multiplayer classroom would continue until the half-yearly exams, but if they wished to continue after that we would.  Although I planned to finish collecting data in March I believed it would be unfair to end the multiplayer classroom abruptly if students were enjoying and responding to it^[2]All students wished to continue.. After the study I also asked the CG if they wanted to try the multiplayer classroom, in case students had talked amongst themselves during the study, perhaps making them feel left out.^[3]There was no ‘bleed-through’ regarding the multiplayer classroom between Form 3 classes.

Possessing the dual role of teacher and researcher raises issues of bias, with the risk that personal beliefs and expectations may unconsciously influence results.  At the heart of my study is, ‘Will this benefit my students?’ As an independent and self-financed study I was not under any pressure from outside sources, and in order to be of any benefit to my teaching it was in my own interest to be as objective as possible, making a conscious effort to keep an open mind.

The interests of the students were my primary consideration and this study was not at the expense of the curriculum; the current ICT syllabus was adhered to at all times. My teaching explanations, class work set, and student practical time on the computers were the same for the MG and CG.  Any additional technologies used, such as individual remote controls to complete class tests, were used in both the MG and CG.

Both the CG and MG had access to online study materials, such as practice tests and past papers; however, the MG had access to their own personal profiles, where individual XP and badges were displayed.  To ensure that all participants had the opportunity to access the Internet, I opened the computer room during the mid-day break; all students were welcome, but with limited computers it was on a first-come, first-served basis^[4]This was not a new practise; I have been opening the computer room during breaks for 13 years..

The questionnaires were worded as clearly as possible and were completed during class time in case anything needed clarification. I read out the questions and was prepared to translate into Maltese if necessary. Learning Support Assistants were present, as usual, to help students with literacy problems.  Students were informed that the questionnaires were anonymous; it was also pointed out that completing them online meant there would be no way of matching their handwriting with past homework or exam scripts. In elucidating these points, it was hoped that students would answer each question honestly. Students who agreed to participate in the semi-structured interviews were told that they could stop the interview or the recording at any point (Gray, 2004).

Data gathering

Triangulation is one of the main benefits of using a mixed method approach, helping researchers gain insights and in-depth understanding by studying a case from more than one viewpoint (Cohen et al., 2007; Robson, 2011). Data triangulation was used in this study in the form of unstructured class observations, questionnaires, and two semi-structured group interviews.  If the results from multiple methods of data collection correspond with one another, the more confident the researcher will be about the validity of findings; contradictions and tensions would also be highlighted (Gray, 2004; Cohen et al., 2007; Robson, 2011; Day Ashley, 2012; Thomas, 2013).

Researchers should be cautious of relying exclusively on one method of data collection, which could give a misleading or biased view of the situation being investigated (Cohen et al., 2007; Robson, 2011). However, not everyone agrees that triangular techniques guarantee “consistency or replication” (Cohen et al., 2007: 144) and one must remember that although triangulation may reduce the chance of error, “it does not eliminate it” (Gray, 2004: 257).

Observations

Observations^[5]See Appendix 3. can complement data gathered through other methods (Gray, 2004; Robson, 2011). Participant responses in interviews and questionnaires may not always be accurate perceptions of their own true behaviour (Gray, 2004; Robson, 2011); however, viewing these data with data gathered through observations can create a truer picture of events and assist in verifying the veracity and accuracy of responses (Gray, 2004; Robson, 2011; Thomas; 2013).

One of the strengths of observation is that events are observed in ‘real-time’ and in context, providing insights “into interpersonal behaviour and motives” (Gray, 2004: 135); thick description is used to give background information to the reader and place observations in context. Two concerns about participant observation are reactivity – how the observer affects the situation being observed – and reflexivity – how events may be affected and proceed differently because they are being observed (Gray, 2004; Cohen et al., 2007; Robson, 2011). One way to reduce the effect of the observer on the group is for them to be so used to her presence that they “carry on as if she were not there” (Robson, 2011: 317); thus having the dual role of teacher and researcher was another advantage for this kind of research.

Observation involves subjective interpretations and is therefore open to criticism of ‘observer bias’ (Gray, 2004; Cohen et al., 2007; Robson, 2011; Angrosino, 2012). A challenge of participant observation is maintaining a “balance between ‘insider’ and ‘outsider’ status” (Gray, 2004: 242).  Robson (2011: 319) says that possessing an ‘insider role’ may “sound warning bells of subjectivity”, but interpreting the meaning behind participants’ actions may only be possible “through participation with those involved”. My role as teacher was already well-established within the groups, giving me the advantage of being familiar with their ‘symbolic world’ (Gray, 2004; Robson, 2011); familiarity with participants’ language terms and expressions can help to improve accuracy (Gray, 2004).

Observers must “make a conscious effort to distribute [their] attention widely and evenly” (Robson, 2011: 328) as some actions will inevitably stand out more than others.  Expectations may also unconsciously affect what the observer ‘sees’ and interprets, so researchers should not be quick to make judgements but keep an open mind throughout the duration of the study (Robson, 2011). To avoid inaccuracies and ‘selective memory’ it is essential that field notes are promptly written up into a detailed account, and further observations are not conducted until the notes from the previous session have been arranged (Robson, 2011).

Questionnaires

Questionnaires^[6]See Appendix 4. are a useful and versatile data gathering tool which can be used in various types of research designs (Thomas, 2013), providing structured, typically numerical, data which is often “comparatively straightforward to analyse” (Cohen et al., 2007: 317). It was anticipated that questionnaires would provide useful feedback that could be compared and combined with the data from the unstructured class observations and semi-structured interviews.  There is a “lack of interviewer bias” (Gray, 2004: 188) when using questionnaires, and it was hoped that anonymity would encourage participants to be truthful in their responses.

Closed questions provide some structure for respondents and produce responses which are easier for the researcher to analyse; however, closed questions cannot yield the potential richness of responses that open questions can, and researchers may miss the opportunity to gather unexpected responses (Gray, 2004; Cohen et al., 2007). For these reasons, I designed questionnaires that combined closed and open questions; the use of comment fields gave participants the opportunity to add any additional comments and give a more precise, personal answer.

Interviews

At the heart of interviewing is a desire to understand the experiences of others and the meaning they make of their experience (Mears, 2012). Although conducting interviews can be challenging, they ‘lend themselves well’ to be used with other methods (Gray, 2004; Robson, 2011) and are potentially a source of rich data, providing illuminating insights into the opinions and attitudes of the interviewees (Gray, 2004; Robson, 2011). They also allow the researcher to immediately clarify any possible misunderstandings, which is not always possible with questionnaires (Gray, 2004; Robson, 2011).

Open-ended questions may be more difficult to analyse than closed ones but they allow interviewees the freedom to give more in-depth responses with the possibility of yielding unexpected answers (Robson, 2011). Unplanned questions may be added to the schedule to encourage interviewees to elaborate on their initial responses (Robson, 2011), strengthening validity (Gray, 2004). The order and the wording of questions may be modified according to “the flow of the interview” (Robson, 2011: 280) and may even divert from its original path which, although unanticipated, may nonetheless be useful (Gray, 2004).

Sensitivity, personal skills and a good rapport with interviewees is necessary if valuable data is to be collected (Gray, 2004; Cohen et al., 2007; Robson, 2011; Thomas, 2013). Trust and rapport are particularly important to create an environment where interviewees feel at ease to share their thoughts, feelings and opinions (Gray, 2004; Robson, 2011; Mears, 2012), so already having a good rapport with my students was an advantage, as encouraging interviewees to share their opinions can be difficult if they “are not used to talking to strangers” (Hurworth, 2012: 177).

Giving the interviewer what he wants to hear is a problem with interviews (Gray, 2004); before commencing, I told interviewees that criticism was welcome, emphasising how important their honest feedback was. Although I had an interview schedule, interviewees were allowed to speak freely and elaborate wherever they wished. Questions included what they liked and disliked about the multiplayer classroom, what they believed ‘worked’ and did not work well, and their perceptions of any changes to their classmates’ motivation and behaviour, as well as their own.

Joint interviews have the potential risk that interviewees may divert each other’s attention, or one or more interviewees may dominate the interview (Gray, 2004); however, a possible benefit of joint interviews is that interviewees might help each other’s responses by filling in certain details, diminishing the problem of poor recall (Gray, 2004).

The interviews lasted approximately 45 minutes, which appeared long enough to allow interviewees enough time to have their say; ensuring that interviews are sufficiently long enough is important to strengthen validity (Gray, 2004). As Gray (2004: 227) advises, I made notes during the interviews in conjunction with an audio recording, freeing me to concentrate on “listening, interpreting and re-focusing the interview”, ensuring that important details were not missed.

Set-up of the multiplayer classroom

Lessons were designed around Sheldon’s (2012) multiplayer classroom.  The MGs were divided into three teams. Students were allocated roles with ‘powers’, an online profile, and could choose a nickname. My adaptation of the multiplayer classroom is described in detail here^[7]See Appendix 5..

Positive actions were rewarded with XP^[8]See Appendix 6.1.1., ‘special points’ (SP) and ‘power points’ (PP)^[9]See Appendix 6.1.5.

Negative actions resulted in ‘hit points’ (HP)^[10]See Appendix 6.1.3. being deducted. Once the HP level reached zero students had to roll the ‘doom die’ for their punishment.

A ‘random event^[11]See Appendix 6.2.’ was held each lesson, which could have a positive or negative outcome.