Factors that improve student retention in school band programs

Hele tekst

(1)Factors that Improve Student Retention in School Band Programs. by Wyn Vallis Humphries B.Mus., The King’s University College, 2002 B.Ed., University of Lethbridge, 2004. A Project Submitted in Partial Fulfillment of the Requirements for the Degree of MASTER OF EDUCATION in the area of Music Education Department of Curriculum and Instruction. © Wyn Vallis Humphries, 2012 University of Victoria All rights reserved. This project may not be reproduced, in whole or in part, by photocopy or other means, without permission of the author..

(2) ii Supervisory Committee. Factors that Improve Student Retention in School Band Programs by Wyn Vallis Humphries B.Mus., The King’s University College, 2002 B.Ed., University of Lethbridge, 2004. Supervisory Committee Dr. Mary A. Kennedy, Supervisor (Department of Curriculum and Instruction) Dr. Garry Froese, Committee Member (Department of Curriculum and Instruction).

(3) iii. Abstract Supervisory Committee Dr. Mary A. Kennedy, Supervisor (Department of Curriculum and Instruction) Dr. Garry Froese, Committee Member (Department of Curriculum and Instruction) This study explored student retention in school band programs by means of a survey. Band teachers from Edmonton Public and Elk Island Public Schools were asked to provide information about their beliefs, values, opinions, as well as current demographic information. Topics of the survey included student factors, and teacher factors such as band assignments, performances, motivational strategies, and administration. Data were analyzed using the online program Survey Gizmo. Examination of the data confirmed that there are common band teacher strategies that improve student retention. Some key strategies are involving students in repertoire selection, being flexible on instrument changes, earning trip and tour privileges for practicing required material, assessing progress in the form of playing tests, scheduling frequent performances, administrator attendance at performances, frequent encouraging comments about progress, and planned listening and performance trips..

(4) iv Table of Contents Abstract ..................................................................................................................... iii Table of Contents ..................................................................................................... iv Acknowledgements ................................................................................................... v Dedication ................................................................................................................. vi. CHAPTER ONE: Introduction ............................................................................. 1 Rationale ................................................................................................................... 1 Statement of Purpose ............................................................................................... 7 Delimitations ............................................................................................................. 7 Limitations ................................................................................................................. 7 Assumptions ............................................................................................................. 8 Organization and Overview of the Remaining Chapters........................................... 8. CHAPTER TWO: Review of the Literature ...................................................... 10 Introduction ............................................................................................................. 10 Teacher Factors ...................................................................................................... 13 Administration Factors ............................................................................................ 15 Social Factors ......................................................................................................... 18 Summary ................................................................................................................ 20 Introduction ............................................................................................................. 22 Rationale for Conducting Quantitative Research .................................................... 22 Research Design .................................................................................................... 24 Survey Instrument................................................................................................... 25 Choosing Survey Gizmo ......................................................................................... 26 Ethical Considerations ............................................................................................ 27 Sample .................................................................................................................... 28 Procedures ............................................................................................................. 29 Data Analysis .......................................................................................................... 30 Summary ................................................................................................................ 32. CHAPTER FOUR: Results ................................................................................ 33 Introduction ............................................................................................................. 33 Recruiting Band Teachers ...................................................................................... 33 Quantitative Results Part I ...................................................................................... 34 Qualitative Results .................................................................................................. 47 Quantitative Results Part II ..................................................................................... 50 Discussion .............................................................................................................. 52 Summary ................................................................................................................ 61. CHAPTER FIVE: Coda....................................................................................... 63 Introduction ............................................................................................................. 63 Survey Conclusions ................................................................................................ 63 Implications for Practice .......................................................................................... 64 What I Learned ....................................................................................................... 68 Future Research ..................................................................................................... 71 Concluding Thoughts .............................................................................................. 72. References ......................................................................................................... 74 Appendix A: Letter to Superintendents ....................................................................... 80 Appendix B: Letter to Principals .................................................................................. 81 Appendix C: Participant Email Invitation ..................................................................... 82 Appendix D: Survey Instrument .................................................................................. 83.

(5) v Acknowledgements I would like to acknowledge the students in my cohort for the time they have spent with me allowing me to share ideas with them. I would also like to recognize the professors who assisted me in my journey: Dr. Kennedy, Dr. Bolden, and Dr. Froese who have been encouraging and helpful throughout the master’s program. I have also had influential and inspiring teachers and mentors from childhood to the present who must be mentioned: Scott Rogal, Murray Smith, Dr. Kobie Kloppers, Russell Whitehead, Laurelie Natress, Shannon Bolichowski, Johan Brinkman, Dr. Ed Wasiak, Dr. Tom Staples, and Dr. Angela Schroeder. I thank them all for their advice and tips that have helped me to improve as a music educator..

(6) vi. Dedication I would like to dedicate this report to my family, my friends, and especially, to my wife, Bethany. I will always remember the amazing support that they provided me. Thank you everyone for pushing me to do my homework. You are fantastic people!.

(7) CHAPTER ONE: Introduction Rationale Since beginning my career as an instrumental music educator I have wondered why certain band programs consistently enroll large numbers of students every year while others struggle to maintain even a minimum number. In particular I have asked myself: How do teachers maintain student interest through to the second year of study? What strategies would help teachers increase enrolment? In my experience, interactions with students, administrators, parents, mentors and colleagues have inspired me to find answers to these questions. More recently, reflection on my own teaching practice has propelled me to focus on this issue. Listening to students and their needs has been a priority since I started teaching seven years ago. Unavoidably, I have witnessed student interest change over time. Experience has shown me that numbers usually decline between the first and second years of study, and less so between the second and third years. When students begin band they are excited about learning to play an instrument. Most students maintain a love for their chosen instrument until at least spring break, when interest begins to wane. Maintaining student interest into the second year is more challenging than the initial attraction to band. To explain: Each year after spring break, some students request to change instruments or to take another course, especially in schools where there are several options to choose from other than band such as drama, foods, photography, outdoor sports, art, hockey, etc. Hearing student.

(8) 2 comments such as, “I want to change instruments,” or “ I want to quit band,” or “ I want to take another course” caused me to question and reflect until I was out of appropriate responses. Feeling somewhat powerless and running out of answers, I turned to my administrators for advice. Fortunately, my first set of administrators,1 who were previous band teachers, hired for a full-time music position prior to graduating from university. Unlike most beginning music teachers, my assignment was entirely music courses and the administration and I worked together to try new ways of maintaining student interest into the second year of study. However, supportive administration of music programs cannot always be sustained. Due to the transient nature of the location in which I worked, the next three school principals had short terms at my school. It was challenging to maintain the team approach needed to grow a band program because of the ‘revolving door’ syndrome affecting important team players, such as my administrators. Unfortunately both for the band program and me, two of these arts-informed administrators were promoted to positions outside of my school in my third and fourth years of teaching. As a result, my ‘dream’ assignment was shortlived. Working in a team with a great administrator is wonderful, but not always possible. However, funding for the music program is also an important element in creating a ‘dream’ band-teaching situation. Preferred administrator assistance, however, does not mean simply the receipt of funds required to run a band program successfully. Admittedly, sufficient funds are necessary for supplies such as reeds, lubrication, and 1. Administrators include the principal, the vice-principal, the superintendent and his/her deputy..

(9) 3 repairs, and the allocation of funds to finance trips and uniforms is a real blessing to any music teacher. But ideal administrative assistance involves support of the band teacher in other ways also, ways which can assist a teacher to achieve program goals. Aside from their other duties, supportive administrators listen to band teachers and their concerns and discuss ways of improving enrolment and retention within the school band program. This is what I experienced as I established my program. Administrators can support band programs, which usually help growth and development, but they can also hinder a program. Although band teachers do not choose their administrators, I believe that administrators can affect the retention of students (positively or negatively). The same is true of parents. I have always had a supportive parent group, which was much stronger in numbers in my first few years of teaching. I have found that parents are more supportive of a band program that the administration supports. In my second to fourth years of teaching, I remember parents expressing their pride and happiness both in the success of my program and the musical growth of the students. Before I began teaching, I remember hearing parents complain about the fact that high school music courses had not been offered for more than fifteen years in the town in which I started teaching. Since I was teaching at the middle school, parents would tell me that I should not let it bother me, but I was very upset. I wondered if the parents might have had more of an impact if they had approached the high school principal in a large group and asked questions. Instead, I heard their questions and concerns at our monthly.

(10) 4 band parent meetings. Most of the meetings were productive with positive and program-focused discussions. Informal conversations with parents before and after the meetings, however, further informed me about issues related to student retention. Our conversations were centered on their children and how they were either interested or losing interest. Some were enrolled, and, according to them, some were going to continue to take band in the future. This communication between the parents and myself had a huge impact on me. Some of their comments have remained with me to this day. For example, I remember one parent remarking that his children loved to play ‘upbeat popular’ and modern music. Another told me that his children really enjoyed the band trips and the friends made in band class. Other parents claimed that when a child has siblings in band he/she is more likely to join. These comments contributed to my growing interest in the retention of students in school band programs. Listening to the parents of my students helped me to win their support. The more I learned about the children within the community, the better the potential was of growing my program. While parents assisted in developing my awareness of the children that I was teaching, visits and conversations with mentors and colleagues also impacted my interest in student retention in band. It was these visits and conversations that provoked me to consider situations different than my own. I remember one conversation with a colleague at an Alberta Music Teachers’ Conference. She and I were discussing how we felt about our current band enrolment statistics. Disappointed with her situation she asked, “What is it that hockey programs have, that band doesn’t have?” This discussion propelled.

(11) 5 me to find out more about maintaining numbers in band programs. In many ways, my mentors and colleagues face similar issues related to retention of band students. Several mentors and colleagues have experienced the challenges of attrition. Many of them have made suggestions for encouraging students to continue in school band programs. Unfortunately suggestions are often context dependent—that is to say, what works in one situation may not work in another. When I asked colleagues how they manage to keep their numbers strong and what strategies work more effectively than others, I received a variety of responses. A few examples follow. Some band teachers told me that they felt compelled to offer at least one overnight trip per year for beginners. Teachers of smaller programs have told me that their second and third year band students look forward to the major trip of one or two nights more than anything else in band. It sounded like a good tactic for attracting students and ensuring that they remain in the program. In addition, some of my mentors and colleagues claim that friends are a big influence in the process. I concur, because I have found that several of my second and third year band students sign up for band because their friends do. I, however, believe that these incentives are only short term. As enjoyable as trips and friends are, I, like others, want my students to continue in band for the enjoyment of music. Further I hope that my students, their parents, and administrators will appreciate and/or take advantage of the proven benefits of music study such as brain development, spatial reasoning skills, and self-confidence. I want to find solutions that are applicable to.

(12) 6 building interest in a meaningful way. Informed by communications with my students, administrators, parents, mentors, and colleagues, I decided to reflect on my teaching practice. I began by asking myself various questions. Do I provide enough successful experiences and quality opportunities for my students? Have my lessons shown my students a connection to their lives through the music that they are creating? Do I transmit my passion and love for music in a way to which they can relate? How can I solve the problem of high attrition rates in my band program? While I recognize the importance of recruiting in a band program, this investigation focuses on the retention of students. Let me explain. I know that band teachers should try to provide successful experiences for their students. Assuming this is a priority for music educators then they should consider also the effectiveness of their retention practices. Further, if band teachers are concerned with building their band programs, then they should reflect on issues of retention. Similar to my situation, they cannot help but talk to students, administrators, mentors and colleagues, and reflect on how the above comments and questions relate to their individual situations. Many teachers would appreciate insights for strengthening their school band program. For the above reasons, I became interested in studying the factors that improve student retention in school band programs..

(13) 7 Statement of Purpose Therefore, the purpose of the study was to investigate the factors that improve student retention in school band programs. In particular, the study explored the factors that improve retention between the first and second years of band instruction. Student factors, teacher factors, administration factors and social factors were investigated. I reasoned that the survey method would be the most appropriate design for my investigation because I would learn about the values, attitudes, and beliefs of other band teachers in my geographical area. Delimitations In order to narrow this project to a manageable size, participants were limited to band teachers in Edmonton Public and Elk Island Public School divisions in Alberta. I teach in Elk Island Public School division and I was particularly interested in learning about the beliefs and programs of my colleagues. Invited participants were qualified music teacher specialists who possessed at least one of the following: BMus, BEd, ARCT, EEd, MMus, MA, MEd, or a PhD. In addition, participants were restricted to those with at least three years teaching experience, who were currently teaching at least two levels of band. Criteria were established to ensure a higher validity of responses. Limitations There were some expected and often uncontrollable challenges. These included the number of completed surveys received, the number of.

(14) 8 incomplete surveys received, and the honesty of the participants. Another was the computer literacy of the participants. Finally, despite making the survey iPhone friendly and compatible with Mac and PC computers, some participants did not have time or even access to the Internet (Dillman, 2000). Assumptions In designing and conducting this study, I assumed that band teachers other than myself were experiencing similar issues related to student retention. In addition, I assumed that the results of this study would benefit not only the student retention in my own band classes, but also the retention practices of other school band teachers. I assumed that my survey instrument was well designed, and that it would measure the opinions, values, and beliefs of band teachers appropriately and adequately. Jaeger (1997) points out two more significant assumptions that I needed to consider in terms of ‘valid substantive generalization.’ They were, “ …(a) that the respondents understood the questions that were asked in the survey, and (b) that the respondents’ interpretations of the questions were consistent with those intended by the survey researcher…” (p. 473). Organization and Overview of the Remaining Chapters There are five chapters in this project. The study is based on survey responses collected from Edmonton and Elk Island, Alberta area band teachers. In Chapter I, I presented the rationale and purpose of the study. Chapter II provides a review of literature associated with student retention in band. The study’s methodology is presented in Chapter III, defining survey.

(15) 9 research and addressing how it has been applied to this study. Results are presented, analyzed, and discussed in Chapter IV. Chapter V concludes the study by summarizing the project and proposing directions for future research. References and appendices, including a list of survey questions follow the final chapter..

(16) 10. CHAPTER TWO: Review of the Literature Introduction Most interesting in the realm of music education and research is the challenge of retaining students in an instrumental music program. Many band directors face this problem from year to year and some are unaware of how to proceed in finding a solution. Although there is plenty of research on student attrition, there is far less research on the issue of student retention in school bands. Hartley (1996) defines ‘retention’ as, “instrumental music students who, once started, remained enrolled in the band program” (p. 306). The recent literature addresses a limited number of reasons why students decide not to continue taking band. Klinedinst (1991) described these reasons as peer influence, external scheduling conflicts, student-teacher relationships, and family (p. 235). Sichivista (2007) lists the internal and external factors that have an impact on students’ motivation and found that “external factors, such as support from parents, teachers, and peers, strongly and positively influence students’ persistence” (p. 55). The review that follows will examine the extant research focussing on four factors that improve student retention in band programs: student factors, teacher factors, administration factors, and social factors. The chapter will conclude with a summary of the research examined..

(17) 11 Student Factors Gouzouasis (2010) suggests that, “ …students will be equally excited about playing a band instrument regardless of whether they have had one year or two years in band. . .” (p 79). However, student interest may change as students grow and develop. In an article offering suggestions on fighting attrition, Sandene (1994) affirms Gouzouasis’ findings reporting that: Initial interest and enthusiasm may wane until [students] decide that other school activities look more fun or interesting; some students may even experience peer pressure to withdraw from a given ensemble or program. As a result of a combination of any number of these factors, it is at this stage (intermediate level) that the student and the parents may be most likely to decide that the student should withdraw from the school music program. (p. 32) Warnock (2005) hypothesized that “attraction [has] a significant influence on determining participation in instrumental music but that the decision to stay in a music program is dependent on achievement…” (pp. 45-46). Regarding achievement, Sichivista (2007) claimed that “ …students tend to avoid tasks that have previously resulted in failure but are willing to engage in activities that have previously resulted in success, even when those activities appear to be challenging …” (p. 57). She asserted that academic success had a strong positive effect on students’ value of music (Sichivista, 2007). Schmidt (2005) agrees with this view stating that “younger students valued music more when they thought they were doing well in the music class and were living up to the.

(18) 12 expectations of the teacher” (p. 63). Connecting the influence of achievement to retention, Klinedinst (1991) found that although “…validity of predicting performance achievement after less than one school year of instruction may be questionable …there is a significant relationship between academic achievement and student retention…” (p. 236). Moving to the influence of student attitudes, Warnock (2005) suggested “...with younger students, positive relationships exist between interests in music and music participation” (p. 44). His study considered how attitudes affect behaviour when joining a musical group. Warnock hypothesized that “Musical reasons, based on interests in music and musical activities, and selfconcepts in musical ability, should predict attraction to participation” (p. 48). Attribution theory is helpful in the area of retention. Nierman and Viek (1997) explain that: Attribution theory suggests that the tendency to approach a new task is influenced by the probability of success and the attractiveness of achieving it. A student who has a high musical aptitude should be more attracted to the challenge of playing an instrument because there should be a good chance that he/she will be successful (p. 382). In terms of aptitude, Nierman and Viek (1997) conducted a study that led them to find that “attitudes and musical aptitude are essentially unrelated” (p. 386). It is important, however, to note the value of the attitudes (McLaughlin, 2007) of one’s students, who will almost always point out the need for change and improvement. When a student knows or is made aware.

(19) 13 of musical abilities, there is a higher probability of the student being attracted to instrumental study (Nierman & Viek, 1997, p. 387). Teacher Factors Moving to teacher factors, generally music teachers are able to control their classroom environment, teaching styles, planning and delivery of curriculum, evaluation of students, and communication. To improve retention in their band program, teachers can adjust these factors as necessary to meet student needs. Supporting this idea, Campbell, Connell, & Beegle (2007) claim that student adolescents “…long for more provisions for the study of music that is relevant to their needs, their interests, and their hopes and desires” (p. 234). Teachers should ensure that they plan and deliver effective and applicable content to students that will encourage them to want to keep attending music classes. For example, Sichivista proposed that “ …music teachers occupy a unique place in the students’ lives because of the opportunities [they have] to create more meaningful musical learning for students by combining in-school and out-of-school musical experiences and by involving parents…” (p. 56). On the other hand McCarthy (1980) concluded that teachers “had only a minimal effect on their students’ attrition…” (p. 67). He (1980) investigated the effect of individual instruction within instrumental music classrooms on attrition from the music program and found that of the 1199 students who took surveys, “…23% were identified as dropouts” and that “…differences in teachers increased dropout predictability by 1.1% …” (p. 67). Although McCarthy’s.

(20) 14 study did not focus on student retention, results showed that variations of private and small group teaching situations only indicated a small percentage of dropout predictability. Individualized instruction is only one way to provide further student success and improve retention rates. There are many other strategies as well that band teachers can use to increase student retention. Katzenmoyer (2007) recommends that band teachers welcome new students, encourage frustrated students to try several new instruments, recognize student strengths, make classes fun, accommodate and help solve logistical problems (transportation of instruments), help students find time to practice, and strengthen relationships by displaying empathy (pp. 37-40). Katzenmoyer also claims that “ …no single factor explains why students quit instrumental music, [and] knowing what the possible reasons might be will help music teachers increase retention rates …” (p. 40). In addition Mixon (2005) emphasizes “…[that] building collegial relationships by getting involved in other classrooms and activities is invaluable for securing time for [the] program” (p. 18). This idea is also echoed by Sandene, (1994), who offers a list of techniques for teachers trying to improve retention: 1 – Publicize the achievement of groups and individuals for school staff members to see. 2 – Get to know classroom teachers personally. 3 – Make sure there are no potential class conflicts that prevent students from continuing in [the] program..

(21) 15 4 – Make a special effort to contact and get to know parents. 5 – Don’t make parents wait until the first report cards are mailed out or until the first parent-teacher conferences are held to inform them of their child’s progress. 6 – Explain to students that they may experience periods of frustration and encourage them to persevere through difficult tasks. 7 – Allow students to review material they may have already mastered. 8 – Schedule a performance early in the year. (p. 33) Teacher evaluation is another way to influence a student’s decision to continue studies in instrumental music. Corenblum and Marshall (1998) conducted a study on grade 9 band students in Winnipeg to predict their intentions to take band in high school. The students were surveyed voluntarily about parental attitudes, student attitudes, teacher attitudes, and extracurricular musical interests. One of their hypotheses proved correct: “Teacher evaluations and grades should directly predict student intentions to continue; that is, positive evaluations should be associated with intentions to remain in the program …” (p. 131). They found that students’ intentions were predicted less so by course grades and more so by teacher evaluations. Corenblum and Marshall (1998) also emphasized that by “…building on student strengths and interests, dropout rates from music classes may be reduced, and interests in other forms of musical expression may be increased” (p. 137). Administration Factors.

(22) 16 Issues concerning funding, administrative support, academic scheduling, facilities, and logistics affect student choices to continue studies in instrumental music programs to some degree. In a study that surveyed music programs, Gau (2004) found that the “ …average retention rate for band and choir [was] 70% …” (p. ii). Having visited music rehearsal rooms across Canada and the world, Lautzenheiser (2005) discovered that less than 17% of students choose to join band, choir, or orchestra because of various issues mentioned above. Hartley (1996) and Delzell and Derksen (1998) made interesting submissions in terms of facilities and grade arrangement. Hartley (1996) explored possible differences of retention of band students between schools that started band in the fifth and sixth grade. Hartley (1996) also related grade level organization to its effect on student attitude toward band participation. In another study, Hartley (1997) found no correlation between the beginning grade level and success or failure in retaining students, but she discovered that the grade six students were more positive toward band participation. Concurring with Hartley (1996) and Delzell and Derksen (1998), Hartley (1997) asserted that there “ …appears to be increased attrition in instrumental music when students change buildings and, probably even more so, when students change teachers…” (p. 18). Jorgensen (2001) advocated that schools offer instrumental music to students earlier than Grade 6 or 7. Conversely, Hartley (1996) suggested that enrolment in grade seven was not significantly affected by the starting grade level or grade-level organization. Harltey’s results (1996) implied that “ …the decision of when to begin.

(23) 17 instrumental music instruction cannot be based on a difference in enrolment between the two grade levels” (p. 314). Boyle, DeCarbo, and Jordan (1995) studied the problem of band student dropout rates from the viewpoint of the teacher. Trying to discover why students quit band, they found that “…student ‘lack of commitment to work’ was the most highly rated reason for student dropout” (p. 1). Boyle et al. posited that many dropouts were those who started band in the early grades. They claimed that if band was offered initially in grade seven, they would likely continue in the program until high school graduation. They, however, also presented an argument against that claim. They referred to Hartley’s (1991) study that indicated, “…the grade at which an individual begins instrumental instruction has no effect on enrollment and retention in seventhgrade students” (p. 23). A further school-related issue is scheduling. Students are often unable to participate in instrumental music because of conflicts with academic scheduling. Delzell and Derksen (1998) noted: “It is quite possible that delaying the opportunity to seventh grade could, in some settings, adversely affect the number of students participating due to conflicting schedules” (p. 19). The authors (1998) supplemented findings by suggesting “ …the academic schedule of a school building can have a significant influence on the quality of the instrumental music program, with one of the key issues being whether the schedule permits reasonable access to enrolment in instrumental music” (p. 19). Gouzouasis cited Boyle, DeCarbo, and Jordan,.

(24) 18 (1995) and Sandene (1994) who each indicated in their articles that “…scheduling emerges as a recurring theme in North American research since grade eight students are limited in their choices of electives and must choose between survey courses of technologies, drama, visual art or band” (p. 77). Administrative support also plays a large part in strengthening enrolment in school band programs. Sandene (1994) advised “…classroom teachers can help students to remember to bring their instruments to school…” and that administrators need to ensure that “…withdrawal from choral, instrumental, and general music programs is to be treated as a schedule change [and] is only appropriate within given times of the year” (p. 33). While operations within the school improve retention, certain scholars have shown social factors to be more influential. Social Factors Social issues carry the greatest influence on retention according to Kinney (2010). She proposed to predict students’ decisions to continue participating in urban middle school band programs using independent variables such as family structure and socioeconomic status. Family structure was defined as “…the number of parents or guardians living at home” (p. 345). According to Kinney (2010) “…[Family structure] is second only to academic achievement in terms of robustness” and “…this factor is ubiquitous in predicting band enrolment across all levels of all other independent variables” (p. 345). Her results, however, were in opposition to the findings of.

(25) 19 Klinedinst (1991) and McCarthy (1980) who determined that socioeconomic status was the strongest predictor of retention. Albert (2006) quotes Bornstein and Bradley’s (2003) definition of socioeconomic status (SES) as “ …the relative position of individuals, families, or groups in stratified social systems where societal values (e.g., occupational prestige, education) are not uniformly distributed” (p. 39). Research such as Corenblum and Marshall’s study (1998) predicted “…students’ intentions to participate in instrumental music based on their family’s SES” and “…perceived parental support” (p. 40). One of their hypotheses was that, “Socioeconomic level should predict students’ outside musical interests and their perceptions of their parents’ attitudes; both of the variables should, in turn, predict intentions” (p. 131). Through a questionnaire and analysis, Corenblum and Marshall (1998) found that: Students perceived this support as being either positive (and they participated in band the following year) or negative (and did not participate in band the following year). Thus, perceived parental support predicted by SES also predicted student participation in the instrumental music program. (p. 41) In terms of family influence, Kinney (2010) interpreted her data by reporting that “students who lived with both parents were more likely initially to enroll and remain enrolled in band programs” and that “…single parents may find it untenable to accommodate [associated after school activities] …” (p. 345). Although family has some influence on students continuing instrumental.

(26) 20 music studies, there is another strong reason on which students base this decision. Further to decision-making, Gouzouasis found that “…musical experience of the family of the students and the environment in which they lived played a huge part” (p. 86). Disappointingly, Lovell, Birch, and Fries (1996) suggested that “an extremely small percentage of students join band for musical reasons; they contend that students predominantly join band to be with friends and to be in a particular organization” (p. 47). Warnock (2005) concurred: “Home influences, based on direct parental attitudes toward music education and SES, should predict attraction to participation” (p. 48). He concluded that, “friends and interpersonal social situations provide stronger influences than direct influences of parents” (p. 53). Summary Understanding the issues that are related to retention can assist instrumental music teachers in strengthening their music programs. Student achievement is strongly influenced by academic success as indicated by Schivista (2007). Jorgensen (2001) emphasized that to improve retention, musical success is critical. Albert (2006) agrees, stating “…further research in this area could be helpful in designing recruitment strategies for instrumental music educators” (p. 42). Finally, Klinedinst (1991) found that predicting student retention could be predicted with more accuracy than predicting student dropout (p. 236)..

(27) 21 In addition, teachers must build strong relationships within their school (Mixon, 2005). Hartley (1997) suggests that teachers make adjustments to how they plan and deliver the curriculum, creating a positive classroom environment, and continue to find ways of attracting students to the instrumental music program. School was another main factor addressed by the literature. When possible, it was found that schools should offer beginner band earlier than is typical. Delzell and Derksen (1998) indicated that administrators should avoid scheduling conflicts so that at least students will have the option to take band. Social factors also heavily predict retention in band. Albert (2006) reminds us that although teachers cannot control social factors like SES, “…we have a role in helping students realize their potential …” and that “…improving our awareness of these influences on instrumental music, music educators have a better chance to make an instrumental music program possible for all children” (p. 44). Finally, Gouzouasis (2010) suggests that future research would be of interest to evaluate factors that influence popular courses (hockey and other physical activity related course) by “…discover[ing] the elements that make those programs popular” (p. 76). The following chapter will describe and discuss the methodology used in this study..

(28) 22. CHAPTER THREE: Methodology Introduction The purpose of this study was to determine factors that improve student retention in school band programs by means of a survey. The literature reviewed indicates a need for research in this area. This chapter provides a rationale for choosing a survey, an explanation of the specific type of method used, and the procedures that were used to compile, record, and interpret the data. A brief description of the sample is also given. Rationale for Conducting Quantitative Research Since my research question asked what factors improve student retention in school band programs, a quantitative design was appropriate as I was looking to investigate specific characteristics of a group of people that have at least one characteristic in common – student retention in their band programs. I chose to gather data by means of a survey because discovering the opinions, values, and beliefs of band teachers could be accomplished most efficiently in this manner. It is the most basic of descriptive research (Phillips, 2008). According to Fowler (1993), a survey is a set of questions that are designed to “…provide statistical descriptions of people by asking questions, usually of a sample (p. 1). Fink (2009) states that, “…surveys are best when you need information directly from people about what they believe, know, and think (p. 11). Furthermore, surveys require a defined population of interest, known as a sample (Jaeger, 1997). Dillman (2007) maintains that, “…the.

(29) 23 sample consists of all units of the population that are drawn for inclusion in the survey…” (p. 196). Quantitative research tries to “describe, explain, or predict human actions” (Hanley, McIntosh, Goolsby, and King, 2002, p. 11). The purpose of quantitative research (especially non-experimental) is to describe specific characteristics of a group of people that have at least one characteristic in common. In contrast, Jaeger (1997) emphasizes that the focus of qualitative research is on specific facts that describe a group and its present conditions. The qualitative researcher “observes people in natural settings in hopes of “…understanding the learner and their environment” (Phillips, 2008, p. 12). Even though qualitative studies deal with words and quantitative studies focus on numbers (Phillips, 2008), Bresler and Stake (2006) state that, “…no research study is purely qualitative or quantitative…” (p. 277). In a qualitative study where themes have emerged prior to, during, and post data-gathering, measurements of opinions and values must be made, quantitatively defining the study. Likewise, in a quantitative study where statistics are analyzed, researchers need to interpret the gathered data, which adds a qualitative character (Phillips, 2008). In the present study, the use of text responses, (i.e., short-answer questions on a survey), provided a qualitative component to the study..

(30) 24 Research Design There are three basic methods of collecting survey data: mail surveys, telephone surveys, and face-to-face interviews (Jaeger, 1997, p. 1). Fink (2009) provides a straightforward chart that lists and compares the various types. One is the interview, which may be conducted over the telephone or in person. The other type is self-administered done by mail or online. Comparing each of the survey types, Fink (2009) supplied five advantages to using a web-based survey that are listed as follows: 1. Communication with the participants is direct via email. 2. Links to unfamiliar words can be given to respondents. 3. Sending email reminders to participants is easier than making telephone calls. 4. Processing data is easy because spreadsheets can be downloaded for presentations. 5. Researchers can now use skip logic instead of the participant answering non-applicable questions. (p. 9) Reynolds, et al. (2007) explain that web-based surveys are “…instruments that physically reside on a network server (connected to either an organization’s intranet or the Internet) and that can be accessed only through a web browser…” (p. 8). Recently, some scholars (Van Selm & Janowski, 2006) have objected to the use of web-based surveys. Cobanoglu (2003) and Couper (2008) refer to the issue of lower response rates on Internet surveys while others such as Kay & Johnson (1999) and Crawford, Couper & Lamias (2001) note concerns.

(31) 25 of occasional coverage bias (by choice or by circumstance). Solomon (2001) claims that even with expansion of Internet availability, there is a lack of connectivity in some samples. To explain, the coverage bias concern might have been valid in this study since some teachers in the sample might still be trying to avoid the Internet and a computer. The web-based survey, however, is also an attractive option for collecting, analyzing, and presenting data. Fink (2009) notes that in a web-based survey, the burden of mailing or making telephone calls has been simplified through the use of technology. In summary, although the web is growing larger and larger every year (Van Selm and Janowski, 2006), low and non-response error still exists (Fowler, 1993). Some authors (Cobanoglu et al., 2001; Couper, 2008) suggest that the use of mixed methods (telephone, fax, and the WWW) can perhaps counter low response rates. Having considered other types of surveys, and knowing that there is still no single perfect design, web-based is still the most preferred. For these reasons, it was logical to choose an online survey for this study. Survey Instrument The questions on the survey address issues that have emerged from my own experience, as well as those that have arisen from the review of literature. I followed Fink’s (1995) suggestion composing questions that were “…pertinent to the survey’s objectives…” (p. 5) which were to confirm factors that improve student retention in school band programs. Then I re-categorized the questions into five logical sections. In the first section entitled Training,.

(32) 26 Qualifications, and Experience, I identified qualified participants and, in the process disqualified those who were not. In the second section, I asked participants about population sizes (geographical and students) and statistics regarding their program situations. In the third section, I questioned participants about their philosophical approaches to the planning of curriculum delivery, expectations of switches between instrument families, and practicing, performances and listening activities both in and outside of class. In the fourth section, I asked participants to respond to statements concerning student support using various rating scales. This section also touched on student interest and success, and included a short answer (qualitative element) in which participants were asked to describe their top motivation strategies for improving student retention. In the last section of the survey, I asked participants to share their opinions by ranking statements related to decisions made by administrators concerning scheduling, attendance at meetings and performances, and financial support for band programs. There were forty questions on the survey: twenty-two check boxes, seven numerical, nine Likert, and two open-ended. Choosing Survey Gizmo Choosing the best online survey was another important decision in this project. I wanted to select and use a survey that would produce a professional look, that was inexpensive, and that was both productive and efficient. Considering my criteria, Survey Gizmo met and exceeded my expectations. I also considered using Survey Monkey, except it was not offering a free.

(33) 27 package to students. Despite the advantages, Survey Gizmo was not a perfect survey instrument, due to the risks associated with participant data. Recruiting participants was a small challenge, as they needed to accept that the United States government could access their responses at any time in the future. Additionally, identifying participants by email also created more risk for those involved taking the survey. There were a few disadvantages, however, these were outweighed by the numerous advantages and therefore I chose to use Survey Gizmo. Ethical Considerations In preparing my application for the University of Victoria’s Human Research and Ethics Board, I made some major decisions concerning ethical issues. These included the process of recruiting participants, the collection of data, and the dissemination of data. Each of these was detailed in the application. An explanation of these considerations follows. The recruiting process was considered to be ethical because participants were adults who were asked to provide informed consent prior to taking the survey. Initially, I wanted to avoid using participant email addresses in the recruiting process. It was, however, deemed necessary to ask for participant emails on the consent form. The collection of data was also a consideration, especially because Survey Gizmo is an American based company. This meant that I had to include the following disclaimer on the consent form:.

(34) 28 Please be advised that information about you that is gathered for this research study uses a web program located in the United States. (Survey Gizmo). As such, there is a possibility that information about you may be accessed without your knowledge or consent by the United States government in compliance with the United States Patriot Act. Therefore, participants were well advised as to how their data might be used in the future. Other than the possibility just mentioned, participants were also informed that I would send the results to those who wished to receive them via email. Finally, reporting and disseminating the data were administered ethically as well. I did not name anyone in the report or when sending out the results (tables and charts including various numbers and percentages, etcetera). Furthermore, email messages were sent using the blind carbon copy (BCC) function so that other participants did not see any email addresses, except theirs and mine. Sample Surveys require a defined population of interest, known as a sample (Jaeger, 1997). Dillman (2007) maintains that “…[t]he sample consists of all units of the population that are drawn for inclusion in the survey…” (p. 196). Since the purpose of the study was to determine the factors that improve student retention in school band programs by means of a survey, I chose as my sample band teachers in the province of Alberta, who were qualified music specialists (see delimitations in Chapter I). To limit the study to a manageable.

(35) 29 size but still gather a sufficiently large sample, I restricted participants to school band teachers in two school divisions in Alberta: Edmonton Public and Elk Island Public. This type of sample is known as convenience sampling (Borg & Gall, 1996). These two divisions total 50 schools and I hoped to gather a minimum of 30 completed surveys. Procedures After receiving approval from the Human Research Ethics Board of the University of Victoria on August 2011 and prior to launching the survey online, my colleagues took the survey, offering suggestions for improvement. This process is also known as a ‘pilot.’ Jaeger (1997) declares that a pilot is “…an essential part of any survey research study…” (p. 469). Following that, he explains that pilot respondents check for clarity, suitability, and length of time taken to complete the survey. The researcher will then analyze the pilot data and make necessary corrections to the survey (Fink, 2009; Jaeger, 1997). Upon receiving responses, I revised the survey in accordance with comments and suggestions. Subsequently, I emailed out a scanned copy of the approval certificate along with a letter and consent form to the Superintendents of Edmonton Public and Elk Island Public Schools. Upon receiving approval from the Superintendents, I sought permission from each of the Principals via email or fax in a similar fashion to the Superintendents and asked them to forward the invitations to their band teachers once they had given their approval. The email message and invitation letter included a link to the Survey Gizmo website, and the password to access the survey. Once band.

(36) 30 teachers received the letter and consent form, they chose to participate by signing the consent form, providing their email address, and finally returning the signed form to me by fax. Once I received a faxed consent form, I emailed the participant in order to provide him/her with the web link and password to the online survey on the Survey Gizmo website. Independently, each participant navigated to the website and completed the online survey. Generally the procedure transpired according to plan. There was however, one change. I was able to retrieve a signature from the Superintendent of Elk Island Public Schools, but Edmonton Public Schools treated my research request differently. In Edmonton Public Schools there is a department of research. While approving my research proposal, the head of the research department claimed that they do not require signatures from principals but only from the participants. Data Analysis Having chosen and described the web-based survey design and the factors associated with it, it was important to consider how to examine the data. Phillips (2008) stressed that the researcher should cautiously collect and present accurate and representative data of the sample. He continued to describe analysis as the “…most difficult section…” and explain that data may be analyzed by some kind of computer program (p. 24). Fink (2009) wrote that, “…analyzing data means using statistical and qualitative methods to describe and interpret participants’ answers to the survey’s questions…” (p. 77). This study used descriptive statistical methods.

(37) 31 and, as per Fink (2009), this type of analysis should “…allow surveyors to make statements about statistical significance, one measure of the meaningfulness of survey results…” (p. 77), i.e., to the band teachers or other researchers reading the results. The analysis used was also partly ‘content analysis.’ Fink (2009) called it “…a method of analyzing qualitative data for the purpose of drawing inferences about the meaning of recorded information such as the open-ended responses and comments made by survey respondents…” (p. 89). Once online responses were received, Survey Gizmo quantified the responses. The program counted and compared the data. I then tasked the Gizmo program to create lists, graphs, charts, and tables, making the results suitable for presentation. I, however, needed to analyze the qualitative responses myself, which were in the form of short-answers. I compiled and analyzed the data through the methods described by Fink (1995), which consisted of information that participants typed into text boxes. This helped me discover themes, which will be addressed along with the quantitative results in Chapter IV..

(38) 32 Summary The chosen method for this study was survey research with both quantitative and qualitative elements. As well, the instrument used was an online survey, called Survey Gizmo, which was free to students, and after consideration, was the best method when compared with in person interviews, mail-out surveys, and telephone interviews. Chapter IV presents the results of the survey, both quantitative and qualitative, and includes a discussion section that offers an interpretation of the results and their connection to my own teaching practice..

(39) 33. CHAPTER FOUR: Results Introduction In this chapter the quantitative and qualitative results of the data analysis are presented and discussed. The data were collected and processed out of a motivation to answer questions posed in chapter I of this study. These questions were: • How do teachers maintain student interest to continue in band programs through to the second year of study? • What strategies would help teachers increase initial student enrolment in band programs? The goal of this research project was to investigate the factors that improve student retention in school band programs by means of a survey. The findings in this chapter confirm that the band teachers in this sample utilize common strategies to improve student retention. Recruiting Band Teachers I initially sent email invitations to 10 Elk Island Public Schools that currently have a band program. Of the 10, seven met the criteria and completed the survey. One teacher was not interested in participating, one teacher was disqualified for failing to meet the participant criteria (see Delimitations, p. 7), and one teacher was considered a non-response as his/her principal did not reply to my email invitation. I also sent email invitations to 40 Edmonton Public Schools. However, five principals replied saying that their schools did not currently offer band,.

(40) 34 two principals reported that their band teachers were not interested or did not have the time to take the survey, and 16 principals failed to respond. Thus, Survey Gizmo collected 17 responses from Edmonton Public School band teachers and 7 Elk Island Public School band teachers. For part of the survey, one band teacher was disqualified for lack of experience. With 24 responses out of 50 potential replies, the response rate was 48%. Below is a summary and discussion of the band teacher responses to each of my closed and open-ended questions.2 The figures in the next section graphically represent the results in the forms of tables, pie charts, and bar charts. The titles of each figure are the questions that were posed to the band teachers on the survey. A short summary of the results follows. The results are presented in three phases: quantitative, part I; qualitative; quantitative, part II. Interpretation of the results follows in the discussion section. Quantitative Results Part I The first two questions asked band teachers about their training and experience. Figure 1 shows that 54.2% had a BMus, 83.3% had a BEd, 4.2% had an ARCT, 8.3% had a Specialized Diploma, and 12.5% had an MEd. Figure 2 shows that 23 teachers taught in schools for at least three years and one had not.. 2. Occasionally, a respondent did not answer a question. However, these instances were few and far between and so their surveys were not discarded..

(41) 1$- . 35.

(42)

(43) . 2+ !1! !

(44)

(45)

(46)

(47) !!!!34+ What post-secondary training do you possess? . %5 . '"%" 6#. #$% #. #. . !. &. %#. '%". $% ()(!!

(48) *+ ! *) .

(49)

(50) ,-

(51) . &.

(52)

(53) .

(54).

(55)

(56) Figure 1 . !. . .

(57) !

(58) !. ) ! !! Have you taught in schools for at least three years? ". &. Value Yes

(59)

(60) ,-

(61) No. . Count 23 1. ()(!!

(62) *+ ! *) &. ". #$%. $. '"%". Percent % $% 95.8% '%" 4.2% %#. Figure 2 Questions three through five asked about teaching assignments. Three 7 +

(63) !+ !* !+!3 0 $%. teachers had 40% band in their assignments, five teachers had 50%, four had 60%, and three had 100%, with the remaining teacher assignments being spread as displayed in Figure 3 below. Figure 4 shows that 23 teachers had at least two levels of band and one did not. Figure 5 presents the number of respondents from each district. .!/#%'. What percentage of band are you teaching this year?

(64)

(65)

(66)

(67)

(68) . .! 0. Figure 3. Count 1 1 1 1 1 3 1 5 1 4 1 3. Percent % 0% " 15% /#%' 25%$% 30% 38% 40% 48% 50% 55% 60% 70% 100%. .

(69) !

(70) !. ) ! !!. $.

(71) 36 Do you teach at least two grades or levels of band? Value Yes No. Count 23 1. Percent % 95.8% 4.2%. Figure 4 In which school division do you teach? Value Edmonton Public Schools Elk Island Public Schools. Count 17 7. Percent % 70.8% 29.2%. Figure 5 Questions six through eight inquired about grade configurations, student populations, and feeder schools. The majority of the respondents had junior high in their school: 21 teachers had Grade 7, 21 had Grade 8, and 19 had Grade 9. The complete grade configurations results are presented in Figure 6. In terms of school populations, 45.8% of schools enrolled between 101-499 students, 41.7% enrolled between 500-999, and 12.5% of schools enrolled more than 1000 students (see Figure 7). The number of feeder schools for each teacher is displayed in Figure 8. Twenty-five percent of teachers had four, 20.8% had three, 20.8% had zero, and 16.7% had two. What grades are in your school? Value K 1 2 3 4 5 6 7 8 9 10 11 12. Figure 6. Count 5 5 5 5 7 7 8 21 21 19 9 9 9. Percent % 20.8% 20.8% 20.8% 20.8% 29.2% 29.2% 33.3% 87.5% 87.5% 79.2% 37.5% 37.5% 37.5%.

(72) . /. "6%#. . /. "6%#. . /. "6%#. 37 7 9 ! !

(73) !+ 3. How many students are in your school? =%#. $//$#%'. #///. . $%6. ! !!. ". %#. . #///$%6. =. $ ;>%. (. /%. -. //%6. <. #%. Figure 7. & !

(74)

(75) . 7 9 *!+ ! ! !+ +34 %5 How many feeder schools does your school

(76) !

(77) ! have?. $//. . $#%'. ;$% #'%". ) ! !!. $. %'. $% $#%. ;%6. "%'. Figure 8. ' !

(78)

(79)

(80)

(81)

(82)

(83)

(84)

(85) "

(86) . . grades of band

(87) !

(88) ! The following two questions concerned classes. #. %'. . . $%. ". #. %'. #. . '%". ) ! !!. $. . ;$%. -. %6#. assigned to respondents and band student requirements to( take non-core $ ;%6 %6 classes outside of band. Eighty-seven and one; half # percent<of respondents $ ;% ; $% taught Grade 7, 87.5% taught Grade 8, and 79.2% taught Grade 9 (see. Figure 9 for complete results). Seventy-five percent of respondents’ students 2+

(89) +! * +34+. %5. were obligated to take non-core classes and 25% were not (see Figure 10). . '6%#. '6%# 6/%. 6#. #. #. #. /%. /%. /%. . . . '%" . #. ;. 6. '. /. (

(90) "

(91)

(92)

(93) .

(94) ". #. %'. $. ;. #. #. . '%". ;. . $%. -. %6#. <. ;%. 38. Which grades of band do you teach?. 2+

(95) +! * +34+. %5 . '6%#. '6%# 6/%. 6#. #. ;. /%. ;. #. #. 6. #. /%. . /%. 6;"%. . '6%#. (. '%#. . '6%#. -. %6. /. 6/%. <. 6. /%. . 6 Figure 9 (

(96) "

(97)

(98)

(99) . /%. '. '%". / . . #. ;. 6. . '. /. 6. . #. % . /%. . . . '%".

(100) !

(101) !. ) ! !!. $. (

(102)

(103) ! !@

(104) 8 1 !!!34+ > 7 1&>%5. Are your continuing band students required to take any non-core classes? 0 #%. .!6#%. Figure 10. )*

(105) "

(106)

(107) +

(108) ,

(109)

(110) .

(111) -

(112) .

(113) . The next three questions were related to program populations,

(114) !

(115) !. . .!. '. 6#. ) ! !!. 0 # specifically the number of first-year students, the; number of students in. $. second year and up, and the total students in each band teacher’s program. 2+

(116) + * !4!5 ! !+ **3. Six band teachers had 30 first year students and two had 120 (see Figure 11 for other remaining program populations). Of students in their second year

(117)

(118) *+ $%6. and up, one band teacher had 250, one had 120, and one had 109. The )

(119) 1 #$%. remaining populations were less than 100 (see Figure 12). Reported total program populations were as follows: one band teacher had 370, one had ?!

(120) *+

(121) 1 $%. 265, one had 202, and one had 190. The remaining band teachers had less

(122) "

(123)

(124)

(125) . . . than)

(126) 1 150 students in their program (see Figure " 13). ?!

(127) *+

(128) 1 . . #$% $%.

(129) !

(130) !. ) ! !!. $.

(131) 39 How many of your students are in their first year? Count 2 1 1 1 1 6 1 1 1 1 1 1 1 1 1 1 1 1. Response 120 13 24 25 28 30 32 33 37 38 40 41 42 47 48 50 60 93. Figure 11 How many of your students are continuing past their first year? Count 1 1 1 1 2 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1. Figure 12. Response 109 12 120 13 15 250 26 27 28 30 33 35 36 40 45 50 52 65 80 85.

(132) 40 How many total students are in your band program? Count 1 1 2 1 1 1 1 1 1 1 1 1 1 1 4 1 1 1 1 1. Response 105 110 115 125 13 135 136 19 190 202 25 265 370 64 65 70 76 80 90 94. Figure 13 The following two questions inquired about the type of band program and structure that respondents offered. Timetabled and year-long courses had the highest number of responses with 54.2%, followed by 41.7% responding with a combination of the two, and 4.2% of courses were outside of the timetable and yearlong (see Figure 14). There were 18 performancebased programs and 6 comprehensive-based programs represented (see Figure 15)..

(133) )*

(134) "

(135)

(136) +

(137) ,

(138)

(139) .

(140) -

(141) .

(142) . . . .!. '. 6#. 0. ;. #.

(143) !

(144) !. ) ! !!. 41. $. What type of band course(s) does your school offer? 2+

(145) + * !4!5 ! !+ **3.

(146)

(147) *+ $%6. )

(148) 1 #$%. ?!

(149) *+

(150) 1 $%. Figure

(151) "

(152)

(153)

(154) 14 . . . . $%.

(155) !

(156) !. Which method do you use to structure program? $ " your #$%band ) ! !!. )

(157) 1 . ?!

(158) *+

(159) 1 . Value Performance-Based (technique and performance-driven) Comprehensive (holistic approach that combines performance, history, theory, musicianship, composition, and improvisation) Non-Performance (no concerts, playing in-class only). Count 18. Percent 75.0%. 6. 25.0%. 0. 0%. Figure 15. The next two questions concerned teacher beliefs regarding the changing of instruments throughout the year. There were 13 band teachers who allowed students to change instrument families and 11 who did not (see Figure 16). Half of the teachers allowed students to change at any time, 35.7% at the end of the year, and 14.3% at the end of the trimester (see Figure 17).. Do you allow your students to change instrument families? Value Yes No. Figure 16. Count 13 11. Percent % 54.2% 45.8%.

(160) 42 (9+

(161)

(162) +!+ 9

(163) ! + ! +. . At what point in the school year4! +

(164)

(165)

(166) 53 do you allow instrument changes to happen? & * A

(167) !$%". (

(168) #%. & *"#%6. Figure 17. &*!

(169)

(170) !

(171)

(172)

(173)

(174)

(175)

(176)

(177) . . . & *. #. "#%6. & * A

(178) !. .

(179) !

(180) !. The following questions concerned band 6teacher of$ (

(181) # expectations ) ! !! $%" outside-of-class practice and whether or not they evaluated such a practice.. Figure 18 shows that 21 teachers expected home or outside of class practice - < ! !

(182) + !

No results found