1:1 Learning Program Evaluation: Data Collection Process
As you may already know, MURSD has invested in 1:1 student access to iPads across grades 5-12. This spring, Mendon-Upton Regional School District is working with Boston College’s Lynch School of Education on a collaborative research and evaluation project to help our community learn more about how students are using iPads for school. This six-month study will employ surveys, focus groups, and classroom observations to document the current range of teaching and learning practices and seeks to provide the community with a richer understanding of the successes, challenges, and opportunities for leveraging educational technology today and into the future at MURSD. The MURSD 1:1 Learning Evaluation team will be working with students, teachers, school leadership, and parents between between February and June 2017. The sections below provide an overview of what types of data will be collected and the process for data collection. Parental consent forms for student survey data can be found here in March of 2017. If you have any questions, please contact David Quinn, Director of Technology Integration.
Classroom observations will be carried out at Miscoe Hill Middle School and Nipmuc High School throughout Spring 2017 to provide a ground-level context and a deeper perspective on teacher and student experiences. Randomly selected classroom observations will occur in all major subject areas across grades 5-12 to document the spectrum of teacher and student practices across a variety of learning settings. Classroom observations will be carried out unobtrusively and passively by a trained team of Boston College and Wesleyan University researchers, all of whom are National Institute of Health certified for human subject research.
A student survey will provide the district with a broad summary of how students engage and use a variety of digital resources to further their education. Every Mendon-Upton student in grades 5-12 will complete a computer adaptive survey in March 2017. This relatively short (under 15 min.) online survey will provide measures of students’ access to technology in school, use of technologies in school across subject areas, personal comfort level with technology, access to technology at home, and various uses of technology at home. Given the district’s focus, the student survey will also specifically measure the degree and variety that technology is impacting student collaboration and communication. The Boston College evaluation team will work with Mendon-Upton school administration to ensure a high response rate through the real-time monitoring of survey data via daily email updates. All individual responses and collected data will be kept strictly anonymous and confidential.
Like the student survey, an online teacher survey will be completed by all Miscoe Hill Middle School and Nipmuc High School teaching staff in March 2017. The teacher surveys will include item sets dedicated to capturing the variety and extent of teachers’ technology use, teachers’ attitude toward technology, teaching, and learning, as well as teachers’ beliefs on student motivation, engagement, communication and collaboration. Teachers will also complete brief item sets that measure more general pedagogical practices and classroom practices. Collectively, these items will help demonstrate how digital technology resources are utilized in the approach and delivery of the curriculum (as well as various aspects of teacher/student interactions). Additionally, a section of the survey may focus on technical aspects of the deployment (e.g., slow network, difficulty accessing on-line materials, timeliness of technical support, etc.) that historically have served as obstacles impeding the full effects of the initiative. Boston College will provide the support for the participating schools to ensure the survey collection is properly scheduled and easy for teachers and students.
Student Focus Groups
In addition to the use of surveys, the evaluation team and school and district administration will schedule a limited number of student focus groups over Spring 2017 school to ensure that student perspective and voice is articulated fully and well-represented. Typically, a voluntarily sample of 5-8 students will be the chosen across grades 5-12 at Miscoe Hill Middle School and Nipmuc High School and asked to participate during the school day in a 40 min. focus group. All focus groups will be led by a senior member of the Boston College research team. Given the targeted outcomes of the MURSD initiative, there will be a emphasis focusing on the impact of digital age resources on teaching and learning, collaboration, and communication, however students more general attitudes towards technology and schooling will also be addressed.
Teacher Focus Groups
Although teachers will be surveyed over the course of the evaluation, a small number of teacher focus groups will be conducted to gain a richer and more in-depth understanding of how digital learning resources have impacted students’ communication, collaboration, as well as general teaching and learning practices. Throughout Spring 2017, teachers serving both Miscoe Hill Middle School and Nipmuc High School will be given an opportunity to reflect on 21st century teaching and learning via teacher focus groups. All focus groups will share a basic core of questions, however, the format will remain open-ended enough to allow respondents to discuss a wide variety of issues. In every case, the evaluation team will schedule the focus groups with the advance approval of school administrators. Each voluntary focus group will be approximately 45 minutes in length and conducted by a trained Boston College researcher.
School Leadership Interviews
Given the importance of school level leadership in the implementation and dissemination of successful educational technology applications within schools, it is proposed that an ongoing series of discussion and interviews be conducted with Mendon-Upton school principals, district and school curriculum leaders, as well as educational technology staff. Interviews with school leaders over the duration of the six-month evaluation period will help illuminate both practices and outcomes across different settings as well as their own practices and attitudes towards pedagogy, educational technology, collaboration and communication.
Parent Focus Groups
To provide the Mendon-Upton community some perspective from the parent community, two parent focus groups will be conducted in Spring 2017. In these sessions, parents of Miscoe Hill Middle School and Nipmuc High School students will be given an opportunity to reflect on their observations of student participation in the 1:1 Learning program. The the aim of the parent focus group sessions will be to provide more vivid and anecdotal evidence to complement the quantitative evaluation data. Researchers and parents will explore the multitude of ways parents have observed their child using the iPads for school-related projects and gather some general impressions of the program. Each focus group will be approximately 45 minutes in length, scheduled through the school and district leadership and conducted by the Boston College research team.