USING ROBOTICS AND ENGINEERING DESIGN INQUIRIES TO OPTIMIZE MATHEMATICS LEARNING FOR MIDDLE LEVEL TEACHERS: A CASE STUDY

This exploratory case study reports findings on 20 middle-level science and mathematics teachers’ perceptions of the effectiveness of a one-year project in which teachers engaged in using robotics and engineering design inquiries in their classrooms. Principled by Bandura’s Social Learning Theory (SLT) and using mixed methods approaches, the study measured teachers' efficacy through the Mathematics Teaching Efficacy Belief Instrument (MTEBI) and observation logs before and after the program. The results of this study showed statistically significant differences between PRE MTEBI and POST MTEBI scores. Furthermore, five themes emerged that illuminated potential affordances and constraints that teachers perceive as opportunities and barriers to employing robotics and design thinking in the mathematics/science classrooms. The reported themes are creating collaborative spaces underpinned by design thinking affords transformative learning; problem-solving through shared inquiry elevates confidence; building connections between mathematical concepts and real-life phenomenon supports a willingness to learn new ideas; system support, resources, and funding are prerequisites to engage in modeling design; and designated curriculum restrains teachers from engaging in extra activities that focus on design thinking.

problem solving, particularly in middle grades. In the US and mobilized by the Race to the Top national funding grants, states and districts are advancing initiatives to enact cutting edge educational programs to accelerate work in improving student achievement. One of the hallmarks of CCMS is the specification of content that all students must study to be ready for high school and college. In this respect, there is an imperative need to equip teachers with the necessary skills and dispositions to transform the middle school mathematics and science curriculum landscape in ways to facilitate students' conceptual understanding and scientific thinking in preparation for high school. Additionally, NGSS call for integrating engineering design into the structure of science education by raising engineering design to the same level as scientific inquiry when teaching science disciplines particularly at the middle level. Engineering standards have been integrated throughout the science and mathematics domains of physical science, life science, and earth and space science. Furthermore, research on facilitating modeling competencies particularly at the middle grade level has undergone a deep transformation due to opportunities offered by the extraordinary rapid evolution of technology and by the development of new pedagogical approaches.
Much has been written about the decline in achievement in middle schools, particularly in mathematics (Wijsman, Warrens, & Saab, 2016;Borman et al., 2019;Malone, Cornell, & Shukla, 2019;Hughes, Im, & Allee, 2015;Evans, Borriello, & Field, 2018). Cited most often are results of the 1999 Trends in International Mathematics and Science Study (TIMSS) showing a decline in mathematics and science achievement from grade 4 to grade 8. And while scores have improved in the US for some groups of students --the TIMSS 2019 test results showed that only U.S. white and Asian fourth-graders scored above the international averageothers are still left behind. Results from the same study show that Hispanics' scores were average, and Black students' scores were below the international average.
In a 2008 press release from the National Council of Teachers of Mathematics, noted that "students' performance continues to be related to their socioeconomic background, with more affluent students scoring higher." These results call for focused attention on mathematics education, middle grades teacher development particularly in urban settings. Similarly, Robinson (2016) addressed the need for research into the implementation of 3-D design and manufacturing particularly especially in the middle school classroom citing several situational factors that contribute to student success in mathematics. He further asserts that "Learning experiences underpinned by design thinking lead to positive student outcomes" (p.88). Furthermore, Bonghanoy et al. (2019) capitalized on the role of transformative professional development to support teachers in engaging in novel teaching experiences and acquiring necessary skills to reflect on their instruction.
The purpose of this paper is to report findings of a case study that focused on examining teachers' perceptions of the effectiveness of engaging in an improving teacher quality project that trained middle level teachers to use robotics and engineering design inquiries in their mathematics and science classrooms. The goals of the program were to strengthen students' problem-solving skills using handson activities in robotics design, provide students with opportunities to simulate real life phenomena using different robotics models, and increase the scope of scientific and mathematical knowledge using hands-on exploration tools for inquiry and modeling. The ultimate goal of the project was to immerse middle level teachers in research experiences that enrich their content experiences first-hand through connections between classroom activities and meaningful applications in robotics and engineering design. Specifically, the study addressed the following overarching research question: What is the effect of engaging in a robotics and engineering design training program on in-service middle-level teachers' mathematics and science teaching efficacy? To what extent do teachers show higher levels of motivation to use robotics modeling in teaching middle-grade mathematics and science?
The study was conducted in the State of Georgia, in the US. According to the Georgia Alliance of Education Agency Heads, Math/Science Task Force (2008), Georgia was one of the states reported to have a critical shortage of qualified mathematics and science teachers, "The most troubling aspect of this situation is that it will worsen dramatically unless aggressive and immediate actions are taken" (p. 3). Moreover, Georgia is identified as the third fastest-growing state in the nation; hence, the student enrolment growth over the next few years requires a significant increase in the number of mathematics teachers at school. Georgia Professional Standards Commission, an agency overseeing teacher licensure, reported a 9.2% in Mathematics teacher attrition per year and considering that attrition rates in high-need urban school districts are generally higher than the state average, the need for high-quality mathematics and science teacher professional development, aimed specifically at teacher training and retention, has never been greater. With more than 17 percent increase in the number of new hires into Georgia public classrooms in 2016, and as the racial and ethnic makeup of enrolment continues to change rapidly, the demand for highly qualified middle school teachers to respond to the wide spectrum of needs is expected to increase even more rapidly than student enrolment growth.
Two high needs systems, Atlanta and DeKalb County School Systems were the collaborating district partners in this project. The project targeted the training of highly qualified mathematics teachers in light of significant percentages of students not meeting expected achievement levels on standardized mathematics tests, including the Georgia High School Graduation Tests (GHSG) (See Table 1). According to a report from the Georgia Professional Standards Commission in 2010, out of 9,773 classes taught in APS, only 8,074 were taught by highly qualified teachers. In other words, 17% of APS classes were taught by teachers not highly qualified in their content area. In light of the APS Middle School Transformation Initiative enacted in 2010, which calls for extended class periods and emphasizes teacher teams specialized in their content areas, the school system is in need to support teachers by providing rigorous professional development in mathematics and science that aims to deepen students' learning of vital mathematical concepts and subsequently prepares them for success in high school (See Table 2). While about 95% of Georgia's middle and high school teachers met the federal requirement of "highly qualified," only 65% of the teachers said in a survey that they had the appropriate certification, according to the study from The Education Trust, a child advocacy group. In DeKalb County Schools, although reports show that only 5% of the schools are not taught by highly qualified teachers, the percentage is much higher considering teachers' growing needs for ongoing support in light of the new reform proposed by Georgia Performance Standards (GPS) and Common Core Standards (CCS) (See Table 3). The study is principled by Bandura's (1997) Social Learning Theory (SLT), which examines the development of self-efficacy as a result of engaging in diverse experiences. Intertwined with selfefficacy is teaching efficacy, or the belief that teachers are capable of impacting student achievement (Granger et al., 2019). Bergman and Morphew (2015), as well as Menon and Sadler (2018), revealed the positive effect that targeted content and pedagogy courses had on pre-service elementary teachers' teaching efficacy before they entered the classroom.
Essential to Bandura's SLT are the following assumptions, (a) the building of self-efficacy and teaching efficacy is personal and develops over time, (b) that experiences affect individuals differently and can positively or negatively affect self-and teaching efficacy, (c) that teaching efficacy is continuously in flux and is dependent upon a myriad of variables (Bandura, 1997(Bandura, , 2001Tschannen-Moran & Hoy, 2001;Voelkel & Chrispeels, 2017). Furthermore, Bandura (1997) identifies teacher efficacy as a type of self-efficacy, which is a cognitive process where a person can construct beliefs about their ability to perform at a provided level of attainment. In this respect, self-efficacy beliefs influence how much effort a person will provide, how long a person will persevere in the face of obstacles, and how they can deal with failures. Bandura (1997) clarifies the difference between selfefficacy and Rotter's (1966) internal-external locus of control by providing data that shows that the two are not essentially the same phenomenon. Tschannen-Moran, Hoy, and Hoy (1998) expand on this idea by stating that: "Beliefs about whether one can produce certain actions (perceived self-efficacy) are not the same as beliefs about whether actions affect outcomes (locus of control)" (p. 211). In this respect, Hendriana, Johanto, and Sumarmo (2018) indicated a strong association between students' problemsolving ability, their self-confidence, and their attitudes toward teaching approaches such as problembased learning. Bandura (1997) provides four sources of efficacy expectations: mastery experiences, physiological and emotional states, vicarious experiences, and social persuasion. The most powerful are mastery experiences because of the perception that performance was satisfactory, then efficacy beliefs are raised, which adds to the expectation that future performances will also be successful (Bandura, 1997). Vicarious experiences are when the skills or tasks are modeled by someone else, and the degree to which the observers identify with the modeler affects the observer's self-efficacy (Bandura, 1997). The more closely that the observer can identify with the modeler, the stronger the impact on efficacy. Social persuasion can consist of a pep talk from a peer or specific performance feedback from a supervisor (Tschannen-Moran et al., 1998). The impact of social persuasion is dependent on the credibility, trustworthiness, and expertise of the persuader (Bandura, 1986). Physiological and emotional states refer to feelings such as anxiety, stress, arousal, fatigue, and mood states (Pajares, 2002). If a person is feeling happy about their performance, this can help with their self-efficacy, and if a person is feeling upset, this can decrease their self-efficacy beliefs (Pajares, 2002). A teacher course or teacher training will display the four sources of efficacy expectation.

METHOD
The study employs a single, embedded, exploratory case study design using quantitative techniques. Yin (2017) argues that a case study allows researchers to focus on a "case" and gathers a holistic and real-world perspective on studying small group behavior or school performance (Yin, 2017). A single case study can provide a unique example of real people in real situations, and this allows the readers to understand the ideas of the research more clearly rather than presenting the reader with abstract theories or principles (Cohen, Manion, & Morrison, 2018). Yin (2017) also stipulates that the use of a case study has the potential to cloud the boundary line between the phenomenon and its context. Furthermore, this case study is exploratory and targets exploring effects that could lead to generating hypotheses that can be further tested in larger scales.
Twenty mathematics and science teachers were purposefully selected by the two high-need partnering schools to participate in a 60-hour professional development modules and in-school follow up support centered around best practices to facilitate modeling using robotics in the mathematics and science classrooms to enhance students' problem-solving skills. This training was part of a year-long project that targets the design and creation of learning environments where middle school mathematics teachers prepare lesson plans and activities to scaffold students' thinking while engaged in project-based inquiries.
Two data collection methods were used: Mathematics Teaching Efficacy Belief Instrument (MTEBI) (Ryang, 2010), and direct observations. MTEBI consists of 21 items on a 5-point Likert scale: The professional development component was aimed at consolidating 20 middle school teachers' modeling skills using robotics-based learning approaches. During hands-on investigations, and using various robotics and design tools, teachers engaged in applying modeling concepts and methods to problem solving in a variety of scientific contexts. The focus is to familiarize teachers with robotics design as a specialized form of mathematical modeling, one of the critical standards for mathematics that is proposed by the Common Core State Standards at the high school level. The product of robotics design in a particular domain is a model of a situation, a structuring and representation of the situation that enables computations to be performed to answer questions, solve problems, predict consequences, or enhance understanding. The trainers, who are researchers in the field of mathematical and scientific modeling, utilized several techniques to motivate critical thinking such as cubing brainstorming, small group inquiries, whole group discussion, cooperative learning, problem-based investigations, and tiered assignments. These routines helped make thinking visible and afforded practical tools for teachers to use in their teaching. During professional development training, teachers: 1. explored the use of online learning tools and installed the Robotic Effector software on their computers to prepare them for designing and programming the LEGO robot; 2. became familiar with LEGO vocabulary and parts of the LEGO kit as they built structural components of a robot and used math, science, and engineering principles to design, build, test, and operate their robots; 3. examined the industrial robots and how they are uniquely designed to accomplish their respective tasks.
At the end of the professional development, teachers were able to: 1. design and program robotic models using LEGO kits that perform simple tasks incorporating modeling skills and using project-based learning approaches; 2. develop lesson and unit plans to integrate robotic design and mathematical modeling using multiple modalities and technological resources that support the infusion of critical algebraic thinking skills into their instructional strategies.
A wiki space was also created and maintained by the researchers during and beyond the implementation of the intervention. The website provided a scaffolding tool to support teachers' implementation of lesson plans and activities created during the professional development and showcased updated teachers' work and designed activities, students' projects, Questions and Answers blog, teachers' conference presentations, etc.

RESULTS AND DISCUSSION
To analyze teachers' MTEBI scores, a paired-sample t-test on a confidence interval of 95% using the statistical software package SPSS was conducted to compare the mean scores on pre-test results of middle-level mathematics teaching efficacy to post-test results. A paired sample t-test is typically used to compare the values of means from two related samples (Gaur & Gaur, 2009). We hypothesized that when teachers are equipped with the necessary modeling skills using robotics-based learning approaches, they will show an enhanced mathematics teaching efficacy to apply these competencies in their classrooms.
The basic unit of analysis to examine observation logs consisted of episodes where mathematics teachers were engaged with modeling activities, either learning or teaching using these skills. Reported texts from observations underwent successive cycles of coding using Saldaña's (2012) step-by-step strategy. Cycle one consisted of magnitude coding, by adding symbolic codes, or abbreviations to indicate the presence, intensity, and frequency, of initial codes. Saldaña (2012) provides that "[m]agnitude codes can be qualitative, quantitative and/or nominal indicators to enhance the description" (p. 59). The second cycle of coding was hypothesis coding and its goal is "to develop a sense of categorical, thematic, conceptual, and/or theoretical organization from your array of First Cycle codes" (Saldaña, 2012, p. 149). Saldaña (2012) also provides that hypothesis coding compliments magnitude coding where the researcher extracts the list of codes that are based on clusters of initial codes. In this stage of coding, initial codes were collapsed and consolidated to form axial codes or categories that informed the emergence of themes.
Before running paired sample t-test on pre and post MTEBI scores, we tested the assumption of normality using Shapiro Wilk and Kolmogorov-Smirnov testing significance at alpha value equals to .05 (See Table 4). Since both tests were not significant, we deduced that our data is normally distributed.  Table 5 and 6). As noted in Table 5, there is a shift in perspective from the PRE to the POST MTEBI showing more than 0.3 mean difference. On the PRE MTEBI, teachers were uncertain of their abilities to use design thinking in the mathematics classrooms. However, this belief has changed as indicated by a POST MTEBI mean close to 4, showing a significant increase in level of agreement from uncertain to agree. This alteration in perspective was supported by participants' interview feedback that confirmed their enthusiasm for using design thinking in the mathematics instruction. Furthermore, Table 6 validates the statistical significance of this shift at a p-value < .05.  (Cohen, 1988). This finding suggested that the professional development that teachers undertook was influential in enhancing their mathematics teaching efficacy.
Results of the qualitative analysis revealed five themes that described teachers' engagement and perception of effectiveness as reflected in levels of motivation to use robotics modeling in teaching middle-grade mathematics. Themes reported were categorized in terms of levels of affordances or constraints. Salomon (1993) defined an affordance as "the perceived and actual properties of a thing, primarily those functional properties that determine just how the thing could possibly be used" (p. 51), while constraints are defined as any physical or logical limitations and ways in which an object can be limited (Salomon, 1993). Three themes that described affordances were: 1) Creating collaborative spaces underpinned by design thinking affords transformative learning; 2) Problem solving through shared inquiry elevates confidence and allows meaning-making and openness to alternative perspectives; 3) building connections between mathematical concepts and real-life phenomenon supports a feeling of safety and the interest and willingness to take risks. The two other themes depicted constraints that teachers saw during implementation: 1) system support, resources, and funding are prerequisites to engage in modeling design; and 2) designated curriculum and teaching load restrains teachers from engaging in extra-activities that focus on design thinking.

Theme 1: Creating collaborative spaces underpinned by design thinking affords transformative
learning.
The initial reaction by teachers as they were immersed in modeling and design thinking was the impact that such activities could have on student learning outcomes. It was evident that as they participated in the professional development, teachers built greater confidence about their instructional practice and enhanced self-efficacy to develop enriching experiences for their students. As one teacher reflected, "I think my students will wholeheartedly engage in robotics activities because they enjoy working with hands-on projects. This approach will make the learning more exciting and relevant to understanding mathematics concepts/standards." Also, another teacher pointed out, "I feel now that I have been introduced to this training, my students will engage in robotics and engineering design activities often." Theme 2: Problem solving through shared inquiry elevates confidence and allows meaning-making and openness to alternative perspectives.
During observations and informal conversations with teachers, it was clear that teachers, challenged by their problem-solving abilities developed a sense of persistence to overcome their vulnerability and engage in non-conventional activities that extended beyond what they used to see. Furthermore, there was an interrogation of teachers' competence to develop activities using design thinking as one teacher noted, "If I have a working understanding of robotics I will be able to communicate and teach the curriculum effectively." Also, regarding student learning, one teacher said, "Students will be actively involved in robotics primarily because they can problem solve. Work cooperatively and to engage hands-on activities."

Theme 3: Building connections between mathematical concepts and real-life phenomena supports interest and willingness to learn new ideas.
A focus on experiential benefits for students was evident in teachers' interactions during the training and their classrooms as one a teacher explicated, "I believe that they[students] will engage in the process of robotics because of the hands-on manipulative which brings a real-world approach." Furthermore, there was a recurrent assertion that mathematics will be more interesting to students and thus they will be more engaged to learn new ideas. One teacher empathically states, "I think my students will be very excited and eager, it will be new to most so I think the exposure will initially capture their interest." Theme 4: System support, time, and material resources and funding are prerequisites to engage in modeling design.
There was a perennial emphasis on contextual factors that are necessary to ensure effective implementation of modeling design thinking in the schools. To illustrate, one teacher maintained, "Pilfering of materials by students is an issue … our school is very low socioeconomic status and students may take things … this ties in to the cost of resources and materials … our school and parents do not have lots of resources." Furthermore, time was an essential factor that teachers perceived as hindering any efforts to integrate design thinking into teaching. A teacher explained "I need time to figure out how to integrate robotics design in my classroom effectively as well as implementing districtmandated initiatives." More eloquently put, a teacher asserted "As long as I have the support and materials, my students will engage in robotics and engineering activities. This will definitely be done weekly, and I would also like to have mandatory 9-week projects related to robotics." Theme 5: Designated curriculum, classroom climate and teaching load restrain teachers from engaging in extra-activities that focus on design thinking.
As one teacher stated: "Time and pacing will always be a concern … Knowing how to integrate robotics with normal standards in an effective manner can be challenging". Another teacher, explained, "Discipline in the classroom is a problem … pieces of Legos will be flying all over the place." Other teachers highlighted similar constraints that relate to student behavior, "Students may lose structure or become frustrated and try to play …". Also, a teacher noted, "It can be challenging to keep students from breaking down to play or recess mode. Students can be poor stewards of breakable or expensive equipment. Other teachers lost TI-83 calculators, but I was lucky." This study investigated the use of a detailed program designed to prepare teachers and equip them with tools and strategies to teach integrative curriculum and motivate student performance. The professional learning experience had a foundation in engineering design and mathematics and addressed the use of new technology (Robotics) in the classroom and thinking processes (Engineering Design Thinking). Engaged in activities that related the acquisition of technical skills embedded in real world context through the lens of design thinking have enabled the teacher to coach students to creatively and innovatively solve problems.
During the study, teachers were situated in a design context that required them to use design thinking tools needed to explore the phenomenon. Teachers first experienced the problem as students, then they were immersed in the problem situation reflecting on their actions during each phase of the design thinking process. Teachers collaborated during each phase of the cycle and discussed challenges and strategies with each other throughout the training. Their teamwork afforded time to sharpen technical skills in the area of robotics and 3D printing. At completion, the teachers discussed how their STEM teaching practices have changed and improved. The impact on the in-service middle-level teacher's mathematics teaching efficacy was evident in instructional planning, preparation, and motivation. Brown et al. (2011) contend that there has been a lack of understanding of what STEM education is in schools today. By the same token, Saxton et al. (2014) assert that a focused perspective of STEM aids the design of instruction and assessment. Furthermore, Chung, Cartwright, and Cole (2014) argue that students who engage in robotics through extra-curricular competitions experience higher performance scores in mathematics and science. In a similar vein, Barrett, Moran, and Woods (2014) aver that student participation in extracurricular engineering activities enhances student scientific inquiry skills. Although these studies showed an increase in performance and engagement, the interventions studied occurred outside of the school day. This study focused on training middle school teachers to incorporate robotics and engineering design thinking in their mathematics classrooms and documented the enhanced effect on their mathematics teaching efficacy and their experiences during implementation. We argue that engaging teachers with modeling explorations facilitated an understanding of how design thinking can provide opportunities to interrogate existing instructional practices and forge an identity as STEM (Science, Technology, Engineering, Mathematics) practitioners. Teachers during professional development were situated as engineers solving engineering design challenges and developing lesson plans to implement in their classrooms. Bybee (2013) affirms that for the United States to gain and maintain global competitiveness a greater effort must be made to educate STEM literate students to form a 21 st -century workforce.

CONCLUSION
As emphatically reflected in this study, participating in the art of learning by doing is what grabbed the attention of the teachers and engaged them in meaningful innovation, creation, and experimentation with ideas. It was clear that exposure to technologically novel inquiries has equipped teachers with thinking skills that helped them form meaning and create strategies needed to aid in the problem-solving process.
Establishing a STEM community during professional development and beyond, where fear was lessened and opportunities for deep conceptual mastery prevailed, afforded teachers more confidence, motivation, and persistence for learning. This in turn reflected on their willingness to foster meaningful learning in their classrooms by immersing the students in transformative experiences that challenge their understanding of the world and encourage them to take control of their learning.