AN ANALYSIS OF LEARNERS’ SOLUTION STRATEGIES IN THE CONTEXT OF MODELLING TASKS

It remains a challenge for teachers to integrate modeling tasks in everyday mathematics classes. Many studies have been conducted that show the difficulties faced by teachers. One of the challenging aspects in this regard is that of assessment. In the present study, a connection between structures of learners’ solution strategies and cognitive considerations is established to develop a practice-oriented instrument to determine and assess the complexity of solution strategies of modeling tasks. In this paper, the selected learners’ strategies’ structure was analyzed in-depth to identify the underlying cognitive structure. The results show that thought operations carried out in parallel complicated a solution strategy. However, the results also support a purely sequential thought operation approach without weighting parallel thought operations, which corresponds to an intuitive assessment procedure by mathematics teachers. As assessment is a great challenge for many teachers in the context of modeling tasks, this study provides a promising frame of reference for further research in this important domain of assessment and modeling.

ended nature of these modelling tasks. Open-ended tasks lack predictability and make use of a variety of solution strategies, thus complicating assessment (Schmidt, 2010). This requires teachers to adopt a pedagogy that reflects this openness to working with different solution strategies. This paper reports on a bigger research study which specifically focused on a particular method to determine the complexity of solution strategies of modelling tasks. It is hoped that the findings of this research might lead to easier manageability of modelling tasks and can also be a starting point for alternative ways of assessment in the teaching of modelling tasks. Therefore, the study considered the following main question: does a thought structure analysis, together with its underlying cognitive and structural assumptions, lead to a suitable characterization of the complexity of solution strategies to modelling tasks?
When assessing the solution to mathematics tasks, their numerical value is often taken as the dominant indicator of a right or wrong solution. When dealing with modelling tasks, however, it is often not meaningful to merely assess a solution's numerical value due to the multiple solution nature of these modelling tasks. A common observation is that modelling tasks cannot be assessed as objectively as traditional task formats (Spandaw & Zwaneveld, 2010). Studies show that assessment of modelling tasks is a major challenge, which often hinders mathematics teachers from integrating modelling in their mathematics classes (Jensen, 2007). The complexity of the solution is a critical measure when assessing student solutions (Shanta & Wells, 2020). Therefore, if we want modelling to be part of mathematics teaching, ways of practical assessment need to be found. This study was thus undertaken to determine the complexity of solution strategies to selected modelling tasks, which can then form the starting point for further assessment considerations.

The Role of Cognitive Structures
A common procedure when solving mathematical problems in general is what Bourbaki (1961, p.163) refers to as "recalling structures". In school mathematics learners are often taught to structure a solution in terms of "a question", "calculation" and "the answer". Those structuring procedures are meant to relieve and support the learners' thinking processes and help to objectivize the individual solution strategies. The latter is closely connected to assessment practices where structures also play an important role. A common assessment procedure that mathematics teachers often use is to provide a sample solution in advance and to identify the necessary intermediate steps to attain a solution which is then scored. Because solutions to modelling tasks are often open-ended and can vary widely, there is, however, no unique sample solution. This complicates an objective assessment of solutions.
The common, mostly intuitive, assessment procedure mentioned above has similarities with structural considerations (e.g. the use of arithmetic trees, Figure 1) within the field of word problems. Breidenbach (1969) looked at the structural-substantial complexity of a word problem among other things, to determine its complexity. For example, in the task illustrated in Figure 1 there are three variables in the first step, where the third variable is uniquely determined from the other two by one operation (in this case it is multiplication). Breidenbach called this arrangement "Simplex". A linking of several Simplexes in the solution process is called a "Komplex". Further developments by Winter and Ziegler (1969) lead to the arithmetic tree illustrated in Figure 1, which is still used in mathematics textbooks today.
An obvious, but so far empirically not validated conclusion, is that a large number of Simplexes and a complicated nesting of them, have an effect on the complexity of the tasks' solutions. Cohors-Fresenborg, Sjuts, and Sommer (2004) describe task complexity, inter alia, by the criterion "cognitive complexity" of thought processes. This allows for simultaneity or nesting of what we call "thought steps". In this context Kaune (2000) also distinguishes between two aspects of cognitive complexity.
These are firstly the processing of multiple information within one thought step, and secondly the influence of previous information and data. Figure 1. An example of an arithmetic tree for a selected task according to Winter and Ziegler (1969) Within the modelling discourse, cognition and assessment of thinking has particular relevance to study how students deal with modelling tasks. Blum and Ferri referred to students' difficulties with modelling tasks by the tasks' cognitive demand (Blum & Ferri, 2009, p. 46). Tanner and Jones investigated the influence of (meta)cognitive skills on modelling (Tanner & Jones, 1993; and assessed, inter alia, cognitive ability.

Cognitive Complexity
The present study investigates the cognitive complexity of a solution strategy by analyzing the structural complexity represented by its arithmetic tree-like structure as illustrated in Figure 1. The coherence of structural considerations and cognitive theories play an important role in solving modelling tasks. In their study, Fletcher and Bloom (1988) assumed that text comprehension is an important problem-solving process in itself, where the problem solver must find a causal chain to link the start to the end of the text. In order to form these links, Fletcher and Bloom (1988) posited that the information concerning the task must be kept in the mind of the problem solver. Results of their study show that it is important for the problem solver to access the immediately preceding information before proceeding to the next step in the causal chain. It can be concluded that the role of the working memory is to keep key information available. This is necessary to link old and new information (Baumann, 2000). By relating these findings to structural considerations of a particular solution strategy represented as an arithmetic tree, statements can be made about its cognitive complexity. On the one hand, an arithmetic tree-like structure ( Figure 1) can be interpreted as a causal chain since the start (the given information in the task text) and end (the solution of the task) is linked by chain links. In the context of solving mathematics tasks we refer to these links as intermediate steps in the solution process. On the other hand, the immediately preceding information can be identified as relevant intermediate steps.
Thus, it can be concluded that the immediately preceding intermediate step must be kept active in the working memory to master the next step in the chain. The assumption that mental processing capacity is limited (Sweller, 1988), leads to the conclusion that a plethora of information which has to be kept active at the same time, can complicate the solution process. Thus, it can be concluded that the working memory load is dependent on the number of intermediate steps necessary to master the current intermediate step in the problem-solving process. This means that the load of the working memory increases with the increasing amount of information needed at various points in the solution process.
Based on these theoretical considerations the following definition of thought structure was taken fundamental for the present study (Reit, 2016, p.  A thought structure thus illustrates the logical sequence of thought operations (Reit, 2016, p. 60).
The present study establishes a framework to structurally analyze student solutions to modelling tasks with the aim of determining their complexity. It is assumed that the number of sequential and parallel thought operations provides information about the cognitive complexity of a solution strategy.

METHOD
This study is framed by three main phases (see Figure 2 for phase one and two). In phase one an empirical study was conducted with 1800 student solutions to selected modelling tasks. The different solution strategies were identified. In phase two of the study, a so-called thought structure analysis (Reit, 2016) was conducted. Here the structures of the various solution strategies were analyzed by identifying their specific thought operations. At this point the number of parallel and sequential thought operations were identified. To translate this data quantitatively, statistical models were developed.
Some of these models differentiated between sequential and parallel thought operations by applying a weighting to the latter. Of particular interest were the differences in the models that applied or did not apply a weighting to parallel thought operations, as we wanted to determine which model could most appropriately identify the complexity of the solution strategies of modelling tasks. In a third phase the evaluation of the complexity models took place. The theoretical complexity of the solution strategies was statistically compared to the average score of the student solutions as a measure for the empirical complexity. This was based on a rubric. In particular, we analyzed which complexity model led to a statistically verifiable and substantially reasonable coherence between complexity and score in terms of "high score, low complexity" and vice versa. These results finally yielded information about the influence of parallel thought operations on the complexity of the solution strategies. For the bigger study referred to above, five modelling tasks were developed according to predefined criteria (see Reit & Ludwig, 2015a;2015b). Three of these five tasks were randomly selected and published in a booklet which was distributed to the 1800 Grade 9 students (15 years of age) from German grammar schools. Each student was asked to solve the three tasks individually. Each student had 45 minutes to complete the booklet. For the purpose of this paper we will discuss "the Potato problem" as illustrated in Figure 3. The students were asked to mathematically estimate the number of French fries which can be cut out of a single potato. . In a second step, each solution strategy was analyzed to reveal its thought structure -see Figure 4 for the Potato modelling task. In particular, single thought operations were identified and a thought structure was set up. This provided information about the number of parallel and sequential thought operations that took place within the respective solution strategy. Figure 4 illustrates a particular student solution to the potato modelling task, representing the solution strategy layer and its corresponding thought structure. The potato and the French fry were approximated to and represented by cuboids. By calculating the number of French fries fitting into height and width of the potato and multiplying these results, the total number of French fries could be mathematically estimated (Figure 4). The thought structure illustrates the thought operations needed for the next step in the solution process. This can be represented as a vector representation ((3,1,1) for the solution strategy in Figure 4), indicating the number of parallel thought operations in each thought structure level. So, in this case there are 3 thought structure levels which have to be mastered consecutively to come to a final solution. In the first level there are 3 thought operations to be linked together mathematically correctly to come to the second level with one thought operation, and in the third level there is also one thought operation to be mastered. It is recognized that besides the cognitive aspects, linguistic formulations also play a role when characterizing the complexity of a modelling task (e.g. Cohors-Fresenborg, Sjuts, & Sommer, 2004;Walzebug, 2014, p. 161). Since these aspects are not considered by the thought structure model explained above, the complexity of the task text is integrated as a separate factor (see Reit 2016, pp. 103). A scheme following Cohors-Fresenborg et al. (2004) was developed by Reit (2016, p. 104) which classifies the complexity of the tasks' text and adds to the cognitive complexity of a student solution.

Complexity Models
The empirical complexity represents the average (normalized) score of the student solutions being clustered to the respective solution strategy. The thought structures currently provide a vectorial representation which we subsequently translated into a scalar. The question arises whether parallel thought operations complicate a solution strategy. Several models (Table 1) were used to translate the vectorial representation of the number of parallel and sequential thought operations into a scalar value.
They differ in the kind of weighting of the parallel thought operations. The aim of determining a theoretical complexity was not primarily to find the best possible fit of a possible complex complexity model, but rather to find a model that could be used in school practice, and that, at the same time reflected the data well.

Table 1. Different Complexity Models Applied to the Potato Modelling Task
The addition model in Table 1  An obvious way to translate a vector into a scalar is to calculate its norm. This is referred to as the norm model. This leads to a quadratic weighting of parallel thought operations as is also done by the quadratic model. Another possibility of operationalizing the theoretical complexity is based on the consideration that the sequence of processing thought operations on the same level in the thought structure is arbitrary. This content-related notion is reflected by the factorial model. Following the factorial model, parallel thought operations are seen to be cognitively more demanding and thus result in a more difficult solution strategy than sequential ones.
For this purpose, a rubric was set up based on the thought structures of the solution strategies so that each thought operation corresponded to an essential intermediate step of the solution. Identifying and assessing essential intermediate steps and scoring them corresponds, we argue, to the common practice for evaluating written student solutions in mathematics. Depending on whether a thought operation was executed completely correctly, with errors, or not at all, 1, 0.5 or 0 credits were awarded. Followup errors were considered, as calculation errors should not be excessively weighted.

RESULTS AND DISCUSSIONS
In this section a comparison is drawn between theoretical difficulty, resulting from the thought structure analysis and operationalized by different complexity models, and the average score of a solution strategy as a measure for its empirical complexity. At first, solution strategies are considered and compared to each other. Then the method is extended to complete modelling tasks where both theoretical and empirical difficulties of the tasks' solution strategies are averaged to characterize the task complexity. To compare theoretical and empirical complexity, a power regression is implemented with a calculation of its goodness of fit in terms of a pseudo-R²-value. This allows for a statistical evaluation of the substantially reasonable relationship that "the more difficult a solution strategy is, the lower is the respective score". Furthermore, this supports a substantial conclusion with regard to the influence of parallelism of thought operations on the complexity of a solution strategy.

Complexity of Solution Strategies
A comparison of the theoretical and empirical complexity depending on the respective complexity model (see Table 1 These results suggest that parallelism of thought operations has an influence on the complexity of the solution strategy, although the kind of weighting used in the analysis of these solution strategies seems to be crucial. This is also underlined by the fact that the application of the addition model can lead to better results than most of the weighting models (maximum, norm and quadratic model) which nevertheless also indicates that a pure addition of thought operations might also be a reasonable analysis strategy.

Figure 5.
Comparison of the average score as a measure for the empirical complexity and theoretical complexity of solution strategies by a power regression and its goodness of fit in terms of a pseudo-

R²-value
The main question this paper asks is whether the thought structure analysis together with its underlying cognitive and structural assumptions, leads to a reasonable and suitable characterization of the complexity of solution strategies of modelling tasks. The results presented above suggest that the factorial model is a suitable analytical tool to analyze the complexities of solution strategies when solving modelling tasks. The factorial model capitalizes on, and considers, the complexities associated with parallel thought operations, which aligns well with Sweller's cognitive load theory (Sweller, 1988).
The good performance of the factorial model confirms that parallelism of thought operations has an influence on the complexity of solution strategies.
Nevertheless, it should also be emphasized that that addition model, as a non-weighting model leads to statistically good results. This suggests that an over-reliance of weighted complexity models and the influence of parallelism of thought operations on complexity cannot be assumed. Furthermore, the statistically good results of the addition model also support the so far intuitive procedure of many mathematics teachers' assessment of mathematics tasks. A common practice among teachers' assessment of learners' solution strategies is to identify reasonable intermediate steps in a solution which are worthwhile scoring. Assuming that the assessment of solutions to a mathematics task is strongly determined by the complexity of the task, the results of the addition model supports the so far intuitive assessment practice, and we suggest that this practice can also be applied to modelling tasks. Baird, Hopfenbeck, Newton, Stobart and Steen-Utheim (2014, p. 21) state that "assessments define what counts as valuable learning and assign credit accordingly". It is also supported by Niss' proverb that "what you assess is what you get" (Niss, 1993).

CONCLUSION
Assessment is of great importance in prescribing and guiding what happens in a mathematics class. Notwithstanding that assessment is a great challenge for many teachers in the context of modelling tasks, this study provides a promising frame of reference for further research in this important domain of assessment and modelling.

ACKNOWLEDGMENTS
This work was funded by Stiftung Polytechnische Gesellschaft Frankfurt. Thanks to Jean Schäfer for proofreading the paper.