Angeliki Mali
email: a.mali@rug.nl

Anne Cawley
email: acawley@cpp.edu

Irene Duranczyk

Vilma Mesa
email: vmesa@umich.edu

April Ström
email: april.strom@cgc.edu

Laura Watkins
email: laura.watkins@gccaz.edu

Identifying sense-making in algebra instruction at U.S. post-secondary colleges

Keywords: Instruction, College mathematics, Postsecondary education, Community colleges, Instructional quality

As part of a large-scale study of algebra instruction at two-year post-secondary institutions in the United States, we have developed a qualitative video analysis framework, Evaluating the Quality of Instruction in Post-secondary Mathematics (EQIPM). In this paper, we present two of twelve characteristics used to describe instruction (Instructors Making Sense of Procedures, and Student

Mathematical Reasoning and Sense Making), which describe students and instructors engaging with mathematics so their ideas or ways of thinking are evident. We use examples taken from our corpus of videos to exemplify these two characteristics of instruction and how they contribute to its quality. We also offer suggestions for professional development activities.

In the United States, 43% of all undergraduate students enroll in public post-secondary institutions that offer courses for the first two years of their undergraduate degree. These institutions, hereafter called community colleges, provide students with many options to further their educational goals, some of which include remediation, transfer to university undergraduate programs, vocational training, general education, continuing education, and workforce development [START_REF] Blair | Statistical abstract of undergraduate programs in the mathematical sciences in the United States[END_REF]. Community colleges target diverse students who are mostly non-traditional (e.g., older, working, or with family responsibilities), offer flexible schedules, and charge very low tuition compared to universities. Because of the wider range of students who enroll, these colleges offer a broad range of mathematics courses, from developmental mathematics (designed to prepare students for collegiate level study of mathematics) to mathematics courses taught in the first two years of an undergraduate major. The failure rates in algebra courses at community colleges range from 30% to 70% [START_REF] Bahr | Revisiting the efficacy of postsecondary remediation: The moderating effects of depth/breadth of deficiency[END_REF]. Because algebra skills are of paramount importance for advancing into higher level mathematics, investigating algebra instruction at community colleges becomes important; however, community colleges are largely under-researched [START_REF] Mesa | Mathematics education at public two-year colleges[END_REF].

In Algebra Instruction at Community Colleges: An Exploration of its Relationship with Student Success (AI@CC, [START_REF] Watkins | Algebra instruction at community colleges: Exploration of its relationship with student success: National Science Foundation[END_REF], we seek to investigate the relationship between the quality of characteristics of algebra instruction and student learning as evidenced by student grade as a percent. In this paper, we exemplify two characteristics of algebra instruction at community colleges, Instructors Making Sense of Procedures, and Student Mathematical Reasoning and Sense Making. We focus on observed instances that evidence mathematical sense-making1 , to start addressing the wider question: What is the nature of sense-making evidenced in algebra instruction at the community college level of post-secondary mathematics education?

Background and supporting literature

We used the instructional triangle to define algebra instruction as the interactions between instructors, students, and the content studied in community college environments (Figure 1, [START_REF] Cohen | Resources, instruction, and research[END_REF]. To characterise algebra instruction, we drew on two frameworks that analyse the quality of instruction created for school-level mathematics education (Mathematical Quality of Instruction (MQI), Hill, 2014; Quality of Instructional Practices in Algebra (QIPA), [START_REF] Litke | The state of the gate: A description of instructional practice in algebra in five urban districts[END_REF]. These frameworks describe and code (with ratings from 1 to 5) instructional practices from video-recorded class sessions by rating consecutive 7.5-minute segments. Although the two frameworks provided useful language to describe the characteristics of instruction, the actual descriptions were tied to elementary or high-school content and interactions that bore little resemblance to instructional practices at community colleges. We retained some of their characteristics, but modified the descriptions for our purposes, through multiple iterations that involved the use of video recordings of 15 class sessions in Fall 2016, ranging between 45 and 120 minutes (see [START_REF] Cawley | An innovative qualitative video analysis instrument to assess the quality of postsecondary algebra instruction[END_REF]. From this work we developed the video-coding framework called Evaluating the Quality of Instruction in Post-secondary Mathematics (EQIPM). This paper includes two EQIPM characteristics, Instructors Making Sense of Procedures and Student Mathematical Reasoning and Sense Making, which emphasise ways in which instructors or students engage in algebraic sense-making in community colleges. Version 3.0 of EQIPM is presented in Figure 2 2 . The definition of ambitious mathematics instruction (NGA, 2010) supports 11 of 12 characteristics of instruction in EQIPM. In ambitious instruction, the instructor engages students in challenging tasks, observing and listening while they work on those tasks providing an appropriate level of support to diverse learners. The instructional goal is to ensure that all students do high quality work instead of only expecting fast and correct execution of mathematical procedures. Classroom Environment emerged from our observations of community college lessons. This characteristic is informed by [START_REF] Danielson | Danielson 2014-15 Rubric: Adapted to New York Department of Education Framework for Teaching Components[END_REF] who advocated that in a respectful and open environment, student opinions are encouraged, and student contributions are validated in ways that make them feel safe. The American Mathematical Association of Two-Year Colleges' IMPACT 3 guide (2018) presents a vision to improve mathematics education in the first two years of college. The IMPACT guide concludes that for students to achieve proficiency in mathematics it is necessary for them to, among other things, (1) know mathematics procedures and execute core computations fluently, (2) make sense of and solve problems, and (3) demonstrate evidence of mathematical understanding (AMATYC, 2018, p. 25). Research by [START_REF] Star | Reconceptualizing procedural knowledge[END_REF] provides additional evidence of the importance of linking conceptual understanding with procedural skills.

Methods

The data for this paper were collected in the Fall of 2017. Trained observers video-recorded all class sessions on two of three topics from 40 instructors: linear, rational, or exponential equations and functions. The instructors, who volunteered to be observed, were at six diverse community colleges, which represent a range of institution size, degree of urbanicity, region (Southwest, Midwest, and Central in the U.S.), and student background. This round of data collection resulted in 131 class sessions, which ranged in duration between 45 and 150 minutes. Fifty-four percent of these sessions were solely in a lecture context and 85% included some elements of a lecture.

Starting in the Summer 2018, 17 trained coders used EQIPM to rate the characteristics of instruction that appeared in each segment 4 (n = 1,576 coded segments of which 1,236 are full 7.5minute segments) of every class session. For each segment, the quality of each characteristic was coded along the 12 characteristics using a rating from 1 to 5. The coders provided a justification for their rating, including the specific time stamp in which the coder noticed evidence that justified that rating, and a brief explanation of the quality of the characteristic based on specific guidelines of EQIPM. For the most part, a rating of 1 was reserved for "no evidence" except for the Mathematical Errors and Imprecisions in Content and Language in which the ratings are reversed. A rating of 3 was considered an acceptable quality, usually indicating a modal rating. A rating of 5 was reserved for exemplary cases; cases we interpreted as best examples of that characteristic against EQIPM. Ratings of 2 and 4 were reserved for 'in between' cases. When some features were present but they 3 Improving Mathematical Prowess and College Teaching (IMPACT) 4 Hill et al. ( 2012) used a segment length of 7.5-minutes after several comparisons of coding using various lengths. All lessons were parsed into 7.5-minute segments; the last segments of the lessons were usually shorter than 7.5 minutes.

Figure 2: Dimensions and characteristics of instruction in EQIPM

were of less quality than what a rating of 3 would have been, a 2 was assigned. When the quality was better than a rating of 3, but not truly illustrative of an exemplary quality, a 4 was assigned.

Instructors Making Sense of Procedures refers to instructor work that fosters students' sensemaking about how a procedure moves from one step to the other, rather than merely reproducing the steps of the procedure [START_REF] Star | Reconceptualizing procedural knowledge[END_REF]. For this code, the coder is trained to assess the extent to which instructor practices of using mathematical definitions, symbols, properties, structures, examples, and interpretations of solutions motivate understanding of a mathematical procedure. Examples of such instructor practices include, but are not limited to attending to the type of solution generated by a procedure and its interpretation, using a series of examples of an abstract formula to extract key features, and clarifying symbols of definitions. For a 7.5-minute segment, an exemplary 5 rating of the characteristic is assigned in the event that the coder interprets that the practices identified motivate mathematical understanding and belong to either a range of instances or a sustained, prolonged instance with instructor practices for fostering student sense-making. A rating of a 5 only indicates that sense-making happened in an exemplary way, attending to elements of a procedure or the algebraic concepts in a procedure. Other codes in the framework will pick up additional elements of instruction that together may allow one to determine high-quality instruction.

Student Mathematical Reasoning and Sense Making entails the components of reasoning and sensemaking for students. Mathematical reasoning includes student practices such as drawing logical conclusions from patterns identified, providing conjectures, offering counter-claims in response to a proposed mathematical idea, and mentioning the mathematical properties that underlie a procedure. Evidence for sense-making includes student practices such as asking mathematically motivated questions while requesting explanations, leveraging previous mathematical ideas while responding to a question, and interpreting the solution generated by a procedure. A 5 rating includes either a combination of instances that meet these criteria or a sustained instance that saturates the 7.5minute segment with student reasoning or sense-making so that student ideas are evident and contribute to the development of the mathematics.

Coders adhered to three additional rules when coding: (a) assigning a higher rating rather than a lower rating when in doubt, (b) rating each segment in isolation from other segments within the same class session, and (c) focusing on evidence that was seen and said in the video, rather than what coders thought should be in the video. Following Hill (personal communication, September 2017), 10% of randomly selected class sessions were double-coded; the coders held calibration meetings to discuss discrepancies in ratings and agree upon final ratings. The data used for the analysis reported in this paper came from the coding of 903 segments.

Findings

EQIPM offers a close and granular examination of the quality of algebra instruction at community colleges, but no element of EQIPM can stand alone to suggest high-quality instruction. Breaking down instruction into parts is a first important step towards efforts to understand the current status of instruction at community colleges, and a valuable activity that may help instructors develop their teaching skills. Qualities we interpreted as exemplary included cases of algebra instruction identified with a 5 rating. Only 89 out of 920 segments (10%) were identified as exemplary in at least one of the 12 codes. There were 398 segments (43%) with a rating of 4 and 5. We identified 29 exemplary segments (5,3%) where instructors supported students in making sense of procedures and seven segments (1%) where students demonstrated exemplary mathematical reasoning and sense-making in class. Examining instances with a rating of 4 or 5, the cases increase to 95 (10%) and 37 (4%) respectively. In light of these percentages in the community college setting, we think that there is much room for improving instruction, and existing practices of exceptional quality can act as resources for improving instruction and student outcomes.

Instructors making sense of procedures

In a lesson on rational expressions, Instructor 110 discussed with the students the procedure of finding domain restrictions. She started her lecture by asking what is and why is undefined, gradually moving student attention to rational expressions and the denominator of . She asked students to find -values for which the aforementioned rational expression is undefined, and students suggested . After the instructor made the calculations for the suggested value and confirmed that the outcome is , which is undefined, she asked whether there were suggestions for other values for which the rational expression is undefined. She waited for 11 seconds for a response, and suggested for which the expression becomes . The students recalled this fraction equals zero so the rational expression is defined for . The instructor then gave the students time to find domain restrictions for a series of more complicated examples of rational expressions with denominators that are binomials, trinomials, and other polynomials (e.g.,

and

). The use of instructor practices was rated as an exemplary case for this characteristic, because the instructor attended to the type of solution (i.e., denominator equals zero) generated by the procedure of finding domain restrictions, first using integers, which the students are familiar with in this level of education, then interpreting increasingly complicated examples of rational expressions. The students successfully suggested -7 for and related back to the information in the beginning of the lecture.

In a lesson on exponential functions, Instructor 102 discussed the reason behind the growth pattern of exponential functions. He created tables of values of a linear function, a quadratic function, and an exponential function in which the x-values started at 0 and increased by 1. He calculated the difference of consecutive y-values of each of the three functions, and found that for the linear function the difference was always 2, but for the quadratic function the difference each time increased by 2. He generalised the finding about the differences by saying for both functions that "It doesn't matter it is a 2, it could be any constant." He then found that for the exponential function the difference of consecutive y-values, even the difference of the difference of consecutive yvalues, would never become constant or increase by a constant value; rather, the difference of consecutive y-values will repeat the growth of the exponential function. The instructor's use of the structure of three kinds of functions-a linear, quadratic, and exponential-to clarify the meaning of exponential growth was rated as an exemplary case for this characteristic. In particular, he used various numerical y-values of the functions and their differences to find patterns and exemplify the exponential growth. Later in the lesson, all students went to the board to determine the type of growth of various types of functions and they were asked to classify the growth of functions as either exponential or linear.

Student mathematical reasoning and sense making

In a lesson on rational expressions, Instructor 610 worked with the students on simplifying rational expressions. After the first half hour of the 150-minute lecture, the instructor asked students to tell her what simplifications she should carry out for the expression . A student correctly suggested the instructor to write , to factor the polynomial in the numerator and denominator, and to simplify the rational expression. Another student correctly suggested for the expression that the instructor could multiply both the numerator and the denominator of by 3 and perform all calculations needed. A third student correctly suggested for the expression that the instructor multiply the numerator and the denominator of by 3, the numerator and denominator of by 5, and perform all calculations needed. This instance where different students contributed to the lecture by leveraging the mathematical idea of Least Common Denominator (LCD) in various cases (e.g., the LCD is the same as or different from both of the denominators) while simplifying expressions lasts for about two of the 7.5 minutes of the segment. A follow-up instance in the same segment lasted for about four and a half minutes. It included a fourth student who orally provided the calculations he performed to find the LCD for the expression

. By factoring the latter expression, it becomes . The student reasoned the LCD was because the LCD included the highest exponent from all multiples of the factors of the denominators. Two other students found the LCD for each of two different rational expressions, using their fellow student's reasoning. The combination of two prolonged instances that saturate the segment with student reasoning and sense-making so that student inputs contributed to the development of the meaning of the LCD was rated as an exemplary case for this code.

A rule for deciding on the rating of Student Mathematical Reasoning and Sense Making was that there must be clear evidence of students engaging in mathematical reasoning or sense-making through verbal utterance(s) and/or through written work. A limitation for finding such evidence was that despite the premium quality of our audio-visual equipment, the recordings sometimes did not capture clearly student voices or written work. Although we found instances that saturated the 7.5minute segments with student reasoning or sense-making, the audible or visible evidence found within a segment could not qualify for an exemplary case. That resulted into lowering the quantity of exemplary cases found for this characteristic of instruction in the community college classes.

Discussion

This paper described two characteristics of instruction captured in EQIPM, Instructors Making Sense of Procedures and Student Mathematical Reasoning and Sense Making. The common theme in the three examples given can be summarized as 1) a sustained effort to provide sense-making behind procedures by use of a variety of examples in varying complexity while 2) prompting students to provide solutions to the examples with an explanation of their reasoning. The instructors' approach paved the way for students to develop proficiency in mathematical procedures; this is a new insight into community college instruction because educational research in this context is still rather limited. Rules were not given to simply be followed but ideas were presented with concrete examples to apply sense-making. We currently work on improving the codes and ratings of EQIPM. In recent iterations for the development of EQIPM, we widened the mathematical content of the codes of the dimension Instructor-Content interaction. Instructor Making Sense of Procedures, for example, includes sense-making whether it occurred while supporting a procedure, a mathematical idea or ways of thinking.

In Instructors Making Sense of Procedures, instructor practices that motivated sense-making included the search for patterns, the use of integers or numerical y-values of functions which were familiar to students, and the careful selection of examples with increasing rigor. The latter can help students build their own understanding of rate of change, and later connect this idea to the concept of derivative in calculus. In Student Mathematical Reasoning and Sense Making, the instructor offered different students the opportunity to suggest how they would simplify rational expressions and to justify their choices, sometimes using their peers' language and reasoning. This practice also provides the instructor with the opportunity to overlay student discussion with more precise language and connect student ideas. Such video-recorded instances of exceptional qualities of algebra instruction at community colleges can be leverage points to discuss with instructors, compare, and build up the current instructional practices that are in place. EQIPM nevertheless is not intended to become a score system to evaluate teachers. It is created to understand what the characteristics are that constitute instruction at community colleges and what features make that instruction of high quality.

The AMATYC IMPACT guide provides a vision for mathematics instruction in the first two years of college and identifies key attributes that point to the importance of students making sense of mathematics and demonstrating evidence of mathematical understanding (AMATYC, 2018). The opportunities for students to develop these attributes, as evidenced by our video analysis, is not common practice at community colleges. Our findings indicate that exemplary implementation of Instructors Making Sense of Procedures, and Student Mathematical Reasoning and Sense Making occurred in only a small number of cases. Professional development using instances of exemplary practices that support high quality algebra instruction can help change the landscape of mathematics instruction at community colleges by helping instructors provide opportunities for their students to develop key attributes tied to mathematical proficiency. In the next phase, we will work to identify ways for instructors to support students to develop mathematical proficiency and to subsequently improve student performance and preparation for higher mathematics.

Figure

  Figure 1: Instructional Triangle (Cohen, Raudenbush, & Ball, 2003)

We use the NCTM (2009) definition of sense-making which accounts for students discovering coherence across mathematical domains and seeing connections between new concepts and their existing knowledge.

The development of EQIPM is ongoing. See[START_REF] Mesa | The structure of EQIPM, a video coding protocol to assess the quality of community college algebra instruction[END_REF] for the most recent changes in the code structure.

Acknowledgment

Funding for this work was provided by the National Science Foundation Award EHR #1561436. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. We also wish to thank the community college faculty who participated in this study. Without them, this investigation would not have been possible.


Vilma Mesa
email: vmesa@umich.edu

Angeliki Mali
email: a.mali@rug.nl

Elena Castro-Rodríguez

University Student Use of Dynamic Textbooks: An Exploratory Analysis

Keywords: Textbook research, instruction, educational resources, didactical tetrahedron, linear algebra

Within the array of resources for teaching and learning, the textbook continues to be the most prevalent one for instructors and students. Textbook formats have been changing from paper to digital, open source formats, including sophisticated tools such as computing cells, annotation tools, and powerful search engines, easing access at relatively low cost. Importantly, open source textbooks never expire or go out of print and can be distributed at no cost to students, making them practically fully accessible. The main study seeks to describe how instructors and students use two open-source, technologically enhanced textbooks (linear algebra and abstract algebra for math majors). We use [START_REF] Rezat | From the didactical triangle to the socio-didactical tetrahedron: Artifacts as fundamental constituents of the didactical situation[END_REF] didactical tetrahedron to investigate how resources support instruction [START_REF] Cohen | Resources, instruction, and research[END_REF]. The tetrahedron suggests a relationship between textbooks, students, instructors, and the content at stake and suggests that instructors, and the nature of linear algebra, can influence the interactions between the students and their textbooks. We investigate the use of the textbook by the students (student-textbook interaction, arrow a in Figure 1) and how linear algebra chapters of the textbook and instructor planning influence such use of the textbook by the students (arrows b and c, respectively, in Figure 1.)

We analyzed bi-weekly log1 data from 102 students from four instructors in four different states who were using a dynamic linear algebra textbook [START_REF] Beezer | First course in linear algebra[END_REF]. The textbook can be accessed on any device with any commonly available browser. It is online, free, open-source, enhanced with computational cells.2 It includes standard linear algebra chapters (e.g., systems of linear equations, matrices, vector spaces, etc.) and is written in a definition-theorem-proof format, with exercises and some solutions towards the end of each chapter. The textbooks used in the project have a tracking system that allows to identify which sections of the textbook are being viewed and for how long.

The analysis of the viewing data revealed, unsurprisingly, that viewing tended to occur during the days when the classes were offered (mostly during class sessions), close to exams days, or when homework was due. The students mainly used solutions of exercises-in 17,405 viewings, 81% of the viewing time was for solutions of exercises, 15% for examples, and 5% for all the other elements. In the log responses students reported that they checked the textbook the day before class or the last day of their break; they also used it to study for the upcoming class, or when they were stuck, missed class, or had not understood their instructor's explanation. Students reported using mainly problems, exercises, and examples as they were preparing for class. When asked about their use of theorems, definitions, and examples, students said those were mainly used when producing notes for later use because they wanted to make sure they were connecting ideas and knew the basic definitions. In addition, students created class notes, homework documents or solutions, and textbook notes in order to improve their understanding, for practice, and reminders or memorization, and used many other resources (classmates, Internet, Google, YouTube, Chegg, Khan Academy, and class lecture videos).

We have been tracing how the various uses students reported relate to the specific content covered in the courses and to decisions instructors had made about the course. We noticed that students did not use features that were not required by their instructors; in general, the students (and their instructors) seemed reluctant to take full advantage of novel features (such as the programming cells). We speculated that by itself, the design of the textbooks was insufficient for pushing users into adopting different ways of using these textbooks in teaching and learning linear algebra. Instructors might need training, perhaps in the form of conversations with designers and authors or in authorship that takes advantage of the open nature of the textbooks, so that they can envision new ways in which these textbooks can be used for teaching and learning. Textbook production is expensive, and thus, research that documents how open access textbooks can be made widely available is important. Yet, without knowing how to best take advantage of the new technologies, we might not realize their potential within mathematics classrooms.

Figure 1 :

 1 Figure 1: Relationships studied in the didactical tetrahedron (adapted from Rezat & Strässer, 2012, p. 241)

A log was an online survey that contained between four and seven questions about the use of the textbook during the past two weeks. We used thematic analysis to group log responses regarding student use of the textbooks.

Students did not use the open-source feature, and infrequently used computational cells.

Acknowledgment

Funding has been provided by the National Science Foundation (IUSE 1624634). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. We thank David Farmer, the American Institute of Mathematics, and Diana Curtis for their research support.


Theorizing coordination and the role of course coordinators

Chris Rasmussen 1 , Jessica E. Hagman 2 and Naneh Apkarian 3 1 San Diego State University, USA; crasmussen@sdsu.edu 2 Colorado State University, USA; jess.ellis@colostate.edu 3 Western Michigan University, USA; naneh.apkarian@gmail.com Keywords: Coordination, calculus, community of practice.

In the United States (US) improvement efforts at the school level have a long history of bringing teachers together to plan lessons, discuss issues of student learning, create assessments, etc. For example, lesson study has been implemented across American schools as one way to create community among teachers and improve teaching and learning (e.g., [START_REF] Lewis | Facilitating curriculum reforms through lesson study[END_REF]. Professional learning communities are another way of building community among teachers to improve instruction. At the university level, coordination of high enrolment introductory courses provides a similar opportunity for bringing faculty at the same institution together to improve student success. Coordination systems are particularly popular as a response to the large numbers of students enrolled in calculus courses and the wide range of instructors teaching them. Robust coordination systems, led by one or more coordinators (who are typically ladder rank faculty) consist of two major elements: uniform course elements (e.g., common text; exams) and regular instructor meetings. Regular meetings convened by course coordinators are important because they provide opportunities to bring faculty together to discuss issues of learning, assessment, pacing, etc. Robust coordination systems can, therefore, have the effect of turning calculus instruction into a joint enterprise, engendering a community of practice [START_REF] Rasmussen | Calculus coordination at PhD-granting universities: More than just using the same syllabus, textbook, and final exam[END_REF].

While coordination systems are widely used in the US, such systems are under-theorized. In this report, we draw on case study data from five universities to begin theorizing about coordination systems more broadly and the role of the coordinator in particular. More specifically, we address the following research goal: How might we characterize coordination systems and the role of coordinators in these systems? A better theoretical understanding of these phenomena will contribute to pragmatic efforts aimed at improving the teaching and learning of high enrolment, coordinated, undergraduate mathematics courses. The 2-3 day case study site visits we conducted were part of a larger national study of successful calculus programs in the US (Bressoud, Mesa, & Rasmussen, 2015). We employed thematic analysis [START_REF] Braun | Using thematic analysis in psychology[END_REF] of the data corpus, which included interviews with faculty, administrators, classroom observations, and student focus groups.

Our theoretical framing of coordination systems and coordinators draws on work in behavioral economics, thus bringing a novel point of view to university mathematics instructional systems. In particular, we draw on the work of [START_REF] Thaler | Nudge: Improving decisions about health, wealth, and happiness[END_REF], who offer an insightful perspective on how libertarian views of free choice can be profitably combined with paternalistic structuring of choices for others in the marketplace. A central construct in their thesis of libertarian-paternalism is that of choice architect. A choice architect is someone who is responsible for organizing the context in which people make decisions. For example, the person responsible for arranging the way in which school cafeteria food is displayed has the potential to influence the food choices that kids make. What should be placed at eye level, cookies or green beans? The power of a choice architect lies in focusing peoples' attention in a certain direction, and "nudging" them to make particular choices, rather than making the choice for them. Similarly, we argue, this is what effective course coordinators can do.

In our analysis of the five case studies, we came to see the coordination systems more broadly as an example of coordinated independence. The notion of coordinated independence, much like that of libertarian-paternalism, embraces the oxymoron that individuals can be part of a system that directs their choices, but maintains a certain level of autonomy. Our retrospective search of the literature for use of the term coordinated independence revealed the original coining of this phrase to describe the premier jazz drummer "Cozy" Cole in the late 30's. Cozy Cole was renowned for the way in which his hands and feet acted independently yet produced a harmonious sound that was highly coordinated. In a similar way, we theorize that effective course coordinators allow for instructor independence yet result in a harmonious calculus instructional system.

Drilling down into the particulars of what the course coordinator does to promote this culture of coordinated independence, we posit that the coordinator functions much like a choice architect as described by [START_REF] Thaler | Nudge: Improving decisions about health, wealth, and happiness[END_REF]. In our analysis we identified the following overlaps in the roles of a choice architect in the market place and the various roles of the coordinator in an educational setting: makes life easier by setting default options; provides feedback to users; makes mappings easy to understand; and provides information about what others are doing. For example, Thaler and Sunstein describe how a computer manufacturer functions as a choice architect by making life easier for users by pre-setting screen-saver default settings. A user can change the default options but doing so takes time and requires taking some action. We found that the calculus coordinator similarly sets a number of "default" options for instructors, including homework problems, exams, class activities, syllabus, and rubrics for exam grading. In our interviews with instructors we learned that the vast majority of instructors very much appreciated the coordinator taking care of these decisions, while still allowing for pedagogical autonomy and some choice within these default setting. Our poster presentation gives illustrative examples of this and other roles of coordinator as choice architect.


Anna-Katharina Roos
email: anna-katharina.roos@mathematik.uni-wuerzburg.de

Gerhard Götz
email: gerhard.goetz@mosbach.dhbw.de

Hans-Georg Weigand
email: weigand@mathematik.uni-wuerzburg.de

Jan Franz Wörler
email: woerler@mathematik.uni-wuerzburg.de

Jan Franz

Wörler Optes+ -

OPTES+ -A Mathematical Bridging Course for Engineers

Keywords: Bridging course, engineers, competencies

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Many students perceive the transition from school to university mathematics as problematic. Existing studies not only discuss difficulties broadly but promise that so-called bridging courses can help to reduce the gap between school and higher education. Also for students studying science, technology, or engineering (STE), mathematics is an important prerequisite for successful studies. For them, mathematics is mainly used as a tool for other subjects. We focus on these STE-students, who have a high heterogeneity with regard to their mathematical knowledge before entering university. Taking this phenomenon into account, creating a bridging course in mathematics especially for this group of students is important in order to build a common level of mathematical knowledge, which professors can take for granted and rely upon when students enter university.

The OPTES+-projectoptimizing the self-study phase

OPTES+ is a project funded by the BMBF (Ministry of Education and Research) that includes several universities in Germany working together to reduce the drop-out rate in STE-related subjects. The heart of the project is an online mathematics course, similar to a traditional mathematics book, but enriched with additional interactive elements. The whole course is implemented in the open source learning management system ILIAS. All the material offered by OPTES+ will be accessible for universities and other educational institutions in the future. The contents of the course are based on the experience of the involved lecturers and the cosh-catalogue (Cosh group, 2014), a catalogue written by representatives of schools and universities to list the minimum requirements in mathematics that students should have when starting their studies in economy or a STE-related subject at university. Moreover, as engineering per se often requires the application of mathematics, not only mathematical knowledge is important, but also the development of mathematical competencies. Weinert defines mathematical competencies as an individual's available or learnable cognitive abilities and skills to solve certain exercises, as well as the connected willingness and ability to use the solutions in variable situations in a successful and responsible way [START_REF] Weinert | Vergleichende Leistungsmessung in Schuleneine umstrittene Selbstverständlichkeit[END_REF]. For that reason, apart from the mathematical contents, the bridging course should also concentrate on mathematical competencies, which leads to the following questions.

Research Questions

(Q1) Which model can be used to describe the learners' mathematical competencies and simultaneously provide a basis for differentiation within the course? (Q2) How can the chosen competency model be implemented into the online course? (Q3) How could, after the construction and implementation of the model, an evaluation and reflection of it take place as a step of design-based research?

The Ability Matrix

We decided to take the HarmoS-model (Huber, Späni, Schmellentin, & Criblez, 2006; Schweizerische Konferenz der kantonalen Erziehungsdirektoren [EDK], 2011) as foundation for constructing our own modelthe so-called Ability Matrix (AM), as the aspects of action used in the HarmoS-model seem to fit well the competencies the future engineer students should acquire. Figure 1 illustrates the AM.

Outlook

While (Q1) and (Q2) concentrate on theoretical considerations that are answered by the construction and usage of the Ability Matrix, we plan to run a pilot study in January and February 2019 to answer (Q3) and to investigate if the chosen competency model and its implementation are suitable for our setting. To do so, we will compare the measured competencies before and after the course to see if the attendance of the course (including the feedback the students get by the visualization of the Ability Matrix) brings any improvement of the students' competencies. Based on the results of this pilot study we will evaluate the used competency model and reflect on possibilities of adjusting it.

Figure 1 :

 1 Figure 1: The Ability Matrix


Vilma Mesa
email: vmesa@umich.edu

Irene Duranczyk

Laura Watkins
email: laura.watkins@gccaz.edu

Megan Breit-Goodwin

Anne Cawley

April Ström

Randy Nichols

Anna Bright

Patrick Kimani

Fern Van Vliet

Angeliki Mali

North Nicole Lang

Carla Stroud

Judy Sutor

Saba Gerami

Dexter Lim

Nidhi Kohli

The structure of EQIPM, a video coding protocol to assess the quality of community college algebra instruction

Keywords: Algebra, instruction, video coding, community colleges

. We present a revision of EQIPM prompted by results of factor analyses performed with preliminary data, then discuss next steps and research implications.

interactions [START_REF] Bryk | Learning to improve: How America's schools can get better at getting better[END_REF]. As part of a larger project that investigates the connection between the quality of instruction and student learning in community college algebra courses, we have developed an instrument, EQIPM (Evaluating Quality of Instruction in Postsecondary Mathematics), that seeks to characterize instruction. This paper focuses on the instrument and its validation, and to this effect we present the fourth revision prompted after results from exploratory and confirmatory factor analyses and a discussion with our advisory board suggesting several areas that needed refinement.

Theoretical Perspective

We assume that teaching and learning are phenomena that occur among people enacting different roles-those of instructor or student-aided by resources of different types (e.g., classroom environment, technology, knowledge) and constrained by specific institutional requirements (e.g., covering preset mathematical content, having instructional periods of 50 minutes, see [START_REF] Chazan | Research on the teaching of mathematics: A call to theorize the role of society and schooling in mathematics instruction[END_REF][START_REF] Cohen | Resources, instruction, and research[END_REF]. We focus on instruction, one of many activities that can be encompassed within teaching [START_REF] Chazan | Research on the teaching of mathematics: A call to theorize the role of society and schooling in mathematics instruction[END_REF] and define instruction as the interactions that occur between instructors and students with the mathematical content [START_REF] Cohen | Resources, instruction, and research[END_REF]. These interactions are influenced by the environment where they happen and can change over time. Empirical evidence from K-5 classrooms (5-11 year olds) indicates that ambitious instruction [START_REF] Boston | Assessing instructional quality in mathematics[END_REF] is positively correlated with student performance on standardized tests [START_REF] Hill | Effects of teachers' mathematical knowledge for teaching on student achievement[END_REF]. Understanding mathematics instruction requires attention to the disciplinary content and the mathematical knowledge for teaching and learning. Therefore, we assume, first, that quality instruction is illustrated through the experiences of instructors and students while interacting with mathematical content that have a significant impact on what students are ultimately able to demonstrate in terms of knowledge and understanding, and second, that it is possible to identify latent constructs that might account for the observed quality of instruction. We seek to corroborate findings from [START_REF] Hill | Effects of teachers' mathematical knowledge for teaching on student achievement[END_REF] in the community college context.

Instruction is central to EQIPM. EQIPM (see Figure 1) was designed to assess the three main interactions that Cohen and colleagues (2003) used to define instruction: (1) Quality of Instructor-Student interaction, (2) Quality of Instructor-Content Interaction, and (3) Quality of Student-Content Interaction, which are hypothesized by our framework for instruction [START_REF] Cohen | Resources, instruction, and research[END_REF], see Figure 1). An additional aspect, Mathematical Errors and Imprecisions in Content and Language, is also hypothesized to be an indicator of the quality underlying all the interactions (Hill, personal communication, 2017). The Mathematical Errors and Imprecisions in Content and Language code is intended to capture all uncorrected errors or imprecisions that may occur within a segment; including those advanced by students. This code is the only code that attends to errors and imprecisions within the segment. Within each quality construct we propose codes adapted from existing rubrics [START_REF] Hill | Mathematical Quality of Instruction (MQI)[END_REF][START_REF] Litke | The state of the gate: A description of instructional practice in algebra in five urban districts[END_REF] in order to capture specific observable behaviors that can be used as a proxy for the construct. Thus, the Student Mathematical Reasoning and Sense-Making code is a proxy for the quality of student interaction with content as it seeks to describe and qualify instances in which students make their thinking evident. Likewise, the codes Connecting Across Representations and Situating the Mathematics provide evidence for such interaction [START_REF] Leinwand | Principles to action: Ensuring mathematical success for all[END_REF]. In a similar way we chose four codes under the instructor content interaction, Instructors Making Sense of Mathematics a code that seeks to assess the extent to which instructors assist students in making sense of the mathematics they are teaching. This code is related, but different from, Mathematical Explanations, which attends primarily to the quality of the mathematical argumentation and justifications that instructors provide for any particular mathematical process or idea. While sense making might use informal language or everyday contexts, explanations require in addition that definitions or proofs be used in sound mathematical arguments. Supporting Procedural Flexibility was included in order to account for the quality of the teaching of procedures, an important component of the teaching of algebra [START_REF] Litke | The state of the gate: A description of instructional practice in algebra in five urban districts[END_REF][START_REF] Star | Reconceptualizing procedural knowledge[END_REF][START_REF] Star | The nature and development of experts' strategy flexibility for solving equations[END_REF]. The last code in this dimension, Organization in the Presentation, originally from [START_REF] Litke | The state of the gate: A description of instructional practice in algebra in five urban districts[END_REF] instrument, was included because there is evidence that such organization contributes to student performance [START_REF] Cabrera | Developing performance indicators for assessing classroom teaching practices and student learning[END_REF]. Finally, the codes under the third dimension seek to capture the way in which instructors and students negotiate the work in the classroom, the extent to which their contributions are taken up to shape the direction of the lesson (Continuum of Instruction), how instructors respond to and seek to understand student misconceptions (Remediation of Student Errors and Imprecisions), and how the interactions produce a class environment that is conducive to learning (Classroom Environment). These codes directly reveal the quality of the student and instructor interactions. The codes under Features of Segment help characterize the activity in the segment without qualification (e.g., Nature of math: procedures, applications, etc.; Modes of instruction: lecture, group work, etc.; Technology use: graphing calculators, etc.). These codes are not assessed on quality, only on presence and are used to describe the lessons.

Understanding how the instrument works

In the Fall 2017 semester, we video-recorded 131 lessons in intermediate and college algebra classes from six different community colleges in three different states. These lessons ranged in duration between 45 and 150 minutes, and were taught by 40 different instructors (44 different unique courses video-recorded; 4 instructors taught 2 sections of a course). The majority of the lessons covered one of three topics: linear equations/functions, rational equations/functions, or exponential equations/functions. These topics were chosen because they offered opportunities to observe instruction on key mathematical concepts (e.g., transformations of functions, algebra of functions) and to attend to key ways of thinking about equations and functions (e.g., preservation of solutions after transformations, covariational reasoning), which are foundational algebraic ideas that support more advanced mathematical understanding [START_REF] Breidenbach | Development of the process conception of function[END_REF][START_REF] Carlson | Applying covariational reasoning while modeling dynamic events: A framework and a study[END_REF]. The development of EQIPM was iterative process similar to that used by Hill and colleagues (2008) and by [START_REF] Litke | The state of the gate: A description of instructional practice in algebra in five urban districts[END_REF]. Their instruments subdivided video-taped lessons into 7.5-minute segments, after testing the efficacy of 3 to 15minute segments, they found that 7.5-minutes provided coders with a segment long enough to capture mathematical practices and events in a cohesive way when rating all segments within a lesson without compromising quality of coding that is experienced in long segments. In the Spring 2018 semester, our iterative process produced the EQIPM version 3a which was used in the first round of coding (AI@CC Research Group, 2017). Coders were given up to four consecutive 7.5minute segments to reduce bias due to familiarity with the previous work. Coders recorded their ratings and justifications in a spreadsheet developed for this purpose after watching each segment. Each segment was rated on each of the codes on a 1 to 5 scale (see [START_REF] Mali | Identifying sensemaking in algebra instruction at U.S. post-secondary colleges[END_REF] for more information on rating and exemplification of codes). Ten percent of segments were randomly chosen for double-coding. Each pair of coders held calibration meetings to discuss codes with ratings with a discrepancy greater than one point; the pairs reconciled their new rating by consensus.

Following an exploratory factor analysis [EFA] using 169 segments (Mesa, Duranczyk, Bardelli, & AI@CC Research Group, 2018) we identified a three-factor structure that corresponded to our hypothesized dimensions (see Figure 2a). This analysis also identified two problematic codes Language, Organization in the Presentation of the Procedure, and Classroom Environment, see Figure 2b). In addition, in both analyses, some codes did not load into the hypothesized dimensions (e.g., Situating the Mathematics, Connecting Across Representations, Student Mathematical Reasoning and Sense Making). We called a meeting with our advisory board to discuss the conflicting results. Their review of the instrument and the findings helped us identify two issues that required substantial revision of the instrument (EQIPM 4.0, see Appendix for the revised definitions of the codes) to better describe the ratings and de-confound: 1) quality with quantifiable elements and 2) low quality with absence. We discuss these next.

Confounding quality with quantifiable elements

For some of the codes we had assigned low or high ratings depending on either the time span in which the behavior was observed (or not observed) or other visible quantifiable aspects of the observed behavior. For example, for the code Student Mathematical Reasoning and Sense Making we were assigning a high rating to cases in which the work was "sustained" over the duration of the segment; if an instance was observed but it was "not sustained" or "not the focus" of instruction in the segment, the instrument directed coders to lower the rating. In another case, in the Connecting Across Representations code we accounted for number of representations that were visible in the segment in addition to the quality of the connections made; a high rating would be assigned only to lessons in which the instructor used 3 or more representations. Such approach resulted in lower ratings for segments in which instructors made really good connections but only used two representations; these segments were rated as a 3. We believe that this attending to number of representations possibly over quality of the connection might explain why this particular code had low loadings in both factor analyses. In EQIPM 4.0 we have eliminated language across all codes that refers to quantity or duration, and instead focused strictly on quality.

Confounding low quality with absence

The rating system that we chose, a 1 through 5 scale, was adopted following prior work by [START_REF] Hill | Mathematical Knowledge for Teaching and the Mathematical Quality of Instruction: An exploratory study[END_REF] and [START_REF] Litke | The state of the gate: A description of instructional practice in algebra in five urban districts[END_REF]. Our instrument included similar language in describing a rating of 1 which was assigned to two scenarios. For example, a 1 rating for Mathematical Explanations read as follows: "No mathematical explanations provided by instructor or student. OR Explanations do not include mathematical reasoning or justification; instructor or students provide only steps of a procedure." This way of defining a 1 was problematic, because in some codes a rating of 1 meant not present (e.g., Situating the Mathematics), in others (e.g., Classroom Environment) it represented the lowest quality, and yet in others it meant both (e.g., Mathematical Explanations, the example above). Our advisory board pointed out that the ambiguity makes it difficult to interpret what a low score in a scale would represent. In EQIPM 4.0, the revision of the meaning of a rating of 1 also required a decision about the scale for assessing quality. We had followed Litke's practice of using a rating of 3 to be "modal practice," meaning practices that could be expected in many instances or that would not be extraordinary (in terms of both high and low quality). Ratings of 2 and 4 were defined as "in between" levels, less than or better than 3, without being the extremes. Defining the scale in this way was problematic for several codes, especially for those in which 1 could mean not present. In those codes the change from 1 to 2 was not commensurable to a change from 2 to 3, 3 to 4, or 4 to 5 on our rating scale. The recommendation was to use a four-point scale in order to force the coders into differentiating four levels of quality, with 1 being the lowest and 4 being the highest. We assigned a rating of 0 for "not present" which will facilitate their exclusion as needed.

The revised EQIPM 4.0 instrument

The current version of EQIPM is presented in Figure 1. Once the data are recoded we will run a split sample EFA/CFA to understand the revised instrument's factor structure. The preliminary analyses with the previous instrument lend support to the structure underlying the instrument. One issue we face is the high number of segments in which lecture is the main mode of instruction (75%, 688 of 920 segments coded as of this writing had lecture combined with other modes of instruction, e.g., group work, and 487 of those used only lecture). We believe that with the revised instrument we will be able to differentiate interactions that are solely led by the student. If there are enough segments in which students lead the interactions, we will run an analysis to see whether the three-factor structure holds.

As we move forward we will account for (1) the multi-level structure of our data, specifically, segments within lessons, lessons within instructors, and instructors within colleges, which will address the possible non-independence of some of these codes, (2) lesson duration (e.g., in some lessons, student work individually at the end) and (3) coder bias. We anticipate that the structure will be stable in these cases as well and that as lecture diminishes as consequence of exposure to professional development, we may see that data will fit with three distinct factors as student interaction with content increases.

Figure 1 :

 1 Figure 1: Dimensions and codes for the EQIPM 4.0 instrument

Figure 2 :

 2 Figure 2: (a) Loadings for the EFA 3-factor solution; (b) Loadings for the CFA 2-factor solution

Evalua t ing Quality of Ins t ruc t ion in Pos t s e c o ndary Ma t he m a t ic s F e atu r es of S e gment Qu a l. S t udent-C o ntent Interaction Qu a l. Ins t ructor-C o ntent Interaction Qu a l. Instructor-S t udent Interaction

	Nature of	Mathematical Reasoning and	Making Sense of	Instructor-Student
	Mathematics in Segment	Sense-Making (S)	Mathematics (I)	Continuum of Instruction
		Connecting across	Connecting across	Classroom Environment
	Modes of	Representations (S)	Representations (I)
	Instruction	Situating the Mathematics (S)	Situating the Mathematics (I)	Inquiry/Exploration
	Technology Use		Supporting Procedural Flexibility	Remediation of Student Errors and Difficulties
			Organization in the
			Presentation
			Mathematical Explanations
		Mathematical Errors and Imprecisions in Content and Language

E Q IPM:

  (Remediation of Student Errors and Imprecisions and Mathematical Errors and Imprecisions in Content and Language). A subsequent confirmatory factor analysis [CFA] with 306 coded segments, suggested instead a two-factor structure (Instructor-Content interaction and Student-Instructor interaction) and four problematic codes (one had poor loading, Connecting Across Representations, and three could not be used in the analysis, Mathematical Errors and Imprecisions in Content and

Acknowledgment

Funding for this work was provided by the National Science Foundation award EHR #1561436. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. We thank Emanuele Bardelli and Inah Ko for analytical assistance and Saba Gerami for comments to earlier versions of the proposal. We thank the faculty who participated in the project. Without them this work would not have been possible.

Appendix: Revised Definition of the Codes

Code

Revised Definition: The following codes assess… Student Mathematical Reasoning and Sense-Making: student utterances that showcase reasoning and sense-making about mathematical ideas (e.g. drawing logical conclusions, providing conjectures, counter-claims, reasoning and engaging cognitively in problem solving) Connecting Across the connections that instructors or students make between and across representations

Representations:

of the same mathematical problems, ideas, and concepts Situating the Mathematics: how instructors or students make connections to other aspects of the algebra curriculum, related topics, or the broader domain of mathematics, situating and motivating the current area under study within a broader context.

Instructors Making Sense of Mathematics:

how instructors attend to specific aspects of mathematics (e.g., solution, symbols, conditions) to clarify their nature Mathematical Explanations:

how mathematical reasons and justification for why something is done are provided Supporting Procedural Flexibility:

how instructors identify what procedure can be applied, and when and where to apply them, or makes connections across procedures Organization in the Presentation: how complete, detailed, and organized the instructor's or students' presentation (either verbal or written) of content is when outlining or describing the mathematics at hand.

Instructor-Student Continuum of Instruction:

how the investment that students make in their own learning and development of mathematical understanding by expressing thoughtful ideas that advance their learning. mathematically incorrect or problematic use of mathematical ideas, language, or notation.

Classroom


Libuše Samková
email: lsamkova@pf.jcu.cz

Using Concept Cartoons in future primary school teacher training: the case of problem posing and open approach

Keywords: Concept Cartoons, elementary school teachers, mathematics education, open approach to mathematics, preservice teacher education

The contribution focuses on an educational tool called Concept Cartoons from the perspective of problem posing activities of future primary school teachers within the framework of the open approach to mathematics. It introduces Concept Cartoons, the open approach to mathematics, and a small qualitative empirical study conducted with future primary school teachers. The aim of the presented study is to observe how open might be problems posed by future primary school teachers in the Concept Cartoon form. The results of the study confirm that Concept Cartoons can be successfully employed in problem posing activities and that they have a potential from the perspective of the open approach to mathematics.

Introduction

The study presented here reports about an educational tool called Concept Cartoons, and about the way how the tool can be employed in future primary school teacher training. In particular, it will focus on the possibility to use Concept Cartoons during problem posing activities and observe the posed Concept Cartoons from the perspective of the open approach to mathematics. The study follows up the contributions of me and my colleagues from previous CERME and ERME conferences, as it discusses mathematics problems of a given structure and their employment in future teacher training [START_REF] Tichá | Word problems of a given structure in the perspective of teacher training[END_REF], problem posing of future teachers [START_REF] Tichá | Problem posing and development of pedagogical content knowledge in pre-service teacher training[END_REF], and future teachers facing or composing potential pupils' misconceptions in the Concept Cartoon form [START_REF] Samková | Using Concept Cartoons to investigate future teachers' knowledge[END_REF][START_REF] Samková | Using Concept Cartoons to investigate future teachers' knowledge: New findings and results[END_REF][START_REF] Hošpesová | Concept Cartoons created by prospective primary school teachers[END_REF]. The issues of teachers posing problems and teachers facing potential pupils' misconceptions are the topics that have been broadly discussed at CERME and ERME conferences in TWGs related to teacher professional development, e.g. by [START_REF] Malaspina | Development of teachers' mathematical and didactic competencies by means of problem posing[END_REF], [START_REF] Milinković | Faults in preservice teachers problem posing: What do they tell us?[END_REF], [START_REF] Kuntze | Assessing pre-service teachers' competence of analyzing learning support situations through a multi-format test instrument comprising of video, comic, and text vignettes[END_REF]. The topic of Concept Cartoons is close to the concept of polyvalent math tasks presented at CERME by [START_REF] Hellmig | Effective `blended' professional development for teachers of mathematics: design and evaluation of the "UPOLA"-program[END_REF].

In the following text, I will introduce the idea of Concept Cartoons and my experience with them, and the open approach to mathematics. Then I will describe an empirical qualitative study addressing the following research question: "How open are problems posed by future primary school teachers in the Concept Cartoon form?"

Theoretical background

Concept Cartoons

An educational tool called Concept Cartoons appeared in Great Britain more than 20 years ago [START_REF] Keogh | Learning in science: Another way in[END_REF], in order to support discussion and involvement of primary school pupils during science lessons. Later the tool also expanded to other school subjects and school levels.

Concept Cartoon is a picture showing a situation familiar to pupils from school or everyday reality, and several children discussing the situation in a bubble-dialog. Each bubble presents an alternative viewpoint on the pictured situation. The alternatives might be correct as well as incorrect; the correctness also might be unclear or conditioned by circumstances not mentioned in the picture. Sometimes the picture includes a blank bubble with just a question mark, to indicate that there might be other alternatives not presented in the picture yet. For a sample see Fig. 1. When working with Concept Cartoons in the classroom, pupils are asked to decide which children in the picture are right and which are wrong, and justify the decision. In that setting, it turned up that the lack of agreement amongst the pictured children encourages the pupils to join the discourse and defend their opinions [START_REF] Naylor | Argumentation and primary science[END_REF].

Besides the original classroom use, we started to consider each Concept Cartoon as an educational model of a contingent situation [START_REF] Samková | Using Concept Cartoons to investigate future teachers' knowledge[END_REF], and more widely as a representation of practice oriented on diagnostic purposes (Samková, 2018b). In that sense, Concept Cartoons do not cover all aspects of school practice but might be regarded as a result of a decomposition of practice according to [START_REF] Grossman | Teaching practice: A cross-professional perspective[END_REF]. Following this line, I created my own set of Concept Cartoons and accompanied them by a list of indicative questions focusing more deeply on conceptions and misconceptions probably hidden behind individual bubbles (Samková, 2018b). We included these Concept Cartoons as a diagnostic tool in future primary school teacher training, and conducted several studies: on aspects of pedagogical content knowledge [START_REF] Samková | Using Concept Cartoons to investigate future teachers' knowledge[END_REF]Samková, 2018b), reasoning on selected topics [START_REF] Samková | On the way to observe how future primary school teachers reason about fractions[END_REF], or comparison between information on mathematics content knowledge that can be obtained through Concept Cartoons and through word problems in standard written tests (Samková, 2018a). In this paper, I will focus on Concept Cartoons within the framework of open approach to mathematics.

Open approach to mathematics

Open approach to mathematics is a method of mathematics teaching that employs problems called open. These problems have multiple levels of grasping (i.e. their starting situation is open), multiple correct ways of solving (i.e. their process is open), multiple correct answers (i.e. their end products are open) and/or multiple ways to transform the problem into a new one (i.e. ways to develop are open) [START_REF] Nohda | Teaching by open-approach method in Japanese mathematics classroom[END_REF][START_REF] Pehkonen | Use of open-ended problems in mathematics classroom[END_REF]. Such problems allow us to perceive mathematics in all its diversity. For samples see Table 1. Also Concept Cartoons themselves may be considered as open problems, since they present various alternatives (of grasping, solving, solutions) in their bubbles.

There are 4 beds of seedlings in a forest nursery, each of them having 5 rows with 240 seedlings. How many seedlings are there?

starting situation is open (it is not clear whether 240 is the amount for each of the rows, or altogether for all 5 rows);

process is open (for the first case above: we may count the number of rows in all beds and multiply it by 240, or count the number of seedlings in one bed and multiply it by 4); 90-minute tickets in Prague cost 32 crowns; 24-hour tickets cost 110 crowns. How much will Ivan pay for the tickets on a weekend trip?

starting situation is open (we do not know how much or how often will Ivan travel around the city, how long the trip will last: less than 24 hours? more than 48 hours? how much more? ...);

end products are open (there are 13 different solutions, from minimal pay of 0 crowns, to maximal pay of 330 crowns). Open problems that have multiple solutions of different levels of difficulty while every pupil is probably able to find a solution appropriate with his/her actual knowledge are called polyvalent [START_REF] Hellmig | Effective `blended' professional development for teachers of mathematics: design and evaluation of the "UPOLA"-program[END_REF]. The Concept Cartoon from Fig. 1 might be considered as polyvalent, since it has three different correct bubbles, and each of the them requires a different level of knowledge to justify its correctness: the easiest is the bottom left bubble where the discounts are in the same order as in reality in the shop; the top right bubble deserves an additional knowledge that changing order of discounts does not change the final price; the bottom middle bubble is the most difficult, it is the only one with some other calculations behind. The incorrect bubble (top left) deserves another type of knowledgethat a combination of discounts cannot be solved by their addition.

Design of the study

Participants

Participants of the research were 29 future primary school teachersfull time students of the 5-year teacher training program at our university. They had already finished all mathematics content courses of the program, and in the time of the study they were attending the course on didactics of mathematics. During the mathematics content courses the participants had several times worked with Concept Cartoons, discussing them in the classroom or responding in written form to various sets of questions related to them, i.e. they got familiar with the format of Concept Cartoons.

Course of the study

In the data collection stage, the participants were asked to create their own Concept Cartoon that could be assigned to primary school pupils during a mathematics lesson. They worked on the task individually, in the form of a compulsory written homework where they introduced the Concept Cartoon, and gave a short explanation of the task and the bubbles.

During data analysis I searched for answers to the research question "How open are problems posed by future primary school teachers in the Concept Cartoon form?". I processed collected data qualitatively, using open coding and constant comparison [START_REF] Miles | Qualitative data analysis: A sourcebook of new methods[END_REF]. During open coding I focused on the composition of the Concept Cartoon, on its mathematical correctness, and on displays of openness in relation to Nohda's definition of open problems (2000). For better arrangement, some of the codes were denoted by plus or minus sign to distinguish between positive and negative aspects (correct or incorrect format, open or non-open parameters of the task, etc).

The following code categories appeared as relevant for my study:

A. the format of the Concept Cartoon (this category included codes proper format, improper format, no bubbles, no alternatives, each bubble solves a slightly different task, independent bubbles, conditional bubbles, etc.); B. grasping of the task in the picture and its levels (codes unique, multiple, unclear, too open to be comprehensible, various parameters, diverse interpretations of the assignment, etc.); C. ways of solving the task in the picture (codes unique, multiple, diverse procedures offered in bubbles, etc.); D. number of solutions and their interpretations (codes unique solution, multiple solutions, unique interpretation, multiple interpretations of unique solution, etc.); E. difficulty and smartness of the posing process (codes detailed preparation needed, smart pre-calculations made, etc.).

Subsequently, I analysed the codes and categories in order to divide the participants into groups according to the level of openness provided in their Concept Cartoons. Since each Concept Cartoon that is properly formatted might be considered as an open task, the participants who were not able to pose their Concept Cartoon in a proper format were assigned to the lowest level group named LL.

The other participants were assigned to groups named L#, where # stands for a number from 1 to 4 denoting in how many code categories of B to E did the individual participant cause positive codes.

Findings Improper vs proper format of the Concept Cartoon (category A)

Three of the participants composed their Concept Cartoon in an improper format; they form the LL group. These participants offered either a picture without bubbles or a picture with bubbles that discussed diverse situations not much related to the picture.

Thirteen of the participants composed the Concept Cartoon in a proper format but with no other openness: they prepared a task with a unique solution, grasped it unambiguously, and did not considered multiple correct ways of solving (see Fig. 2a). These participants form the L0 group.

Openness of the Concept Cartoon with proper format (categories B to E)

The other participants composed a properly formatted Concept Cartoon that can be considered as open: seven of them caused positive codes in one of the categories B to E (they form the L1 group), six of them caused positive codes in two of the categories B to E (they form the L2 group). The distribution of the participants among the code categories with positive codes is shown in Fig. 2b, a detailed description of particular open aspects is given in the following text.

Multiple levels of grasping the task in the picture (category B)

Eight of the participants presented in their bubbles various ways how the pictured task might be grasped. For instance, Francis prepared a task about clocks (Fig. 3), and contrasted various possible ways of grasping the term "most hours". Her Concept Cartoon provokes many questions: "What does it mean most hours?", "Shall we take the biggest number in the notation, or the latest hour?", "Is 10 in the morning the same as 10 in the evening?". Annie prepared a task about 30 candies in a bag, and contrasted relative and absolute meanings of numbers (see Fig. 10 in [START_REF] Samková | On the way to observe how future primary school teachers reason about fractions[END_REF]. Morris prepared a practically based task about part-time job offers, and contrasted three different ways to make money (weekly wage, monthly wage, and daily task wage; see Fig. 8 in [START_REF] Samková | On the way to develop open approach to mathematics in future primary school teachers[END_REF]. Four of the participants posed tasks that allow multiple correct ways of solving; e.g. Viola offered in her bubbles two diverse ways of solving a task about Easter eggs (see Fig. 4). 2016). In each of these Concept Cartoons, the particular solutions of the task are of the same difficulty, based on a common idea, i.e. these Concept Cartoons cannot be considered as polyvalent.

One of the participants (Helen) posed a task with a unique solution but interpreted the solution in multiple ways. In her bubbles she described various properties of the solution.

Difficulty and smartness of the posing process (category E)

Four of the participants presented Concept Cartoons that required detailed preparation of the content of bubbles and smart pre-calculations; all of them belong to L2 group. For instance, Morris in his Concept Cartoon about part-time jobs prepared five bubbles based on diverse wage conceptions but the diverse calculations hidden behind particular bubbles lead to numbers that are almost the same. Since the aim of his task is to compare the jobs and find the most suitable one, such an arrangement makes the task open for wide discussions. Ruth in her task about who is the fastest runner assigned that Rex ran 10 rounds in 3 min 20 sec, Punťa ran 7 rounds in 2 min 27 sec, and Max ran 5 rounds in 1 min 50 sec. One of the misconceptions in her Concept Cartoon originated in an erroneous conversion of the times into decimal notation (e.g. 3 min 20 sec converted to 3.20 min). She prepared the numbers in the assignment so smartly that with that erroneous conversion the order of the runners differs from the order in the correct solution.

Conclusions

The presented study hopefully enriched the puzzle on "How can we meaningfully employ Concept Cartoons in future teacher education" by another piece of knowledge, by studying the tool during problem posing activities within the framework of the open approach to mathematics and by looking for answers to the research question "How open are problems posed by future primary school teachers in the Concept Cartoon form?".

The results confirmed that Concept Cartoons can be successfully employed during problem posing activities, and that they have a potential also from the perspective of the open approach to mathematics. Although for the participated future primary school teachers it was the very first opportunity to create their own Concept Cartoons, almost all of them were able to compose a properly formatted Concept Cartoon, and almost half of them were able to compose a Concept Cartoon that positively displayed openness in one or two of the following aspects: multiple levels of grasping the pictured task, multiple correct ways of solving the task, multiple number of solutions and their interpretations. Some of the participants also proved their ability to thoroughly think about the task they pose, and presented Concept Cartoons that required detailed preparation and smart pre-calculations.

The described results of problem posing activities of future primary school teachers also illustrate how Concept Cartoons may help to elaborate a wide range of pedagogical content knowledge: authors of the Concept Cartoons had to pose an appropriate mathematical problem, present at least one of its correct solutions and several possible pupil misconceptions, and combine them into one complex bubble-dialog task.

Such results are in accordance with the recent research showing the importance of implementation of mathematical problem posing activities into future teacher education (Singer, Ellerton, & Cai, 2015).

Figure 1 :

 1 Figure 1: An original Concept Cartoon with three correct bubbles, one incorrect bubble, and a blank bubble; taken from (Dabell, Keogh, & Naylor, 2008: 4.11)

Figure 2 :

 2 Figure 2: The Concept Cartoon posed by Beata, translated from Czech (left); numbers of participants displaying particular open aspects in their Concept Cartoon (right)

Figure 3 :

 3 Figure 3: The Concept Cartoon posed by Francis, translated from Czech

Figure 4 :

 4 Figure 4: The Concept Cartoon posed by Viola, translated from Czech, graphically adjusted

Table 1 : Samples of open problems (left), with their open-approach characteristics (right)

 1


Marianne Schäfer
email: mschaefer@mathematik.uni-kassel.de

Rita Borromeo Ferri

Generation and generation tasks in mathematics didactics

Keywords: Generation effect, mathematics didactics, desirable difficulties, mathematical-didactic knowledge, teacher education

As part of the PRONET project of the University of Kassel 1 , the further development of teacher training through the "implementation of a coherent professionalisation concept" is being investigated. The aim of the subproject is to combine mathematical-didactic knowledge with pedagogically relevant and cognitive-psychologically anchored concepts, including the "desirable difficulties". The generation effect is one of the four best-known "desirable difficulties" and aims to ensure that knowledge is not merely predetermined but is generated to a certain extent from one's own knowledge. In order to achieve this, we have developed for the first time generation tasks on mathematical-didactic topics of a basic lecture for mathematics teachers at their study beginning. The generation tasks and first results are part of this paper and are complemented by the basic theory of "desirable complications".

Theoretical Background

The focus of this contribution lies on the generation and is reflected approach by the university students. For this reason Generation and generation tasks in mathematics didactics training of future teachers are illustrated more detailed.

Desirable difficulties of learning

One possible encouragement of sustainable knowledge building is the concept of 'desirable difficulties' [START_REF] Bjork | Memory and metamemory considerations in the training of human beings[END_REF][START_REF] Bjork | Making things harder on yourself, but in a good way: Creating desriable difficulties to enhance learning[END_REF][START_REF] Lipowsky | Schulpädagogik Heute[END_REF]. The term "desirable difficulties" is a cognitive psychological concept and a didactic procedure that makes learning more difficult in the short term, but sustainable with the goal to a better retention of the learning content (ibid.). Desirable difficulties include such popular techniques as the testing effect, the distributed learning, the generating effect, and the interleaving practice [START_REF] Bjork | Making things harder on yourself, but in a good way: Creating desriable difficulties to enhance learning[END_REF].

Generation effect

The generating effect appears, when the information is generated from own knowledge (e.g. Salmecka & Graf, 1978). The focus of our research is on the generation effect. This effect has been recognized, reproduced and characterized as extremely stable within controlled learning environments and with simple learning content (e.g. word pairs) (Slamecka & Graf, 1978;[START_REF] Bertsch | The generation effect: A meta-analytic review[END_REF]. [START_REF] Chen | The Worked Example Effect, the Genera-tion Effect, and Element Interactivity[END_REF] have shown that the generation effect is more distinctive if very less is dictated by the task itself or by solution examples and teacher interference. They also found that this increase and stabilisation of the generation effect depends on the type of task. While learners benefited more from the generation in simple tasks, the learning outcomes in complex tasks through guided learning were higher (ibid.). In the field of mathematics, the generation effect is only within the framework of multiplication tasks (cf. [START_REF] Mcnamara | A Procedural Explanation of the Gen-eration Effect for Simple and Difficult Multiplication Problems and An-swers[END_REF] but is not researched with reference to mathematical-didactic knowledge. It becomes clear from the previous studies that generation tasks can extend beyond word pair formation, but that the complexity should not be increased immeasurably, otherwise there is a danger that there will be no more lasting learning effect.

Generation and generation tasks in mathematics didactics training of future teachers

Generation in mathematical didactics means therefore to stimulate the cognitive thought processes of the students and to have new information generated from themselves by means of suitable generation tasks. The self-dependent generation of knowledge takes place within the framework of mathematics didactic generation, for example by linking to previous experiences, for example by generating examples. Another possibility is the creation of individual memory aids in which students use their personal thinking structure to memorize new knowledge. In particular, it should be noted that the generation tasks developed include the basic, recurring contents of mathematics didactics on the one hand and are not too complex in terms of the task and the focus of content on the other. In addition, it must be taken into account that the prospective teachers can build on their own knowledge, although the "new knowledge" cannot be presented to them in advance, otherwise there is no generation effect. Therefore, the above-mentioned lecture contents were chosen which are simple enough to dispense with illustrated examples and complex enough to serve as a basis for further study and structural (mathematical-didactic) expertise [START_REF] Borromeo Ferri | Aufbau von nachhaltig gelerntem mathematikdidaktischen Professionswissen im Studium durch die Integration von Generierungsaufgaben[END_REF].

Research questions

On the basis of the theoretical background to the generation effect, it was shown that the specific generation can be regarded as a research gap in mathematics didactics. It is to be examined whether and how the generation effect is shown by the use of generation tasks with a mathematical-didactic focus and how effectively the knowledge thus acquired is consolidated among the students. An allencompassing answer to this main question is not yet possible at this point in time, as the statistical data could not yet be fully evaluated. Therefore, in this paper we limit ourselves to demonstrating what marked generation tasks and highlight the following two research questions:

1. Are there any differences between the generation and reproduction cohort regarding the perception of the learning process? 2. Are there any differences between the generation and reproduction cohort regarding the perception of the tasks?

Methodology

The study is conceived as a prospective and hypothesis-generating comparison of two cohorts attending the mathematics didactic basic lecture at the University of Kassel, Germany. The cohort of the winter semester 2016/2017 represents the experimental group and works on the generation tasks. The control group is represented by the cohort of the winter semester 2017/2018 and works on the reproduction tasks. The teacher training students attend the lecture for the first time and are on average in the 1st to 3rd semester. The semester differences are due to the fact that the lecture is attended by various teacher training courses (secondary school, grammar school and vocational school). In order to ensure the comparability of the two cohorts, both the lecture contents and their sequence are held identically by the same professor in both years. Likewise, all questionnaires, feedback forms, the subject areas of the tasks used and the survey times are identical. The study design is graphically illustrated below (Fig. 1).

Figure 1: study design

Due to the fact that this paper has put the emphasis on generation tasks, an insight into what generation tasks are will now be given. The aim of these tasks, as explained above, is to move students of teaching professions to generate their own knowledge out of themselves in a mathematics didactic basic lecture. What does it mean to generate knowledge and how should this be done? Producing knowledge out of them means nothing other than that the teacher training students open up new fields of mathematics didactics for them with little input. For example, the general mathematical competences will be addressed within the framework of the lecture. This is followed by a generation task, which calls for a memory aid to be considered in order to memorize these six competencies. The desirable difficulty for the teacher training students now consists in using a useful memory aid for them without knowing the deeper theoretical contents of the competences. The theoretical contents of the mathematical competences are discussed after the generation task. In this approach, the main difference to reproduction tasks also becomes clear, since these are always carried out after a theoretical input and usually allow this to be summarised. An exemplary comparison of generation and reproduction tasks in the field of general mathematical competences is given in Table 1. A total of six grading tasks have been developed and applied for use in the mathematics didactic basic lecture. These deal with basic mathematical-didactic content on the topics (1) mathematical educational standards, (2) general mathematical competences, (3) mathematical competence: mathematical modelling and the modelling cycle, (4) mathematical competences: solving problems mathematically, (5) mathematical thinking styles and (6) psychology of mathematics. Explain (briefly) your procedure and briefly justify your choice of memory aid.

reproduction task Julia, Stefan, Sarah, Carsten and Nadine take the school bus to school every morning. Since they get on at the first stop, they always manage to sit in the five seats of the last row. One day Julia remarks: "The school year takes about 210 days. Is it possible that we five always sit in the last row in different ways?"

What is the guiding idea behind the task? Which mathematical competence(s) are relevant to answer the question?

In order to answer the research questions underlying this paper, we have developed, used and evaluated a feedback questionnaire that students of teaching professions can use with regard to their perception of the learning process and their perception of the tasks questioned. The perception of the learning process questions whether there has been in-depth information processing or whether students have perceived it as in-depth information. This scale comprises four items, an item example is here: "I have succeeded in developing my own thoughts and ideas on this subject". The perception of the tasks questions to what extent the students come to the conclusion that the respective task was meaningful, helpful, stimulating or similar. This scale comprises five items, an item example for this is "The task used in the pre-presentation illustrates the benefit of the learning content". The two perceptions were recorded using a 6-stage, end-point-based scale. The scale ranges from the negative endpoint "Does not apply at all", which is rated one in the evaluation, to the positive endpoint "Fully applies", which is rated six. In order to analyse the differences, a grouping variable was first created that distinguishes between the cohorts and their tasks in generation tasks and non-generation tasks (reproduction tasks). Subsequently, a scale was introduced for each perception and each task, so that in the end 12 scales were created. Six scales for the perception of the learning process and six scales for the perception of the tasks. These 12 scales were checked for their one-dimensionality by means of a main component analysis (PCA), which turned out to be given for all of them. The scales for the perception of learning process show a reliability of . The scales for the perception of the tasks show a reliability of . All items can be described as selective in their respective scales. Due to the sample size of N = 117 student teachers, the verification of the normal distribution has been recorded. For this reason, a T-test for independent samples is performed to investigate the differences and Cohan's is calculated manually to determine the effect strength3 .

Results

1. Are there any differences between the generation and reproduction cohort regarding the perception of the learning process?

When the mean values and the standard deviation are considered for the first time, the first differences can be seen purely descriptively on all task scales with the exception of the scale for the fourth task. These descriptive mean differences are checked by means of a T-test for independent samples. It turned out that two of the six task scales for the perception of the learning are a significant difference in the mean value in favor of the cohort that worked with the generation tasks. This significant mean difference is reflected in the tasks three and five. Task three is the generation task that deals with the mathematical competence of mathematical modeling and the modeling cycle. This is a medium effect ( ). Task five contains the three levels of representation according to Bruner (enactive, iconic, symbolic) (Brunner et al., 1971). This is an almost medium effect (

). Table 2 below gives an overview of all mean values, standard deviations, sample distribution, significance level and effect strength for the respective task scales one to six. 2. Are there any differences between the generation and reproduction cohort regarding the perception of the tasks?

The second question dealt with the question of whether differences in the perception of the tasks of the two cohorts were recognizable. Here, too, the first mean value differences could be seen purely descriptively on all six scales. A T-test for independent groups was also performed to test the significance of the differences. It was found that the performance of the cohorts in two tasks differed significantly. These significant differences were seen in the scales for tasks three and four. Task four dealt thematically with the graduated aid according to [START_REF] Zech | Grundkurs Mathematikdidaktik[END_REF]. It is striking that the significant effect was only in the case of task three in favor of the cohort that worked with the generation tasks. This is a medium effect ( ). In the case of task four, the effect in favor of the not yet generation group is significantly positive, although this is only weak ( ).

Table 3 below gives an overview of all mean values, standard deviations, sample distribution, significance level and effect strength for the respective task scales one to six.

Conclusion

The differences determined in the course of the T-test provide initial indications of a possible effect of the generation tasks, as these are assessed by the students as being much more meaningful. They acknowledge the in-depth information processing and the value of the transfer and assess it positively. As part of a further study, we also analyse the extent to which students have gone through a "generational process". In order to be able to work this out, the generation tasks and interviews are evaluated with regard to the solution process and reflection. The first interview and task analyses showed that the experimental cohorts were able to reproduce the contents of the generation tasks better than the control cohorts were able to reproduce the contents of the reproduction tasks. Furthermore, an initial analysis has shown that no appreciable proportion of self-reflection is discernible in the context of reproductive tasks. In contrast, there are different approaches to reflection in the solutions of the generational tasks. On the one hand, individual solution steps are reflected and corrected if necessary. On the other hand, the individual participants reflect the entire solution process. In particular, the reflexion-in-the-action [START_REF] Korthagen | Schulentwicklung und Lehrerbildung[END_REF] is very pronounced in the generation tasks. One possible reason may be that the generation tasks address deeper cognitive structures, similar to problem-solving tasks, and thus their contents are anchored more sustainably in the memory. So far, however, the results are purely indicative, as only a few interviews and task solutions of the teacher training students have been evaluated so far. For this reason, it is necessary to deepen the analyses in the upcoming research process. For this purpose it is planned to develop a detailed coding manual for the analysis of the generation tasks and interviews on the one hand and to evaluate the questionnaire and the knowledge tests on the other hand. Further evaluations then compare the cohorts and include other factors such as choice of subjects, motivation (MOLAMA), learning strategies, mathematical thinking, study satisfaction or mathematical convictions.

Table 1 : Example Generation and Reproduction Task 2 generation

 1

task The general mathematical competences of the lower secondary level are:  argue mathematically  solve problems mathematically  mathematical modelling  use mathematical representations  to deal with mathematics symbolically/formally/technically  communicate mathematically Create a personal memory aid to memorize general mathematical skills.

Table 2 : table of results for the perception of the learning process

 2

			perception of the learning		level significance of	Effect strength Cohan's
	task
	number		N	MW	SD
	1	GA NGA	73 35	4,03 4,25	0,86 0,81	p=0.204
	2	GA NGA	65 35	4,19 4,25	0,86 0,72	p=0.720
	3	GA NGA	73 35	4,63 4,16	0,68 0,90	p=0.003	0.63
	4	GA NGA	65 35	4,06 4,07	1,05 0,82	p=0.961
	5	GA NGA	65 35	4,40 3,99	0,75 0,98	p=0.023	0.48
	6	GA NGA	66 35	4,10 3,81	0,76 0,82	p=0.086
	GA = generation tasks and NGA = non-generation tasks

Table 3 : table of results for the perception of the tasks

 3

						level of	Effect strength
			perception of the tasks	significance	Cohan's
	task
	number		N	MW	SD
	1	GA NGA	73 35	3,92 4,29	1,08 0,77	p=0.078
	2	GA NGA	65 35	3,91 4,31	1,10 0,94	p=0.071
	3	GA NGA	73 35	4,67 4,25	0,80 0,92	p=0.014	0.52
	4	GA NGA	65 35	3,89 4,30	1,21 0,82	p=0.045	0.38
	5	GA NGA	65 35	4,43 4,11	0,99 0,76	p=0.106
	6	GA NGA	66 35	3,86 3,52	1,06 0,86	p=0.103

GA = generation tasks and NGA = non-generation tasks

The project is financed as part of the PRONET project from the "Quality Initiative for Teacher Education" programme of the Federal Ministry of Education and Research in Germany.

Translation of the German original by the authors.

Cohan's d is calculated using the online calculation tool: https://www.psychometrica.de/effektstaerke.html and to the rehearsal again with http://www.soerenwallrodt.de/index.php .