Emma Howard
email: emma.howard@ucdconnect.ie

Maria Meehan
email: maria.meehan@ucd.ie

Andrew Parnell
email: andrew.parnell@mu.ie

An exploration of the relationship between continuous assessment and resource use in a service mathematics module

Keywords: Continuous assessment, student participation, remediation, undergraduate mathematics, service mathematics

Previous research has shown that students' use of module resources strongly relates to the timing of the module's continuous assessment. In our case study of a large first-year mathematics module for Business students, Maths for Business, we examine this relationship and the resources relied on by students for completing their continuous assessment. In Maths for Business, students have the choice of using live lectures or online videos or a combination of both. We find that students who incorporate lectures into their approach engage consistently throughout each week with module resources, while others adopt a just-in-time approach for each weekly quiz. We also show that the introduction of remediation of quizzes can boost participation with resources, in particular feedback resources.

Introduction

Research shows that student engagement in a module is strongly related to the timing of assessment [START_REF] Rust | The impact of assessment on student learning[END_REF][START_REF] Marriott | The use of on-line summative assessment in an undergraduate financial accounting course[END_REF][START_REF] Holmes | Engaging with assessment: Increasing student engagement through continuous assessment[END_REF]. In this study, we focus on this relationship in a service mathematics module, showing that through module design which includes frequent continuous assessment, students can be encouraged to consistently access resources both prior to, and post assessment (through feedback). While this relationship seems evident, we analyse electronic data to demonstrate it. To this effect, we analyse resource usage in a large first-year undergraduate module entitled Maths for Business (based on the 2015/16 student cohort). Continuous assessment is an integral component of Maths for Business, with 40% of the final module mark designated to a student's best eight (out of ten) weekly quizzes. We investigate which resources students access when preparing for the assessment; and when specific resources (lectures, online videos, worksheets, Maths Support Centre and quiz solutions) are accessed; and, whether the approach taken relates to a surface or a deep learning approach to learning [START_REF] Rust | The impact of assessment on student learning[END_REF].

While some research focuses on resource usage in preparation for continuous assessment [START_REF] Holmes | Engaging with assessment: Increasing student engagement through continuous assessment[END_REF], we also wish to investigate the relationship specifically with feedback resources. In 2016/17, we introduced a 'remediation system' whereby students who did not receive the full five marks on their weekly quiz, had the opportunity to resubmit their quiz one week later with an explanation of their error(s) and corrections for one additional mark. We examine how the introduction of the remediation system increased students use of feedback resources. Subsequently our research questions are:

1. In Maths for Business, what is the relationship between the timing of continuous assessment and module resource usage? 2. In Maths for Business, what resources do students access when preparing for their continuous assessment? 3. Is there a relationship between students' choice of resource and their learning approach? 4. Through incentivising remediation, can students be encouraged to access module resources?

Unlike our prior work on Maths for Business [START_REF] Howard | Live lectures or online videos: Students' resource choices in a first-year university mathematics module[END_REF][START_REF] Howard | Quantifying participation in, and the effectiveness of, remediating assessment in a university mathematics module[END_REF], this study focuses on examining resource usage in the context of the timing of continuous assessment, rather than the reason behind students' choice of module content resources [START_REF] Howard | Live lectures or online videos: Students' resource choices in a first-year university mathematics module[END_REF], or, whether the remediation system benefitted students based on their end-ofsemester examination [START_REF] Howard | Quantifying participation in, and the effectiveness of, remediating assessment in a university mathematics module[END_REF].

Theoretical Framework and Related Literature

Engagement and participation

In educational literature [START_REF] Fredricks | School engagement: Potential of the concept, state of the evidence[END_REF], student engagement is considered under three sub-themes or constructs: behavioural, cognitive and emotional. Behavioural engagement [START_REF] Fredricks | School engagement: Potential of the concept, state of the evidence[END_REF] encompasses positive student conduct and participation or ontask behaviour for example effort, persistence, concentration, attention, asking questions, and contributing to class discussion. [START_REF] Fredricks | School engagement: Potential of the concept, state of the evidence[END_REF] note there is a difference between active and passive behaviour. In this study, we refer to participation as the interaction students have with resources which is measured by log or attendance data. When we consider students' participation or resource usage, we are considering a narrow, limited definition or application of behavioural engagement.

Continuous assessment and engagement

Holmes (2018) argues for the importance of student engagement owing to its relationship with student satisfaction and the quality of the student experience. Her research aims to improve the student experience by influencing the relationship between resource-use and assessment. [START_REF] Holmes | Engaging with assessment: Increasing student engagement through continuous assessment[END_REF] shows that the introduction of frequent low-stakes assessment leads to a consistent increase in the use of online resources in a university geography module. This particularly benefits students with "weaker" prior knowledge in a module, as research suggests that they benefit more from participation with resources than their peers [START_REF] Nordmann | Turn up, tune in, don't drop out: The relationship between lecture attendance, use of lecture recordings, and achievement at different levels of study[END_REF]. [START_REF] Marriott | The use of on-line summative assessment in an undergraduate financial accounting course[END_REF], and the references within, argue that assessment is the main variable in accounting for how students approach their learning. [START_REF] Rust | The impact of assessment on student learning[END_REF] expands on this relationship, by noting that students will study the content or resources that they believe will be examined. [START_REF] Anastasakis | An activity theory investigation of tool-use in undergraduate mathematics[END_REF] uses activity theory to explore in detail engineering and mathematics undergraduates' resource-use. [START_REF] Anastasakis | An activity theory investigation of tool-use in undergraduate mathematics[END_REF] found that assessment rules were the main regulator of students' use of resources. Other, less significant factors included: students themselves including their learning preferences, degree and year of study; resources' usability; the level, nature and familiarity of the mathematics; and, lecturers and past members of students' community. In addition to encouraging pre-assessment resource usage, continuous assessment can also encourage engagement through the feedback provided for assessment.

Alternative to examining students' specific resource-use, one could consider whether students take a surface or deep learning approach to learning. A surface approach is often associated with rote learning and extrinsic motivation, whereas a deep approach is associated with learning for understanding and intrinsic motivation. [START_REF] Biggs | The revised two factor study process questionnaire: R-SPQ-2F[END_REF] presents a survey to examine these approaches, the 'Revised Two Factor Study Process Questionnaire: R-SPQ-2F'. [START_REF] Trenholm | Investigating undergraduate mathematics learners' cognitive engagement with recorded lecture videos[END_REF] used the R-SPQ-2F to examine learning approaches in two mathematics modules where students had the option of attending lectures or/and viewing recorded lecture videos. Despite the modules being located in different universities, they found in both cases that a decrease in live lecture attendance combined with regular use of the recordings was associated with an increase in surface approaches to learning.

Module Context

Maths for Business is a large first-year undergraduate module with approximately 500 business students enrolled annually. In Maths for Business, students have the choice of completing the module material by attending approximately 32 live lectures or watching 67 online videos or a combination of both. Each short video (average length of 7.5 minutes) was designed and created by the lecturersecond author of the paper. These videos were created using the Explain Everything app (https://explaineverything.com/), and consisted of the lecturer writing on "skeleton" slides while explaining the topic. Students also have access to the Maths Support Centres (the university drop-in centre and a dedicated module drop-in centre). All resources in Maths for Businessworksheets, lectures or online videoslist the specific learning outcomes which they address. The module is examined through an end-of-semester written 2-hour examination and ten weekly quizzes. These contribute 60% and 40% respectively towards a student's final mark. Students best eight (out of ten) quizzes contribute 5% each to the continuous assessment mark of 40%. From week three of the twelve-week teaching semester, all students sit the weekly quizzes on Tuesdays under the supervision of a teaching assistant. Further detail on the module can be found in [START_REF] Howard | Live lectures or online videos: Students' resource choices in a first-year university mathematics module[END_REF] While the content and aims remained the same, the quizzes were changed to ensure students did not copy from the prior year's quizzes.

In Maths for Business in 2016/17, a 'remediation system' was introduced [START_REF] Howard | Quantifying participation in, and the effectiveness of, remediating assessment in a university mathematics module[END_REF]. In the remediation system, any student who did not receive the full five marks on a quiz, had the opportunity to resubmit their quiz with corrections and an explanation of their error(s) for one additional mark. Table 1 provides an example of the remediation time-frame. Both the pdf and video solutions highlighted relevant video resources for the quiz questions. Owing to the fortnightly remediation cycle, students only had the opportunity to remediate the first eight quizzes. Students were encouraged to use the Maths Support Centre for help with their remediation, particularly students who received a low score on their quiz.

Data Collection and Analysis

We use a case study approach as we are not controlling the behavioural approaches of students and it allows for a mixed method approach. The case study approach allows for the examination of the real-life relationship between continuous assessment and resource usage in the context of Maths for Business. In accordance with our permission from the UCD Ethics Committee, for two years of Maths for Business, we collected students' lecture attendance, log data for online resources, Maths Support Centre attendance, Irish post-primary terminal mathematics examination grade (as an indicator of prior mathematics learning) and marks in the module. In total, there were 522 students in 2015/16 and 480 students in 2016/17. In addition, a survey that included the 'Revised Two Factor Study Process Questionnaire: R-SPQ-2F' [START_REF] Biggs | The revised two factor study process questionnaire: R-SPQ-2F[END_REF] was distributed to the 2015/16 cohort at the end of the teaching semester. We received 161 survey responses. For this paper, we are examining the results from the 'R-SPQ-2F' and one open-ended question from the survey, "How do you study for the weekly quiz, making specific reference to the resources you use (online videos, lecture slides, external sources, friends, worksheets, class notes, Monday drop-in session, Maths Support Centre)?". These survey results are beyond our prior research [START_REF] Howard | Live lectures or online videos: Students' resource choices in a first-year university mathematics module[END_REF].

To examine the time dynamic between continuous assessment and resource usage, we use descriptive statistics in the form of graphs. We do not include resource usage data relating to the revision and examination period of the semester (prior to the end-of-semester examination) as the number of views increases substantially during this time as expected.

To analyse the open-ended survey question on which resources are used in studying, we use thematic analysis [START_REF] Braun | Using thematic analysis in psychology[END_REF].

Results

RQ1: What is the relationship between continuous assessment and module resource usage?

Based on the 2015/16 cohort, analysis of the video views shows that students access videos prior to the weekly Tuesday continuous assessment (on Monday and Tuesday). Similarly, students mainly access the worksheets and Maths Support Centre prior to the weekly quiz. Alternatively, we could investigate the timing of resources based on a finer scale which considers students' main content resources. If we classify students according to their use of live lectures or online videos or combination of both (see [START_REF] Howard | Live lectures or online videos: Students' resource choices in a first-year university mathematics module[END_REF] for details of the classification by clustering analysis), we have four distinct groups: Dual-Users, Lecture-Users, Video-Users and Switchers. Switchers either start the semester predominantly using lectures and progress to predominantly using videos or choose between videos and lectures for content based on the timing of a lecture. Lecture-Users and Video-Users predominantly use lectures and videos respectively, while Dual-Users use both lectures and videos to review the same material. Figure 1 shows the standardised number of video accesses for each cluster over the semester (views are divided by the number of students in each cluster). When examining video accesses in relation to the time in the semester for Dual-Users, Video-Users and Switchers, the distribution is of weekly peaks prior to the continuous assessment albeit with increasing peaks for Switchers as the semester progresses. However, for Lecture-Users, this pattern is not maintained initially. As the semester progresses, the limited number of videos seem to be accessed for assessment purposes. Dual-Users, and to a lesser extent Lecture-Users, use resources throughout the semester whereas Video-Users and Switchers use them on Mondays and Tuesdays. There are limited accesses of resources once their associated quiz has occurred.

RQ2: What resources do students access when studying for their continuous assessment?

Our findings from the first research question suggests a strong relationship between the time when students' access module resources and the date of the continuous assessment. After each quiz, students prepare for the next quiz rather than revising past quizzes. In order to understand the resources students use to study, towards the end of the 2015/16 semester, we asked students how they studied for the weekly quiz. From thematic analysis [START_REF] Braun | Using thematic analysis in psychology[END_REF] of the responses, a central and important theme identified is 'Order of Study'. Students often explained the order of their study approach using the word "then". For example, one response consisted of:

Firstly I go through all the videos and write out notes. Then I attempt the worksheet and highlight all the good examples and things that I need to remember. I then look back over the notes I took and the completed worksheet. I went to the math support centre when I couldn't get the answer to a question.

Generally, students' study pattern for the weekly quiz fall into three stages:

Stage 1: Review the relevant module material -this is completed through videos or/and lectures.

Students may also create their own notes at this stage.

Stage 2: Attempt/complete the worksheet, may refer to videos to help with the worksheet.

Stage 3: If stuck on the worksheet, seek assistance from friends and Maths Support Centre. There are students who deviate from this, for example students who listed a single resource as their response such as worksheets or videos. Male students were more likely to list a single resource, whereas in contrast female students or Dual-Users were more likely to seek support as described in stage 3. Similarly to Figure 1, worksheets and Maths Support Centre were accessed in the days prior to the quiz. We have no data for when friends were consulted.

RQ3: Is there a relationship between student' choice of resource and their learning approach?

Using Biggs et al.'s (2001) 'R-SPQ-2F', for the 2015/16 cohort, we examined students' approaches to learning, namely surface versus deep approach. The results from the five-point Likert scale questions were summed (as outlined in [START_REF] Biggs | The revised two factor study process questionnaire: R-SPQ-2F[END_REF] to extract a numerical measure for students' surface and deep approach to learning. Surprisingly, using an analysis of variance approach, there was no statistical difference between the surface and deep approach of Lecture-Users, Dual-Users, Video-Users and Switchers. Following [START_REF] Trenholm | Investigating undergraduate mathematics learners' cognitive engagement with recorded lecture videos[END_REF] methodology, we also divided users into regular or low video users using the median number of videos accessed. Upon examination, there was no difference in the learning approach of these two groups. However, [START_REF] Trenholm | Investigating undergraduate mathematics learners' cognitive engagement with recorded lecture videos[END_REF] measured the learning approaches of students at two points in the semester whereas we measured the learning approach at one point, the end of the semester. They found an increase in the surface approach of students who regularly used lecture-capture videos combined with lower lecture attendance. The difference in results could possibly be caused by our lack of examining learning approaches at two time points or an innate difference between lecture-capture and short-designed videos for mathematics courses or possibly owing to a difference in survey respondents as our survey respondents were biased towards higher marks in comparison to our overall population.

RQ4: Can students be encouraged to consistently use module resources?

In 2015/16 cohort, resources were accessed prior to the quizzes. In 2016/17, we introduced the remediation system where students could gain an extra mark out of five quiz marks if they resubmitted their quiz within one week of having it returned with an explanation of their error(s).

While the primary motive of this was to encourage students to identify and learn from their errors in a timely manner [START_REF] Howard | Quantifying participation in, and the effectiveness of, remediating assessment in a university mathematics module[END_REF], a potential side-effect was students engaging with quiz or/and feedback material post-quiz i.e. within a week after the quiz. The feedback resources provided included: the graded quizzes; oral tutor feedback; the lecturer providing a video entitled "Most Common Errors"; and, a pdf copy of the quiz solutions that also highlighted the most relevant online videos from the module. In addition, students could seek help from friends and the Maths Support Centre. We found [START_REF] Howard | Quantifying participation in, and the effectiveness of, remediating assessment in a university mathematics module[END_REF] that the main resource students made use of was the pdf solutions, with limited use of the videos and the Maths Support Centre. Upon further examination, with a focus on timing, students accessed the pdf solutions at the end of the remediation cycle i.e. prior to resubmission of their quiz. This just-in-time approach results in a similar distribution or pattern to the video accesses (Figure 1). This is in contrast to the prior year of 2015/16 as seen in Figure 2. For the teaching semester, the number of views for the pdf solutions doubled between the two cohorts (see Figure 2). Figure 2 indicates that students can be encouraged to positively participate with feedback resources.

Discussion and Conclusion

In Maths for Business, we have examined resource-use in relation to its main influencercontinuous assessment. Continuous assessment is linked to students' resource use in two distinct forms: firstly, in preparation for the continuous assessment; and, secondly through students engaging with feedback on their assessment. Resources are accessed prior to the Tuesday quiz except for live lectures which are timetabled. Although the students are using a just-in-time approach, they are still accessing the resources weekly, and in that sense, consistently. The use of resources is divided into three stages or into the following order: module material review; practice questions; and, external help. While not all students will pursue each stage, females are more likely to seek external help from friends or the Maths Support Centres. For the 2016/17 cohort, a positive outcome of the remediation system was that students were encouraged to use feedback resources (pdf solutions). However, similar to preparing for the weekly quiz, a just-in-time approach was adopted for the feedback resources. As a limitation, we note that without qualitative data, we cannot confirm for what percentage of students this was the case. In comparison in 2015/16, there is no obvious distribution to viewing the pdf solutions (after the initial weeks). However, the number views are significantly lower.

In the broader context, continuous assessment can be used to encourage engagement by students and subsequently student satisfaction and the quality of the student experience [START_REF] Holmes | Engaging with assessment: Increasing student engagement through continuous assessment[END_REF]. Arguably, continuous assessment may lead to a 'study-to-the-test' approach. In Ireland, students' learning in the post-primary environment tends to be structured, leading to students struggling with the transition to becoming an independent learner at third level. Maths for Business is an entry-level module and the continuous assessment within allows students to maintain some structure following the transition between education levels. Students' end-of-semester evaluations show an appreciation for the weekly quizzes and the opportunity to have module marks prior to the final examination through low-stakes assessments. This approach may not be applicable for specialist or higher-level mathematics modules. Currently, there is debate surrounding whether recorded lectures or specialised videos have a negative effect on students' learning when used as a replacement for the traditional lecture. In contrast to the current literature, we found no relationship between surface or deep learning approach and students' choice of resource. More research is needed to identify any influencing factors, for example the nature of the video, in this relationship.

Figure 1 :

 1 Figure 1: Maths for Business video accesses for 2015/16. The number of accesses for each cluster has been divided by the number of students in each cluster to allow for comparisons

Figure 2 :

 2 Figure 2: The number of pdf solution views for Maths for Business students by year and day.Duplicate views by a student on a specific day have been removed

  . Between 2015/16 and 2016/17, there were no changes to the learning outcomes of the

	module.
	Date	TA for quiz	feedback by	Oral	completed	Quiz	returned	Quiz	submitted	quiz	Remediated	available	made	resources	Feedback
	27 th of September		-		1		-			-	-
	4 th of October		1		2		1			-	Quiz 1 pdf & video solutions
	11 th of October		2		3		2			1	Quiz 2 pdf & video solutions

Table 1 : Example of the remediation timeframe

 1


Aida Todri-Sanial
email: todri@lirmm.fr

Physical Design and Optimization Methods for TSV-based 3D and Monolithic 3D Integration

This talk aims to unveil how to effectively and optimally design 3D circuits covering aspects from 3D stacking to monolithic integration. As first, an overview of physical design challenges with respect to conventional 2D circuits will be provided, followed by an in-depth look into the power and thermal integrity challenges and solutions. More specifically, design challenges related to power, signal and clock distribution will be covered and also present some of the current solutions. Secondly, an overview of the reliability concerns for 3D circuits and their mitigation techniques will be presented. The talk will conclude with current challenges and future directions for 3D physical design.


Mitsuru Kawazoe
email: kawazoe@las.osakafu-u.ac.jp

How to assess students' learning in mathematics literacy education: An attempt to use students' comments for assessment

Keywords: mathematics education for non-specialists, student attitude, alternative assessment Research objective and questions

It is increasingly important for all students, regardless of educational level, to acquire mathematical skills for use in various contexts, especially in real-world situations. An education that fosters such skills is often referred to as mathematical literacy education, the aim of which often includes improving students' attitudes toward mathematics and deepening their interest in the subject. This raises the question, "How can we assess the success of such courses?" Accordingly, this study focused on students' text feedback, which is a rich resource that represents students' learning and attitudes (cf. Di [START_REF] Martino | Me and maths': Towards a definition of attitude grounded on students' narratives[END_REF], and examined whether students' comments could be useful in assessing their mathematical literacy learning. With this aim, this study analyzed the students' reflection comments collected during one semester. Our research questions are as follows:

(RQ1) What categories are identified in students' reflection comments? (RQ2) Do the students' responses stem from their actual learning? Or do they instead stem from their prior learning in high school or even from their attitudes at the beginning of the course?

Two analyses on students' comments in mathematical literacy education.

A qualitative and quantitative analysis was conducted on students' comments collected during a mandatory one-semester mathematical literacy course for first-year students in social sciences and humanities majors. The course was designed as "learning mathematics through application," but the lecturer emphasized the nature of mathematical knowledge in each lesson. Students' reflections were collected over 14 weeks from an area reserved for comments on each lesson's worksheet. There were 68 first-year students in the class, and a total of 952 comments were collected. We distributed a questionnaire at the beginning of the course; before the students attended the class, we collected data on their courses in high school (science-oriented or humanities-oriented) as well as their attitude toward mathematics (good/bad at math, like/dislike math), using a five-point Likert scale.

The first analysis focused on RQ1, and an open coding method was used to determine a set of categories in order to understand how students described their learning. In the analysis, students' comments were segmented, and we assigned one code to each segment. The analysis was conducted with the help of the following three models: a three-dimensional model for attitudes toward mathematics by Di [START_REF] Martino | Me and maths': Towards a definition of attitude grounded on students' narratives[END_REF]; the three phases of learning by Polya (1981, chapter 14); and the ICE model (Fostaty [START_REF] Young | Assessment & Learning: The ICE approach[END_REF], in which learning outcomes are distinguished in three levels. We found seven important categories that indicate learning progress or attitude improvement. We listed the categories with short explanations, and indicated the percentage of comments containing segments that were assigned with the codes: IA (expressing their Interest in Application, 17.1%), IM (expressing their Interest in Mathematics, 17.2%), MR (expressing their Metacognitive Reflection, 7.9%), EL (expressing their Enjoyment of the Lesson, 7.9%), RQ (Raising Questions that extend their learning, 5.1%), MM (expressing their Motivation in using/learning Math, 8.5%), and RA (expressing their Reduced Awareness of being bad at math, 2.4%). IM and RA were regarded as being related to "vision of mathematics" and "perceived competence" in the model of Di Martino and Zan, respectively. MR was regarded as being related to a phase of "verbalization and concept" in Polya's model and to a level of "connection" in the ICE model.

The second analysis focused on RQ2. Using statistical analysis, this study examined the codes IA, IM, and MR, and investigated whether the frequency of comments corresponding to each category per student depended on students' courses in high school or on their attitude toward mathematics at the beginning of the course. For the second analysis, this study set the following groups based on the students' answers for the aforementioned questionnaire: S+/S-(students who took science-oriented/humanities-oriented courses in high school), G+/G-(students who are good/bad at math), and L+/L-(students who like/dislike mathematics). The numbers of students in each group were 24 (S+), 43 (S-), 13 (G+), 38 (G-), 19 (L+), and 29 (L-). Next, the differences in the frequency of comments corresponding to each category per student between S+ and S-, between G+ and G-, and between L+ and L-were examined. A significant difference was found in the frequency of MR per student between L+ and L-. A Mann-Whitney test indicated that the frequency of comments containing descriptions corresponding to the rate of MR per student was greater for L-(Mdn=1) than for L+ (Mdn=0), U=371.5, p=0.032. No significant difference was found for other codes, and no significant difference was found between the differences in students' courses in high school.

Discussion.

The first analysis showed that students' comments collected from lessons in a mathematical literacy course contained multiple descriptions, including both their attitudinal change toward mathematics and their metacognitive reflections. An interesting result of the second analysis was that less-motivated students showed better results in terms of metacognitive reflection. The second analysis also suggested that students' comments that showed their interest in mathematics or their metacognitive reflection did not stem from their prior learning in high school or their attitude toward mathematics before attending the course. These results seem to indicate that students' comments are potentially useful to assess their mathematical literacy learning. However, more research is needed.

One of future tasks is to analyze the changes in students' comments over time.

Acknowledgment

This work was supported by the JSPS KAKENHI Grant Number JP16H03065.


Christiane Kuklinski
email: kuklinski@idmp.uni-hannover.de

Michael Liebendörfer

Reinhard Hochmuth

Rolf Biehler

Niclas Schaper

Elisa Lankeit

Elena Leis

Mirko Schürmann

Features of innovative lectures that distinguish them from traditional lectures and their evaluation by attending students

Keywords: Mathematics Education, Novel approaches to teaching, Observation, Secondary-Tertiary Transition, Higher Education

To support students at the often-times difficult transition from high school to university mathematics, some universities in Germany have introduced innovative lectures. The WiGeMath project evaluated to what extent these measures fulfill their self-set goals. During the evaluations, specific questionnaire items and lecture observations were applied to investigate which characteristics made these lectures different from traditional ones and in how far students valued these features in achieving the courses' learning objectives. The results show that students in the redesigned lectures value guidance and clarification rather than abstractions and that redesigned lectures include more social interactions and student engagement than traditional lectures.

Background and aim of the paper

Many students face difficulties at the transition from high school to university mathematics [START_REF] Gueudet | Investigating the secondary-tertiary transition[END_REF]. Some universities give support by establishing redesigned lectures, by which we mean lectures that are usually held during the first semester and, unlike traditional lectures, do not focus on university mathematics content. They may, for instance, rather help students who have already failed certain tests to revise the necessary foundations for their further studies or they focus on study techniques like problem solving. Good experiences with lectures or courses that do not focus on the presentation of new theory have been documented in the UK with focus on problembased learning of analysis [START_REF] Alcock | The Warwick analysis project: Practice and theory[END_REF] or problem solving activities [START_REF] Tall | Changing attitudes to university mathematics through problem solving[END_REF]. Meanwhile, traditional lectures are still both widespread and criticized for neither promoting students' active learning nor exploiting their potential in drawing students into higher mathematical thinking [START_REF] Pritchard | Lectures and transition: from bottles to bonfires?[END_REF]. Promoting innovative support is the goal of the WiGeMath project (german for Effects and success conditions of mathematics learning support in the introductory study phase), which is a joint research project of the Universities of Hannover and Paderborn [START_REF] Colberg | Wirkung und Gelingensbedingungen von Unterstützungsmaßnahmen für mathematikbezogenes Lernen in der Studieneingangsphase[END_REF]. It evaluates diverse support measures including redesigned lectures. First, the project developed a taxonomy that serves to categorize features and goals of support measures [START_REF] Liebendörfer | A framework for goal dimensions of mathematics learning support in universities[END_REF][START_REF] Kuklinski | Evaluating Innovative Measures in University Mathematics -The Case of Affective Outcomes in a Lecture focused on Problem-Solving[END_REF]. This taxonomy was then used to lead guided interviews with the lecturers about the courses' goals. Previous research showed that redesigned lectures were successful in meeting at least some of their goals [START_REF] Kuklinski | Evaluating Innovative Measures in University Mathematics -The Case of Affective Outcomes in a Lecture focused on Problem-Solving[END_REF]. Students' mathematical self-concept and self-efficacy did not decline significantly in these lectures unlike in traditional lectures [START_REF] Rach | The Transition from School to University in Mathematics: Which Influence Do School-Related Variables Have[END_REF], and toolbox beliefs decreased. Our research questions are (1) which course features named by lecturers did students find helpful in reaching learning goals and (2) which observable characteristics distinguish these courses from traditional ones.

Method

With regard to research question (1), we will first report on a questionnaire survey and then, for research question (2), on lecture observations. After we had interviewed the lecturers about their concepts of the redesigned lectures, covering their envisioned teaching strategies, the learning goals they wanted students to achieve and which activities they hoped the students would engage in to reach the goals, we developed questionnaires that aimed at evaluating if attending students felt the lecturers succeeded in meeting their own goals. We applied a special concept for lecture course evaluations called the Bielefelder Lernzielorientierte Evaluation (BiLOE, Frank & Kaduk, 2017) which serves to investigate to what extent students feel they have reached certain learning objectives, which activities they found helpful in achieving those they feel confident they achieved and what reasons they see for not reaching the others. The items looked as displayed in Figure 1. This questionnaire was given to the students towards the end of the term. The students were asked only to fill out the part of the helpful activities for those goals they felt they had reached or rather reached. An overview of our samples is given in table Moreover, we conducted lecture observations. We used the WiGeMath taxonomy to develop a guideline to observe the redesigned lectures and created an observation sheet where the observer has to mark different categories as they apply during five-minute-sections that we split the whole session into. The observation sheets for the non-participating observation contained a series of tables like in figure 2. As to the categories of jargon of mathematics and memorizing, these were coded if the lecturer explained the use of a mathematical formula/ character or named a character like or if he asked students to learn a definition, proof scheme or calculation rule by heart, respectively. For each of the evaluated lecture courses, we employed two student assistants to observe the lecture course at three times during the semester, once in November, once in December and once in January (the semester lasts from October to January). The observers studied mathematics but were not regularly taking part in the lecture courses observed. They had been given the observation sheet and a manual including examples and they had been trained using a short video episode of a lecture to make sure they had understood the categories and their handling in the way we had intended them. The students were advised not to communicate so we would get independent results from them which led to a few disagreements in their codings but their overall assessments matched. To have a reference point as to the deviation that the evaluated lecture courses showed from traditional lecture courses, we also had two students evaluate a traditional linear algebra section and a traditional analysis section for first semester students.

Results

We will first look at the results of the BiLOE and then analyze the lecture observations. For the BiLOE, the lecturers named some activities that they expected students to engage in, which did not seem to deviate from traditional lectures. These included attending the lecture sessions, working on homework alone or in groups or revising the lecture notes.

Yet, there were other activities that seemed to be of a more innovative and supportive kind. These activities very much appealed to the students as can be seen from the percentage of students that found them to be rather helpful or very helpful in achieving the lectures' learning objectives.

Although the lecturers were free to name any activities they wanted without having been given categories or guidelines, the activities they named could be grouped into five different classes, namely reflection, testing oneself in a safe place, work with examples, work with prototypes and social interactions, see table 2.

With regard to research question (1), the categories of reflection and work with prototypes seemed to be the most helpful ones in the eyes of the students as more than 60% (for some of the activities a lot more than that) found them rather helpful or very helpful for all given learning objectives. All categories seem to reflect a learning atmosphere that is closer to school life than to university life.

Work with examples

Revising the examples from the lecture Having the lecturer demonstrate how to prove Having the lecturer explain different proof methods in detail 40% -77% 82% -92% 63% -86%

Work with prototypes

Getting to know problems with many different facets Working with very appealing proofs 78% -85% 60% -100%

Social interactions

Asking the lecturer for help when I got stuck Asking the lecturer for feedback even where my solutions were correct Working on content together with others 59% -79% 54% -72% 40% -88% With question (2) in mind, we will now focus on the observation results. Although the predominant media used was limited to the blackboard almost exclusively, just as in traditional mathematics lectures, we found some interesting deviations in the social interactions that took place as well as in the way that students were engaged in problem and task solving processes. In all lectures, we observed various instances where the lecturer asked questions and students answered, indicating that interactions took place regularly. Besides, students also asked questions repeatedly. This suggests that the learning atmosphere in the lectures made students comfortable to openly express their comprehension difficulties. Yet, we did not encounter instances where a student asked a question and another then answered. In consequence, discussions did not take place and neither did typical teaching conversations. Although we observed a few phases of single, partner or group work, these stayed an exception which also indicates that the lecture atmosphere stayed distinct from a typical school classroom. Rather, the format of a lecture remained central and the two forms of interaction that were predominant in all courses were the lecturer talk facing the blackboard and the lecturer talk facing the students. But the fact that other forms of interactions did occur is enough to make the lectures distinct from traditional ones. In fact, phases where the students were actively engaged in problem or task solving processes did not occur in the traditional lectures we observed (Analysis I in Figure 3). In contrast, such phases were observed in all redesigned lectures and there was even one observation where students were active to this effect in 95% of the time (cf. Figure 3). As students rated the category of testing themselves in a safe place as helpful in the BiLOE, it seems like they profited from these periods of student task solving.

Discussion

Methodological discussion

To examine the features of redesigned lecture courses and in how far students find them helpful, we decided to make use of an established evaluation instrument called the BiLOE and to hire students who observed lecture sessions focusing on certain categories based on the WiGeMath taxonomy which we provided them with. To be precise, in applying the BiLOE we used an established evaluation method that we connected to the WiGeMath taxonomy and we developed an observation instrument that is easy to use with small training effort of the observers. Moreover, we did not rely on single observations but employed two raters.

The results of the BiLOE must be interpreted as personal estimates only. Though the students indicated how helpful they found the given activities to be in achieving the learning goals, we cannot say whether they actually did achieve these goals. Other indicators like exam results or homework assignments would have had to be examined to make a qualified statement in this respect. It would also be possible to use a pre-and post-test design to measure the students' understanding of the content. Moreover, the results we got from the BiLOE may be biased as we only surveyed those students that attended the lectures. Had we questioned all registered students of the course, results might have been different.

Concerning the observations, the two student observers did not always agree in their codings. However, we could not calculate inter-rater reliabilities due to unmatched timings by the observers. We recommend parallel timing for both observers in future research and maybe a more elaborate training. Yet, our results show that the use of simple methods can reveal important lecture features.

Discussion of the results

The data show that successful, innovative lectures may benefit from the five elements of reflection, testing oneself in a safe place, work with examples, work with prototypes and social interactions. It is possible to implement such elements in traditional settings and students value these elements.

According to [START_REF] Slomson | What makes a good maths lecture?[END_REF] the format of lectures remains the dominant way of teaching for university mathematics students at the beginning of their studies and he claims that they have hardly changed for 40 years. Our results show, that changes are possible. They contradict the text of [START_REF] Wood | Learning in lectures: multiple representations[END_REF] referring to [START_REF] Gibbs | 53 Interesting Things to do in your Lectures[END_REF] who claim that university lectures are more or less uninterrupted monologues by lecturers where student activity was limited to listening and notetaking. The innovative approach of the projects in WiGeMath is visible as in the traditional lectures we only encountered lecturer talk and no problem solving by the students during the lecture. In fact, [START_REF] Slomson | What makes a good maths lecture?[END_REF] gives four characteristics of lectures and one of them is that there is scarce interaction between the lecturer and his audience and the focus of the lecturer lies in transmitting a set amount of material. Moreover, [START_REF] Wainwright | Observation of Reform Teaching in Undergraduate Level Mathematics and Science Courses[END_REF] also observed traditional mathematics lectures and did not encounter any instances where students collaborated or interacted with other students or the lecturer. Yet, students do seem to be interested in being more engaged during lectures just as we found in our research. In fact, [START_REF] Willcoxson | The impact of academics' learning and teaching preferences on their teaching practices: A pilot study[END_REF] found that students value teacher-student and student-student interactions in the lectures they attend and as [START_REF] Cavanagh | Students' experiences of active engagement through cooperative learning activities in lectures[END_REF] makes clear lectures that engages students in more than just note-taking have positive effects in students' approaches to learning and their long-term understanding.

That students found social interactions and the work with examples helpful in achieving the learning goals goes hand in hand with the findings by [START_REF] Slomson | What makes a good maths lecture?[END_REF] that students prefer lectures with exactly these features and by [START_REF] Anthony | Factors influencing first-year students' success in mathematics[END_REF] that students believe that availability of support helps to lead to students' academic success. Moreover, [START_REF] Wood | Learning in lectures: multiple representations[END_REF] explain the lasting percentage of students attending lectures where they could easily access the material online by the importance that these students must place on human contact in their university learning experience.

In another study that evaluated the success of so-called lectorials which blend traditional lecturing with more engaging activities where students learn cooperatively also found that most of the students valued these activities as they helped them in deeper understanding the content and in staying interested [START_REF] Cavanagh | Students' experiences of active engagement through cooperative learning activities in lectures[END_REF].

While students in our surveys found the work with prototypes and examples helpful in achieving the learning goals, Anthony (2010) [START_REF] Cavanagh | Students' experiences of active engagement through cooperative learning activities in lectures[END_REF] found that students valued an abundance of examples in the lectorials he surveyed as they increased motivation and interest in the tasks.

An important question is, however, what caveats lecturers see in these innovative formats. A major concern might be the pressure they feel to cover all the contents [START_REF] Johnson | It's about time: the relationships between coverage and instructional practices in college calculus[END_REF]. We should then discuss extending the time given for such lectures as well as the sustainability of students' learning.

Implications for research, policy and teaching

Our results indicate that lecturers should consider restructuring their lectures so that students feel more secure to engage with the contents as well as with the lecturer and other students. Where possible, an abundance in examples might help students in understanding and then feeling more confident in the lecture content. So even though the traditional format of the lecture cannot be said to be useless in transmitting mathematical knowledge, our results indicate that students value social interactions and more lively formats which make the content illustrative by means of examples.

The lecture courses we evaluated differ from traditional lecture courses in more respects than the ones we focused on in this paper. In the categorization of the WiGeMath taxonomy, their focus on revisions of content and on study techniques or mathematical working techniques rather than on mathematical content would be differences in the individual and system-related goals of the measures. Yet, the measures also show specificities in their frame conditions and their characteristics. For example, the measures for engineering students in Kassel and Stuttgart are set out with small groups where the students work in an atmosphere that resembles school lessons. The lecture courses in Kassel and Oldenburg, where we focused on the pre-service teachers, indeed address this group predominantly rather than traditional lecture courses which mathematics majors and pre-service teachers attend together. The lecture course in Paderborn uses two tests in the course of the semester to prepare the students for the final exam and the structure of university mathematics exams in general whereas the lecture course in Würzburg does not include any examination at all. In this text, we looked at some further aspects of the lecture courses' learning culture and didactical features but to evaluate in how far students find these other features helpful in their learning process might be an endeavor worth undertaking in the future.

Figure 1 :

 1 Figure 1: Example items of the BiLOE

Figure 2 :

 2 Figure 2: Extract from the observation sheet

Figure 3 :

 3 Figure 3: Engagement of students in problem solving in some of the lectures

Table 1 : Overview of the sample size ( ) and the numbers of female (f) and male (m) participants

 1 1.

	Cohort	Engineering students	Pre-service teachers
	Location	Kassel	Stuttgart	Kassel	Oldenburg	Paderborn Würzburg
	(f/m)	13 (0/13)	55 (11/44) 12 (6/6)	102 (54/48) 45 (18/27) 41 (20/21)
	Method	Paper-pencil		Online	Paper-pencil

Table 2 : Activity categories and percentages of students who found the activity rather helpful or very helpful for all given learning objectives

 2

  found that students did not allocate an important role to the practice with examples for achieving academic success. Yet, Anthony (2010) refers to the selfdirected study of examples rather than the work with examples during lecture time. Conversely,

Acknowledgment

The WiGeMath project is funded by the German Federal Ministry of Education and Research (BMBF), grant identifiers 01PB14015A, 01PB18015A, 01PB14015B and 01PB18015B.


Elisa Lankeit
email: elankeit@math.upb.de

Rolf Biehler
email: biehler@math.upb.de

Students' work with a task about logical relations between various concepts of multidimensional differentiability

Keywords: College mathematics, multidimensional analysis, task design, tutorial groups

The purpose of this study is to identify student difficulties concerning the linkage of various concepts of differentiability in . For that reason, we designed a task where students are encouraged to explore the logical relations between these concepts and analyse how university students work with this task. Based on a detailed a priori analysis of the task building on TDS, we present results of an analysis of students' written work on the task, produced in tutorial group meetings. We identify different types of reasoning and compare difficulty levels of the different relations.

Introduction and literature review

Differentiability and derivatives are essential topics in school and university mathematics and have been studied for many years (e.g., [START_REF] Orton | Students' understanding of differentiation[END_REF][START_REF] Zandieh | The evolution of student understanding of the concept of derivative[END_REF]. However, as [START_REF] Rasmussen | Research on calculus: what do we know and where do we need to go?[END_REF] point out, studies looking beyond first topics of calculus, for instance in multivariable calculus, are rare. From informal discussions with students and lecturers, and from our own teaching experience, we know that multivariable differentiation, an exciting and essential topic covered at the university level, is not an easy topic for many students. The transition from the one-dimensional into the multidimensional case is far from trivial, especially since there are inequivalent concepts of differentiability in . Nevertheless, the literature on the difficulties of students working with these different concepts and relating them to differentiability in the onedimensional case is scarce. [START_REF] Martínez-Planell | On students' understanding of the differential calculus of functions of two variables[END_REF] studied students' understanding of the differential calculus of functions of two variables using APOS theory and found out that the geometric interpretation of functions in two variables and especially of their partial derivatives and total differential are problematic for students. However, they focused on the geometric understanding and did not define the total differential as a linear function and especially did not look at the students' understanding of the relationships of the different concepts. Since we could not find any literature concerning students' understanding of the relationship between the different concepts of differentiability in as such, we had to refer back to the one-dimensional case. Of course, there exists only one concept of differentiability (with different interpretations), and the concept of left-or right-hand differentiability usually plays no important role, whereas directional differentiability becomes an important notion in the multidimensional case. Students' knowledge and difficulties in linking differentiability, continuity and integrability in the onedimensional case could provide helpful insights. [START_REF] Juter | The validity of students' conceptions of differentiability and continuity[END_REF] and [START_REF] Duru | Pre-service mathematics teachers' conceptions about the relationship between continuity and differentiability of a function[END_REF] found that many students believed continuity implied differentiability in the onedimensional case. Additionally, students who found the correct relations between these concepts were often not able to give sufficient justifications. When looking at the relations between differentiability, continuity and integrability, success rates are even lower [START_REF] Sevimli | Understanding students' hierarchical thinking: a view from continuity, differentiability and integrability[END_REF]. In some cases, students used the teaching sequence as an argument for the stated relations. Generalising from this, we expect even more difficulties in the multidimensional case, which may exacerbate, if the knowledge on the one-dimensional case is still fragile. A further challenge for the students is the linkage of the one-dimensional to the multidimensional case. This linkage requires that the students have acquired the interpretation of the derivative as the best linear approximation and not just the interpretation of it as a tangent line, as a local rate of change or as the limit of the difference quotient (for these different interpretations, see [START_REF] Greefrath | Aspects and "Grundvorstellungen" of the Concepts of Derivative and Integral[END_REF].

In this article, we are especially interested in the way students work with different concepts of differentiability in . We designed a task asking to explore the logical relations between these. The task was given to students in their weekly tutorial group meetings in an Analysis II class (which is, from an international perspective, more on the level of upper-division proof-oriented Real Analysis courses in the US than typical lower-division Calculus courses). We will present the task and selected results from our study examining how students in their second or higher semester worked on the designed task, showing success rates for rating and justifying the validity of different relationships and selected ways of argumentation. Thereby we want to improve understanding of students' knowledge and learning of differentiability concepts and hopefully contribute to designing suitable learning environments regarding this subject area in the future.

Theoretical framework

We base our task design and analysis on Brousseau's Theory of Didactical Situations (TDS) [START_REF] Brousseau | Theory of Didactical Situations in Mathematics[END_REF]. A situation describes the circumstances in which students find themselves concerning their milieu (the set of objects on hand, available knowledge and interaction with others). The distinction between didactical and adidactical situations is essential. A situation is of adidactic nature if the teacher does not instruct, but students work autonomously and learn by adapting to the milieu whereas, in a didactical situation, acculturation happens through institutionalisation and devolution. For a more detailed description and a well-presented introduction of TDS, see for example [START_REF] Artigue | Introduction to the Theory of Didactical Situations (TDS)[END_REF].

According to Gravesen, Grønbaek, and Winsløw (2016, p. 10), "exercises are the cores of situations of learning". They developed a framework for task design and analysis, based on TDS and focusing on four specific "potentials" of tasks. The adidactic potential of a task describes the possibilities of a task to be used in an adidactical situation: Which possibilities does a student have to engage with the task and develop new knowledge independently without additional interactions with the teacher? A task's research potential is measured by the possibilities to engage with one or more research like activities formulated by [START_REF] Gravesen | Task Design for Students' Work with Basic Theory in Analysis: the Cases of Multidimensional Differentiability and Curve Integrals[END_REF]. Linkage potential represents the possibilities for students to connect "old" and "new" knowledge. At last, deepening potential means the task's potential to intensify and elaborate the students' knowledge regarding results, notations and methods.

Research questions

We designed a task with adidactical, research, linkage and deepening potential and put the students into an adidactical situation of working on that task. We investigate how the students link the different concepts of differentiability in and want to identify students' difficulties in this subject area. In this paper, we want to answer the following research questions: RQ 1: Which of the logical relations seem to be more difficult for the students? Is the order of difficulty levels the same for the decision of validity and the associated argumentation? RQ 2: What kind of reasoning do the students use when explaining the relationships? RQ 3: How is this reasoning related to the reasoning and identified problems that we anticipated in the a priori analysis? In this paper we focus on students' work on the two questions "Does partial differentiability imply differentiability?" and "Does continuity imply the existence of all directional derivatives?".

Methodology and study design

The method of didactical engineering [START_REF] Artigue | Didactical engineering as a framework for the conception of teaching products[END_REF], evolved from TDS, provides the link between the theoretical framework of TDS and the practical implementation in this study to answer our research questions. It is composed of a preliminary analysis of epistemological, cognitive and didactical dimensions of the mathematical knowledge that is to be learned in the situation. It includes the design of the situation including an a priori analysis with the students' expected behaviour based on the choice of the didactic variables, the experimentation (usually a classroom implementation of the situation) and an a posteriori analysis with evaluation and comparison of a priori and a posteriori analyses. Guided by these steps, we will now provide a short description of our approach.

Step 1: Designing a task that supports our research interests. We used the described method of didactical engineering and conducted a preliminary analysis at first (not shown here). For this, we analysed some broadly established Analysis II books used in German universities (e.g., [START_REF] Heuser | Lehrbuch der Analysis[END_REF] concerning the essence of the concept of multidimensional differentiability. When designing the task, we oriented ourselves by the four "potentials" according to Gravesen et al.'s framework. The task analysed in this article aims at supporting the students in gaining an overview of how the different concepts of differentiability are connected. After working on this task, students should be able to explain how the concepts are connected and to give counterexamples of functions having only some of the properties. The task discussed in this paper was preceded by a prompt to recapitulate which of the connections the lecture had already covered. The discussed task was given to the students after they completed this. In the task, students are asked to examine whether different implications (called "T1" to "T15") are valid or not. The diagram used in the task can be seen in Figure 1 together with the success rates. We created the visual representation of the logical relations (similar to a concept map) in order to support students' reasoning. In particular, the diagram is to support an abstract knowledge network, which students can use as an abstract reasoning tool that some implications are true / cannot be true because other logical relations have already been proven. We were interested in how the students would use this network vs try to proof or refute a particular implication by specific arguments.

Step 2: Conducting an a priori analysis of the task. The a priori analysis of two implications (T10 and T15) can be found in the next section. It was conducted before the implementation of the task.

Step 3: Task milieu and data collection. The situation of working on the task is characterized by the task itself and the milieu in which the students found themselves. The task, which was coordinated with the teacher and teaching assistant, was given to the students of an Analysis II course in their weekly tutorial group meeting. Most of the students worked in pairs or small groups and discussed the task. They also used their lecture notes and could ask the tutor if they needed help. For this particular meeting, particular instructions for the tutors were developed: They were asked to only give strategic hints and mostly just let the students work on their own. The written work the students (n=31) produced during the tutorial group meetings was collected anonymously, scanned and then handed back to the students (without comment or marking).

Step 4: Analysis of data. We analysed the students' written solutions concerning two aspects: the correctness of students' solutions and their ways of argumentation. At first, we used a normative approach by rating the solutions: For every implication arrow, there are two separate tasks: A. deciding whether the implication is true or false, and B. stating a correct justification which could be a counterexample or a proof. Two types of missing values are coded: If there are none of the boxes checked (A) or nothing was written down (B), but subsequent subtasks were worked on, we coded it as a "missing, but attempted", i. e. we believe the lack of written work was not due to time.

If nothing is written down but none of the subsequent tasks was worked on, we coded it as "missing". For the last subtask, we decided to count as attempts every work where one of the boxes for T5 or T15 was checked, or anything was written down for B. The correct decision of A is rated one point if correct and 0 if wrong. B is rated as follows: a (nearly) right justification (i.e. correct with a minor formal error) is rated two points. We give one point for a meaningful attempt for a justification, including argumentations containing consequential errors or correct counterexamples without arguments why these are counterexamples. Wrong or incomprehensible justifications, including wrong counterexamples, are rated zero points. Two total solution rates for A and B were calculated. Here, we defined the success rate per task as the sum of the scores divided by the number of students who attempted the subtask multiplied by the maximum score (1 for A and 2 for B). For analysing the students' ways of argumentation, we used qualitative content analysis [START_REF] Schreier | Qualitative Content Analysis in Practice[END_REF] with categories deducted from ned the a priori analysis complemented with inductively found categories.

A priori analyses

The a priori analysis is the first result and serves as a research tool in the data analysis as well since the results will be compared to our expectations.

A priori analysis of "Does partial differentiability imply differentiability?" (T10)

Most students will probably state correctly that this implication is not true. A hint for this could be that if it were true, it would have probably been covered in the lecture when the other direction was discussed. Moreover, why should the lecturer introduce two different concepts if they turn out to be equivalent? We call these reasons "didactical-contract reasons". They are not mathematical reasons but based on assumptions on what constitutes the didactical contract [START_REF] Brousseau | Theory of Didactical Situations in Mathematics[END_REF]) of the lecture. For proof, there are several options. Since the students are in their second semester or higher and are used to using counterexamples, we expected that they would look for an example of a partially differentiable function that is not differentiable. The function used in the lecture to show that partial differentiability did not imply continuity (Example VII.1.5, see Figure 1) and stated in part b) of this task, which is such an example, should come to mind quickly. Some students will probably only give the counterexample without explaining why it contradicts the implication. Others will explain why the function is not differentiable by calculating the error term φ and showing that does not converge to 0 as . Another possibility is that students will argue on the abstract level of logical relations and might take advantage of the argumentation that is not continuous and therefore cannot be differentiable due to the contraposition of T3. However, it may also occur that students think the implication is true in the first place or that they have difficulties to construct own counterexamples. Based on this analysis, we expected that most students would solve task T10 successfully.

A priori analysis of "Does continuity imply the existence of all directional derivatives?" (T15)

It is important to note that the lecturer defined directional derivatives as one-sided limits in this lecture. Students could have developed a general sense of what we may call "levels or degrees of smoothness" in . These students might quickly come to the conjecture that this implication is probably not true by making the courageous generalisation from that only continuity will not imply any differentiability whatsoever. A strategy may also be to check the meaning of this implication in Students may know that -continuity for n = 1 is the same as one-dimensional continuity. Moreover, the existence of directional derivatives for n = 1 means that the function has a left-and a right-hand derivative (which need not be the same). With this clear picture, they remember that is continuous and has both left-and right-hand derivatives, so the question is whether there is an -example that is continuous but has no left-and right-hand derivatives. Alternatively, students may initially think that is a counterexample but then discover with a closer look that this is not the case. Some students might take this as a hint that the implication is, in fact, true, mainly because the only examples for continuous but non-differentiable functions coming to their minds might be jagged functions. Another example for a non-differentiable but continuous function that the students could know is the square root function in 0, which will provide a valid counterexample for T15, having a "vertical tangent" in . If they do not know this example, it might be tough for them to construct a counterexample themselves. It could also happen that students look for counterexamples not in but in or higher dimensions, but we did not expect that they would be successful with that. Students might also try to use the logical network, but this will not be rewarding.

Description of selected results

Our expectation that the implication T10 would be one of the most straightforward tasks was correct (see Figure 1). A success rate of 100% for A (decision what is correct) is notable and satisfying. We credited 72% of the attempts in T10B with 2 points, 10% got 1 point, 7% 0 points and 10% of the attempts did not contain any written work concerning this task. The success rates in A for the recognition of the correctness of the implication T15 are lower than we expected. We expected low success rates in B for the justification in T15, but the fact that the rates lower than the rate for T2 (where the counterexample is not intuitive at all) is somewhat surprising to us. None of the students received 2 points for their reasoning, 35% of the attempts got 1 point, whereas 26% of the attempts got 0 points and 39% was coded as missing but attempted.

Results of the qualitative analysis of the written reasoning (B) in subtasks T10 and T15

For T10, we found the following categories from the a priori analysis and the data. We assigned two points for stating the function from Example VII.1.5 in the lecture as a counterexample and showing "by hand" that it is not differentiable by calculating the error term (coded "B1") or the argumentation that it cannot be differentiable because it is not continuous ("B2"). Another argumentation assigned two points was working logically with the diagram saying that if T10 were true, together with T3, it would imply T6 which is wrong without stating a counterexample ("A").

A different correct counterexample with appropriate argumentation ("E1") would be assigned two points as well. We assigned one point for stating the function from Example VII.1.5 as a counterexample without an explanation ("B0"). Giving a wrong counterexample ("E0") resulted in 0 points. Typical examples for the two most common argumentations for T10, "A" and "B2", can be found in Figure 2. Concerning T15, we assigned two points for stating a correct counterexample with appropriate reasoning. We expected the square root function as a counterexample, with different possible explanations: stating that it is continuous but not all directional derivatives exist ("W1"), this explanation with specifying the point of interest ("W2") or a calculation that the directional derivatives in don't exist ("W3"). We gave one point for stating the square root function as a counterexample without an explanation ("W0"). The statement that the absolute value function is not a valid counterexample ("Bx") was also rated one point. Additionally, some students gave explanations using the logical structure and wrong implications they thought were true in preceding tasks to argue without a counterexample, namely T12 and the fact that T4 is not true ("Z1") or T13 and the fact that T5 is not true ("Z2"). Stating the absolute value function as a counterexample resulted in 0 points. For both implications, other argumentations were coded as "X". The number of students who used these different kinds of argumentation can be found in Table 1 (T10) andTable 2 (T15). It can be seen that we could not assign some of the categories to any student, e.g., "B1" for T10.

Discussion of results

RQ 1: Most of the students were able to decide correctly for most of the implications whether they were true or not. The calculated success rates give hints concerning the level of difficulty of the implications. We can conclude that deciding whether an implication is true and stating the right reason are not the same, as illustrated by T15. It corroborates findings from studies reviewed in the introduction that students often cannot give correct justifications for relationships between different concepts in the one-dimensional case. However, students in our study were in general more successful than in some of the described studies. That could indicate that working with logical relations between new concepts becomes more accessible for students as they pursue their studies. RQ 2: When they could, nearly all students used the logical structure of the diagram for answering whether a new implication is true, instead of constructing a new proof or a counterexample. In some cases, properties of examples were established by reference to the logical structure instead of verifying the property with calculations with the example itself (for example in T10). When they could, students traced the question back to one-dimensional cases and tried to find counterexamples there (which was possible for many of the implications and was our purpose in the design, utilising the task's linkage potential). The teaching sequence was not given as an argument (as was the case in [START_REF] Sevimli | Understanding students' hierarchical thinking: a view from continuity, differentiability and integrability[END_REF], which could be either because students knew from experience that this is not true, or because these concepts (except continuity) were introduced in a short period after another. RQ 3: The a priori analysis was based mostly on our intuition because we could not find literature concerning didactics of multidimensional calculus, which is something we want to contribute to in the future. There were some lines of argument we did not anticipate, especially concerning incorrect reasoning. We found it surprising that a not negligible part of the students incorrectly considered the absolute value function as a counterexample in T15 although it was proven a week before that all directional derivatives of the absolute value function exist. There were some lines of argument we expected some of the students to use that were not used at all (like showing that the error term in T10 does not have the required property). This fact might, however, also be due to the rather small sample of only 31 students. Our estimates of the subtasks' difficulty were most often appropriate.

For further investigation, including what leads students to their decision, we additionally filmed eight pairs of students while working on the task. The analysis of these is expected to provide more insights concerning the questions where the students got stuck and why. This analysis will be especially interesting because often students who made the wrong decision did not explain why they thought that (e.g., in T15). Our data will be analysed more thoroughly in the next months.

Figure 1 :

 1 Figure 1: The two success rates for each implication. The first number per arrow depicts the success rate for A (correctness of decision); the second number is the B success rate (correctness and quality of argumentation). We marked correct implications white, wrong implications dark grey. The implications without success rates had been covered in the lecture before.

Figure 2 :

 2 Figure 2: Typical examples for "B2" (left) and "A" (right) (translated by the first author)

Table 1 : Student answers for T10

 1

	Points	Code	Number of	Points	Code	Number of
	(10 B)		students	(15 B)		students
		2 B1	0		2 W1/W2/W3	0
		B2	13		1 W0	1
		A	8		Bx	1
		E1	0		Z1	2
		1 B0	1		Z2	1
		X	2		X	2
		0 E0	1		0 B	5
		X	1		X	1

Table 2 : Student answers for T15

 2