Nacho Monreal Galán

Noemí Ruiz-Munzón

Berta Barquero
email: bbarquero@ub.edu

Central dialectics for mathematical modelling in the experience of a study and research path at university level

Keywords: Mathematical modelling, study and research paths, dialectics, questionsanswers, media-milieu

This paper presents the a posteriori analysis of a study and research path (SRP) on comparing reality and forecasts of the number of users of certain social networks, which appears as a teaching and learning proposal for mathematical modelling. We analyse the main elements of the SRP that have been experienced with a first-year course at university in management sciences degrees in two consecutive courses, 2015/16 and 2016/17. We focus our analysis on two essential dialectics for mathematical modelling to be developed: the questions-answers and the mediamilieu dialectics. In particular, we take empirical results from the two successive implementation of the SRP to outline through which mechanism these two dialectics could be prompted.

INTRODUCTION: THE SRP AS TEACHING PROPOSAL FOR MATHEMATICAL MODELLING

The starting point of the research is the problem of inquiring into the conditions that can help, and constraints that hinder, that mathematical modelling can be integrated and developed in the teaching and learning of mathematics into current educational systems, in particular, at university level. Researchers and practitioners agree that teaching should not be focused only on the formal transmission of knowledge, but also should provide students of the tools for enquiring into the study of real phenomena and integrate mathematics as an essential modelling tool. This change requires moving from a more traditional pedagogical paradigm of transmission of knowledge, which mostly focuses on introducing students to already built mathematical knowledge, to a paradigm of inquiry where the solving of problematic questions leads learning processes and motivates the study of new knowledge.

In the particular case of the research on modelling and their applications and on inquiry-based approaches some big steps have been made showing how, under certain suitable conditions in different educational levels and curricular frames, modelling activities may be successfully put into practice [START_REF] Artigue | Conceptualizing inquiry-based education in mathematics[END_REF]. However, although school institutions and researchers agreed that modelling should play an important role for a change towards a new pedagogical paradigm, the real situation in school and university is not satisfactory (Stillman et al. 2013) and the dissemination and long-term survival of these teaching proposals based on modelling follows as a big challenge for mathematics education [START_REF] Galbraith | Beyond the low hanging fruit[END_REF], Burkhardt 2006).

In the case of applications and modelling a shared excitement unites many who have enthused about early experiences in the field, for example when students unleash latent power that for whatever reason had remained fettered in their previous mathematical life. However this very exhilaration can work against further progress, both individually, and particularly at a system level, by creating a sense of adequate achievement that obscures the reality that there is so much more to do.

In our research, developed in the framework of the anthropological theory of the didactic, we focus on the use of the study and research paths (SRP) as epistemological and didactic model [START_REF] Chevallard | Teaching Mathematics in Tomorrow's Society: A Case for an Oncoming Counter Paradigm[END_REF]Winslow et al., 2013;[START_REF] Serrano | Recorridos de estudio e investigación en la enseñanza universitaria de ciencias económicas y empresariales[END_REF] to face the problem of moving towards a functional teaching of mathematics and, particularly, where mathematics are conceived as a modelling tool for the study of problematic questions. According to [START_REF] Barquero | Didactic engineering as a research methodology: from fundamental situations to study and research paths[END_REF], the starting point of an SRP should be a lively question of real interest for the community of study (students and teacher/s). The study of Q 0 , called the generating question, evolves and opens many other derived questions Q 1 , Q 2 ,…, Q n . The continuous looking for answers to Q 0 (and to its derivative questions) is the main purpose of the study and an end in itself. As a result, the study of Q 0 and its derived questions Q i leads to successive temporary answers A i that can be helpful in elaborating a final response R ♥ to Q 0 . These first characteristics can be associated to the first level of analysis of the SRP that we here consider, it consists in the dialectics establishing between the questions posed and the likely answers appearing (questions-answers dialectic) which also provide the basic structure of an SRP to be implemented and to be enriched after each implementation. This first layer refers to the evolution of questions to be faced and the necessary knowledge to be used. Another central dimension for an SRP is the media-milieu dialectics, which constitutes the second level of analysis. As described in the aforementioned investigations, the implementation of an SRP can only be carried out if the students have some pre-established responses accessible through the different means of communication and diffusion (that is, the media), to elaborate the consecutive provisional answers A i . These media are any source of information, such as: textbooks, treatises, research articles, class notes, or the teacher acting as main media. However, the answers provided are constructions that have been elaborated to provide answers to questions that are different to the ones that may be put forward throughout the mathematical modelling process. Thus they have to be re-constructed according to the new needs. Other types of milieus will therefore be necessary to test the validity and appropriateness of these answers. This second level of analysis put attention to the evolution of the students' milieu.

With this aim, we present an analysis of a particular SRP about the evolution of users of certain social networks that we will analyse in term of these two central dialectics and, more concretely, focusing on two critical questions:

(1) How to enhance dialectics between posing questions and looking for answer as engine of the modelling process? How to transfer to students the responsibilities of posing questions and looking for answers? (2) What milieu is necessary for students to facilitate a rich development of modelling? How a richer media-milieu dialectics can be developed?

DIDACTIC ANALYSIS OF A MODELLING PROCESS: THE CASE OF AN SRP ABOUT THE EVOLUTION OF THE NUMBER OF FACEBOOK USERS

We focus on analysing the case of an SRP on Comparing forecasts against reality in the case of Facebook users' evolution. The first time it was experienced was during the winter term of the academic year 2015-16 with first-year students of Business Administration Degree and of Innovation Management (BAIM), all from the 'Escola Superior de Ciències Socials i de l'Empresa-Tecnocampus', Pompeu Fabra University (see Barquero, Monreal, Ruíz-Munzón & Serrano, 2017). During the academic year 2016-17 it has been implemented again in the same university degree. The SRP has run in a modelling workshop that was optional activity for students during these last two academic years. In this paper we analyse and compare both implementations by using two central dialectics: the questions-answers and the media-milieu dialectics.

The initial situation starts from real news about a research performed by Princeton in 2014, in which it was predicted that Facebook would lose the 80% of its users before 2017. Hence, the generating question Q 0 presented to students is about: Can these forecasts be true? How can we model and fit real data about Facebook users' evolution to provide our forecast the short-and long-term evolution of the social network? How can we validate the conclusions of Princeton? The experimentation was structured in three interconnected phases linked to the generating question Q 0 , building up the a priori design of the SRP, then reflected in the design of the c-book unit. A first phase that focuses on the open research of real data about Facebook users, a second one focused on finding mathematical models (mainly based on elementary functions) that may provide a good fitting to real data, and a third one about the use of these models to forecast the behaviour of the social network in short, medium-and long-term in terms of number of users and about how to decide about best and most reliable model.

Previously, during the first term (4 ECTS of the subject) students had been getting familiar with the main properties of some groups of functions (polynomial, rational, irrational, exponential and logarithmic functions) as well as with basic topics on differential calculus and its applications to the study of the monotony and optimization of one real variable functions. Actually, before starting the first session we asked the students to answer a test on some of the mathematical tools that mainly make the workshop up, as indentifying some types of elementary functions or the concept of fitting model in certain scatter plots.

In the first experimentation 27 students, working in 'consultant teams' of 3-4 people, got the order from MS2 Consulting ('Mathematical Solutions Squared') previously described as Q 0 and they were asked to deliver a final report by the end of their work as an oral presentation as response to the MS2 request. The implementation combined face-to-face sessions in the teaching device called 'Math modelling workshop' (in a total of six 90-minutes weekly sessions) for the miss-in-common of the junior consultant teams' partial reports, with work out of the classroom. For the second experimentation 12 students (18 students started the workshop, but they left it in the second session due to external matters) worked also in teams of 3-5 people. This time we opened a Moodle virtual classroom to provide the students the teaching aid of the workshop, as well as some communication and collaborative tools (forum, a different wiki for each phase, etc.) to write their progress and pose their new questions. The generating question Q 0 was presented in a small dossier, next to the initial subquestions of each of the SRP phases (Q 1 , Q 2 and Q 3 ). The workshop run over seven face-to-face 90-minutes sessions before the final session, in which students should present their conclusions in an oral presentation in front of an external committee with representatives from MS2 Consulting.

Next we sketch in the case of the two implementations how the different dialectics were prompted by both: (a) the design of the unit (by its initial design but also by the different changes introduced according to students' requirements: new questions and answers not envisioned, new media required, etc.) and (b) the didactic gestures and devices to manage its implementation.

Integrating the dialectics of questions-answers as engine of the SRP

The a priori design of the SRP was basically the same in both implementations, structured in three interconnected phases linked to Q 0 , which guided the design of the workshop throughout its implementation. A first phase focuses on the open search of data about Facebook users; a second one focused on mathematical models (mainly based on elementary functions) that might provide a good fit to Facebook users data; and a third part focused on the use of these models to provide short-, medium-and long-term forecasts about the number of users of Facebook and on how to decide on the best and most reliable model. Figure 1 (and the explanation below) shows the link between different questions (Q i ) that were planned as likely to appear in the real implementation of the SRP and some expected answers (Ai) from the working teams. The only difference of the second design with respect to the first one was motivated for the context in which Q 0 was presented originally: the predictions made by Princeton were supposed to happen in 2017, and this year was present tense for the students of the second experimentation. Hence, we decided to make the same questions, but giving freedom to students of focusing in any other social network students were interested in. Q 1 : Which data sets about the users of the social network are better to consider in our research?  A 1 : Each group look for the data to be used and shared; the whole community agree on the terminology (year, period, units, etc.) and on the dependent and independent variables to take into account.

Q 1.1 : Which time intervals may be considered? Q 1.2 : How can data be well-organized? Q 1.3 : How to organise and visualise data? Q 1.4 : What can we say about the growth tendency of the data analysed? Q 2 : Which mathematical models provide the best fitting of data about the network users?  A 2 : Each consultant group is asked to propose and justify three mathematical models fitting real data.

Q 2.1 : Which models (based on elementary functions: linear, parabolic, exponential, etc.) may fit the data? Q 2.2 : How can the coefficients of the model be determined? Q 3 : How can we decide about the 'best' fitting model? Can we use this model to predict the future evolution of users?  A 3 : The teams need to create tools to justify why a mathematical model/s is/are the 'best' with respect to: (a) fitting data and (b) forecasting the evolution of users.

Q 3.1 : How can we compare the error committed between reality and forecasts provided by models? Q 3.2 : Can be the same model used for the short-and long-term forecasts?

Let us now comment the main features of the a posteriori analysis of the experimentations, referring here to the questions-answers dialectics level.

Regarding the first phase, we should remark the ease with which the students found real data about the evolution of the social net. The students mainly found the information by means of a graphical representation. This fact strongly determined their analysis, since they mainly focused in the graphical analysis growth tendency of the data, but not in their numerical versant, making Q 1.4 being treated before the other ones (it was considered that students would have data in table format before having graphs). With respect to the first experimentation, the fact that many groups found the same data triggered an intense debate and interchange of ideas among them, which took us to consider a brainstorming session about the previous hypothesis in the classroom, and as a consequence, the duration of the first phase was extended from 3 to 4 sessions. Due to the wealth of answers collected during the brainstorming we asked the students to deliver a first report in a poster format, synthesizing their findings, conclusions and new questions made by them. In the second experimentation the fact that students could choose a social network implied a disruption with the usual topos of the students in the process of study, since they were responsible on the delimitation of the field of study. They noticed about the difficulty of finding reliable data of some of their choices (Snapchat, Instagram, Twitter…), so finally only Facebook and Instagram were object of study, and not only the number of users with respect to the time, but also other variables that could have a relation. Another question that raised here was the role of the intervals of time of the data obtained, and how to work when data are not regularly spaced in time. These questions enriched the a priori design of the SRP. The presentation of the first phase was done on the third session, and there had not been interchange of ideas with other groups during the first phase. Furthermore, we asked students to present their plan of work: the questions that they wanted to deal with, when and how. This showed that each group had planned the next steps in many different ways and with many different variables. Nevertheless, the lack of time and our interest in the study of one real variable function made us proposed the students to use only the variables "Time" and "Users".

Let us focus now on the second phase. In both experimentations the analysis of the different proposals made arise a non-expected aspect: the use of piecewise functions.

Then the expected answer to Q 2 about the consideration of models based elementary functions (linear, quadratic, exponential, etc.) was extended. In the case of the first experimentation, since many groups worked finally with very similar data on the worldwide evolution of FB users, we took two new decisions: (a) give each team a second set of different data, corresponding to different geographical areas, in order to contrast their hypothesis and extend their study; and (b) ask for more than one fitting model for each data set. This was not necessary in the second experimentation, since each group had different data sets. Besides, in both workshops new questions and answers appeared at this stage with respect to the change of tendency of the fitting models, in accordance to a particular action or to decisions of the corresponding social network (IPO, new rival social nets, purchases of the company, new developments, etc.), which determined the moments of change of tendency. Furthermore, in the second experimentation we let the students choose a software for representing their data and the functions. This made question Q 1.2 emerge again, since they needed to adapt their data to the different software used. Just one group decided to use Geogebra, so they were provided the applets we used in the previous experimentations [START_REF] Barquero | Linking transmission with inquiry at university level through study and research paths: The case of forecasting Facebook user growth[END_REF]. Instead, another one decided to make interpolation in order to find functions fitting their data, so they used Symbolab and added some questions about how to solve non-linear systems of equations. The third group worked with Excel for representing a scatter plot, and used linear and nonlinear regression. This motivated a big change in the SRP, since question Q 3.1 emerged naturally in the exposition of their findings at the beginning of the third phase (since they have used the R-squared of their model given by the software). This gave birth to an interesting discussion on different ways of measuring the error, and the professors had to present this question as a central matter.

Concerning the third and last phase, in both experimentations we only had two faceto-face sessions of the workshop, but were not enough for a rich development of Q 3 . Although this time constraint, in the first implementation of the SRP there were some applets designed and made available for students to help on the simulation of models and its contrast to real data. It helped students to delve into Q 3 , but not many new questions appeared from this work. With respect to question Q 3.2 only one group dared with long term forecasts to give a date for the moment in which the users of the chosen social net would start decreasing. Both implementations finished with a final presentation of their modelling work and conclusions to an external committee.

Before finishing, we should remark that in both implementations the common discussions, presentations and brainstorming session became the main device for students to formulate and organize new questions, debate answers and contrast them.

The progressive enrichment of the milieu: the media-milieu dialectics

Since we have the first layer of analysis of the SRP in terms of the arborescence of the questions-answers, it is important to ask when, where and how questions can arise and answers can be developed. It is at this new level when there may appear the different elements taking part of the milieu, composed of varied elements: questions, temporary answers, pre-existing answers in or out school, means to validate answers, experimental data, etc., accessible through different kind of media (textbook, lectures, website resources, etc.). The relation among these elements can be analysed through the media-milieu dialectics. The constant dialectics between the search for data (for instance, real data about users of social networks, or about the company changes) and pre-existing answers (ways to organise data, common models to fit population evaluation, elementary functions, tools to control error, etc.) that exist in different media available for students (web resources, contents of Mathematics course, answers from lecturers from other courses...) and the creation of the appropriate means (milieu) to integrate (or refuse) them has been central in our SRP. Let us stress the importance of some of them.

In the first phase of the SRP, it was important to some groups the topics worked in another course called 'Introduction to digital communities' (running in parallel to the workshop) who helped on providing a general sense and functionality to Q 0 and to show how the students could look for real data and some techniques to organise them. All these elements took part of the media accessible to students, at the time it enriches students' milieu mainly composed at this stage of the data sets that each team chose to work with (shared and debated early with the whole class in the first implementation, and before the second phase in the second implementation; even strongly in this case, since the variety of data found was higher and let them make comparisons between different social networks). All these elements helped them to prepare a first report with the first temporary answer A 1 (a poster format given in the first implementation, and a face-to-face presentation in the second). Here we should remark the importance of making their plans explicit (especially in face-to-face sessions) to construct a common frame to be the source of new questions, as well as and to integrate in their milieu new concepts about modelling, and ideas of other groups that could help them. It is in the second phase in which we find more differences. In the first experimentation, the a priori design contained some Geogebra applets proposed to help students to explore different models based on elementary functions (Q 2 ). These applets provided the main media for students to visualize data jointly with model simulation, and also took part of their milieu as main tools for contrasting, comparing and deciding on the 'best' models to choose. Nevertheless, there were other tools not planned in the a priori design (as piecewise functions, or Gaussian functions, most of them part of their milieu, since they had been introduced in previous courses) but provided by designing new applets. In the second experimentation only one group used these applets, so their path followed was more similar to the first ones; but two groups decided to use other software mentioned above (that they could know from Statistics or other subjects), which made the main difference with the first implementation: meanwhile the first applet seemed to drive students to apply only a trial-error method, tools like interpolation or regression made students arise an earlier answer. Here again the common forum stated as a face-to-face session motivated an enrichment of the student's milieu. Regarding the third phase, there were several important questions that were not addressed properly, such as Q 3.1 about the way to measure the differences between data and forecasts, but here there is a main difference between both implementations. In the first one the students assumed and uncritically used the milieu made available through the design of an applet, a sort of black box to get immediate answers. Instead, in the second one students had to construct their own tool for measuring the error, and one group made it with Excel. Just one group could answer Q 3.1 but the answer was totally produced by them, so they could communicate it to the rest of the class, extending the appropriate milieu to other groups.

FINAL REFLECTIONS

First of all, we should mention that students are not in general motivated to validate their results after a work of research, since a lecturer will finally do it. In this workshop students were responsible to validate or justify every decision they made by the end of each phase. And this is the main reason why other questions arise and contribute to enrich the a priori design of the SRP.

In this paper we focus on the case of an SRP on comparing forecasts against reality in the case of the evolution of the number of users of certain social networks to show the use of two dialectics: the one of the questions-answers and of the media-milieu, corresponding to two of the three complementary level of didactic analysis of teaching and learning processes [START_REF] Chevallard | Un concept en émergence : la dialectique des médias et des milieux[END_REF]. Besides their analytic use, they suppose a productive framework to enrich teaching and learning practices, in particular, on modelling.

In what concerns to the questions-answers dialectics, the generating question Q 0 about the controversy of the article by Princeton was adopted by the students with a great interest from the very beginning and, up to the end of the process, was kept alive. From the two presented implementations we can underline very important conditions that were created. First, the flexibility of the lecturers and designers team that were opened to readjust the schedule according to students' team work. Furthermore, they were very attentive to integrate in their presentations all new questions and means that the students asked for. Second, students were very active on the sessions to share their proposals, making derived questions emerge naturally, some of them planned in the a priori design, some others that extended the initial proposal. Regarding the media-milieu dialectics, in the case of this SRP, we took several decisions along the implementation of transforming the media offered to students to help them in the modelling process and also to observe the impact new media had on students' milieu. Nevertheless, giving students the chance of using their own ICT tools, as was decided for the second experimentation, enriched the media-milieu dialectics, since it helped to arise other different answers that had not happened during the first experimentation. We may insist again on the role played by very important contributions, such as collaboration with other subjects, focusing some workshop sessions on discussing external answers that students brought, the creation of applets to foster students' experimental work, among other interesting aspects.

Figure 1 :

 1 Figure 1: Tree of questions and answers of the different phases of the SRP

ACKNOWLEDGMENTS

The research leading to these results has received funding from the Spanish R & D project: EDU2015-69865-C3-1-R and EDU2015-64646-P.


M Wold
email: margrethe.wold@uia.no

Students' usage patterns of video recorded lectures in an undergraduate mathematics course

Keywords: teachers' and students' practices at university level, the role of digital and other resources in university mathematics education, video recorded lectures, streaming media

It is customary for some universities to offer recordings of live lectures to students. Whether this improves learning and academic performance is debated in the literature. As with most technology, correct usage can lead to increased academic performance, but there are also usage patterns that can be considered counter productive, especially for learning mathematics. In order to investigate patterns in students' usage of such online recorded live lectures, we analyze log files from a server holding the recordings of an undergraduate mathematics course. This poster presents results from the statistical analysis and discusses some of the usage patterns found in light of Moore's theory of transactional distance.

INTRODUCTION AND THEORETICAL FRAMEWORK

It is becoming customary for some universities to stream lectures and make recordings of lectures available online, in particular for large enrolment courses. Students appreciate the flexibility that access to recorded live lectures give [START_REF] Yoon | Student perceptions of effective use of tablet PC recorded lectures in undergraduate mathematics courses[END_REF], but it is unclear whether it improves learning and academic achievement, particularly in subjects such as mathematics where conceptual understanding is more important than rote learning. Several studies indicate a weak association between frequent use of online recorded lectures and poorer performance [START_REF] Brooks | Modeling and quantifying the behaviours of students in lecture capture environments[END_REF]Howard, Mehan, & Parnell, 2017;[START_REF] Inglis | Individual differences in students' use of optional learning resources[END_REF]Ward, 2011, Trenholm, Alcock, &[START_REF] Trenholm | Mathematics lecturing in the digital age[END_REF]references therein). [START_REF] Yoon | Student perceptions of effective use of tablet PC recorded lectures in undergraduate mathematics courses[END_REF] suggests that easy accessibility of recorded lectures may give some students a false sense of security, resulting in procrastination and missing the lecture completely in the end. One of the three variables in Moore's theory of transactional distance [START_REF] Moore | Theory of transactional distance[END_REF] is autonomy, the ability a student has to manage their own learning (the other two variables are structure and dialogue). Within this theory, students with a high degree of autonomy would be able to utilize recorded lectures better than students on the other end of the autonomy-spectrum.

RESEARCH TOPIC AND RESEARCH QUESTIONS

The research topic of this poster is students' usage patterns of online recorded lectures in an undergraduate mathematics course. Data are collected from log files of viewings of recorded lectures in a first year (second semester) course in linear algebra and differential equations at the Department of Engineering Sciences at the University of Agder held in spring 2016, with a total of 381 enroled students. We analyze the data to answer the following research questions: How many students watch the recordings? Do students watch the recordings regularly or in a more random fashion? How much of each recorded lecture is watched? When do students watch the recorded lectures? What is the delay between when the lecture was given and when the recording is watched?

RESULTS

We present the results of the analysis in this poster in terms of histograms and bar plots with accompanying texts. In short, our results are as follows: Out of 381 enroled students, approximately 80 students (21%) use the recordings on a regular basis, with a marked increase in viewings around sports-and Easter holiday. Closer analysis shows that typically all lecture content in the videos are watched, suggesting that most students watch the whole recording instead of just smaller parts of it. The maximum number of viewings on a day happens on the same day as the live lecture, and the viewings then drop during the following two-three weeks after a lecture. There are relatively few viewings before the exam suggesting that students do not use the recordings for exam preparations.


Jokke Häsä
email: jokke.hasa@helsinki.fi

Juuso Henrik Nieminen

Johanna Rämö

Henri Pesonen

Matti Pauna

Determining your own grade -Student perceptions on self-assessment in a large mathematics course

Keywords: Assessment practices in university mathematics education, Novel approaches to teaching, Self-assessment, Student perceptions, Ownership of learning

In our poster, we report on students' perceptions on self-assessment in large undergraduate course context taught with the DISA model (Digital Self-Assessment) in which the final exam is replaced by self-assessment. The students found that practising self-assessment skills was valuable and encouraged them to study for themselves. At the same time, self-assessment was considered unfamiliar and therefore difficult. We argue that the DISA model encouraged students to take more responsibility on their own learning.

INTRODUCTION AND THEORETICAL BACKGROUND

Self-assessment skills are often mentioned as an important ingredient in becoming an effective life-long learner [START_REF] Boud | Sustainable assessment: rethinking assessment for the learning society[END_REF]. Good self-assessment skills are vital also during studies, as they form a key part of self-regulation [START_REF] Zimmerman | Becoming a self-regulated learner: An overview[END_REF]. In university mathematics, self-assessment practices have been shown to promote learner autonomy and mathematical communication skills [START_REF] Stallings | Student self-assessment and self-evaluation[END_REF]. In a study for engineering students, self-assessment was linked to better time management and more effective learning [START_REF] Friess | Formative Homework Assessment Strategies to Promote Student Self-Reflection and Improve Time Management: A Pilot Study[END_REF]. In earlier studies, the use of self-assessment is usually limited to low-stakes training exercises or controlled self-grading of the final exam. We, however, do not want merely to introduce a reflection tool, but instead question thoroughly the traditional concept of external assessment in mathematics. The aim of our study is to develop the theory of self-assessment in the specific field of mathematics.

In the DISA model, the traditional end exam is removed, and final course grades are awarded by students' self-assessment based on a learning objectives matrix. We use [START_REF] Yan | A cyclical self-assessment process: towards a model of how students engage in self-assessment[END_REF] model of cyclical self-assessment process as our theoretical framework. In the DISA model, the development of self-assessment skills is supported with various feedback methods throughout the course. The students' reflective skills are formatively practised with several self-and peer-assessment exercises. In an earlier study [START_REF] Nieminen | Replacing Exam with Self-Assessment: Reflection-Centred Learning Environment as a Tool to Promote Deep Learning[END_REF], we found that mathematics students connected the self-assessment training and the lack of exam with deep learning approach. The purpose of this poster is to present students' perceptions on self-assessment in the DISA model.

DATA COLLECTION AND ANALYSIS

The participants of the study were students of a linear algebra course taught with the DISA model in a Finnish university in autumn 2017. Data was collected after the course with a questionnaire with open ended questions (n=113). A qualitative content analysis was conducted to study the student perceptions on self-assessment.

RESULTS

Data analysis of qualitative survey data resulted in two main categories. One category was "Ownership of learning", containing a theme of positive perceptions on the self-assessment method and another theme concerning developing one's own reflection skills. The other category was "New kind of learning culture". Students' comments in this category were either "micro level" comments concerning the course arrangements or "macro level" comments concerning self-assessment as part of a new kind of learning culture. Our results indicate that the theory of cyclical selfassessment can be applied to the field of mathematics. Furthermore, we argue that digital learning environments can be used in large mathematics courses to practice self-assessment skills.


Floridona Tetaj
email: floridona.tetaj@uia.no

STUDENTS' UNDERSTANDING OF MATHEMATICAL MODELS IN THREE NORWEGIAN BIOLOGY COURSES

Keywords: mathematical models, mathematical understanding, biology students

INTRODUCTION

Development in Biology over the last three decades have greatly increased the need for using mathematics in this field. This has influenced biology students which need to develop mathematical knowledge to be able to work with contemporary biology models, including frameworks that are applicable in analyzing the overwhelming flow of biological data (i.e. [START_REF] Labov | Integrated biology and undergraduate science education: A new biology education for twenty-first century?[END_REF]. Although a quantitative approach is often used in university biology courses, yet they remain largely qualitative and descriptive [START_REF] Nelson | MatheBench Biology Modules, Web-Based for All Biology Undergraduates[END_REF]. In Norwegian context, where this study will be conducted, there is an increased awareness towards mathematics from the Departments of Biology. However, there is a lack of information on students' understanding and usage of mathematics within this context. This poster addresses this issue with a particular focus on the use of mathematical models (MM) as dynamic tools that allows us to observe various aspects of students' understanding.

UNDERSTANDING OF MATHEMATICAL MODELS

Mathematical understanding has been the scope of many researchers. [START_REF] Sfard | Reification as the Birth of Metaphor[END_REF] distinguish between operational and structural way of understanding. While the operational understanding includes highly manipulative skills and use them as principal means in their quest after meaning, the structuralist is more capable of direct-grasp understanding. She defines as reification the transition from an operational to a structural way of thinking, and states that this transition is a basic phenomenon in the formation of a mathematical object.

According to [START_REF] Niss | Models and Modelling in Mathematics Education[END_REF], a mathematical model can be defined as mapping (translation), f, from a mathematical domain, D, to a mathematical realm, M. In this context, D and M, represent not only sets of objects but also collections of relationships, phenomena or questions, while f operates on objects and the relationships, phenomena or questions. It is important to point out the distinction between mathematical model and mathematical modeling. While mathematical model is a tool for facilitating quantification, analyses, predictions and gaining insight for a real-world situation, mathematical modeling is the process of the creation of such tool (GAIMME, 2016). In this study, I focus only on the use of mathematical models as a tool that allows one to observe different shades of students' mathematical understanding, and not on the process of mathematical modeling. Observing biology students while engaging in biology problems that use mathematical models as their representation, I aim to describe their way of thinking and reasoningas operational or structural thinking.

METHODOLOGY

This study is a descriptive case-study in a naturalistic paradigm. For the pilot phase of this study, have been selected three different biology courses from a Norwegian university (two courses of bachelor level and one in master level). These courses have been selected considering their use of mathematical models. In these courses, mathematics is implicitly presented using mathematical models (i.e., population dynamic models). All students taking these courses have taken previously at least one mathematical course (mostly, Calculus course or Statistic course).

For collection of the data, I plan to video-record sessions in these courses when students are engaged in group-work. I will use semi-structured interviews with some of the students after some sessions, and personal written notes.


Ignasi Florensa
email: iflorensa@euss.es

Marianna Bosch²

Jordi Cuadros²

Josep Gascón

Helping lecturers address and formulate teaching challenges: an exploratory study

Keywords: university teacher education, pedagogy, knowledge to be taught, scale of levels of codeterminacy, teacher problems

In this paper we present an exploratory study on the kind of questions or difficulties lecturers point out at the beginning of an educational courseto be addressed in it. These questions happen to be very general and are poorly connected to the knowledge to be taught. We suggest a twofold interpretation of this phenomenon. On the one hand, and in the line of the didactic transposition theory, teachers do not allow themselves to raise questions about the knowledge that is supposed to be their main field of expertise. On the other hand, the prevailing institutional pedagogy does not provide teachers with a fruitful enough conceptual frame to formulate this kind of questions. From the experience of several lecturer education courses, we postulate that didactics can help university teachers better interpret their practice and question it in a more productive way.

The second course was held at EUSS-UAB in Barcelona, an engineering school offering Mechanical, Electronic, Electrical and Management engineering degrees. The participants were in-service teachers with different educational backgrounds and research fields. The course was organised in six 2-hour sessions. It was based on a study and research path for teacher education, an inquiry-based teaching format focused on the study of a professional teaching question (Florensa, Bosch, & Gascón, 2017). The question addressed was "Could modelling be the main motivation of my subject?" It was approached through different phases where participants experienced an inquiry study process in the position of the students, then analysed the process experienced and finished by designing an inquiry study process for their subjects.

Both courses started by asking the participants to provide a list of questions or difficulties they would like to address with the help of the educators. In all the cases, we were surprised to find there was only a small number of questions that dealt specifically with the knowledge to be taught. Teachers mainly mentioned general issues related to assessment, class management, coordination or student characteristics (diversity, lack of motivation, the role of mathematics in their subject, etc.). They rarely included their subjects in the questions and, when they did, the problems formulated were very general.

We compared this result with an investigation research carried out by [START_REF] Cirade | Devenir professeur de mathématiques: entre problèmes de la profession et formation en IUFM[END_REF] in pre-service teacher education in France within the anthropological theory of the didactic (ATD). In this research, during 3 editions of a 25-week course in 3 academic years, the participants who were doing an internship in secondary schools were asked to formulate a question every week. These "questions of the week" constituted the basis of the course, despite the fact that only a small sample of them could be addressedall in all, more than 7,000 questions were collected. Cirade provides a systematic gathering and analysis of the teacher-students' spontaneous questions and uses them to identify the mathematical difficulties teachers encountered and their trouble in making them explicit. The kind of questions raised at the beginning of the coursewhich coincided with the beginning of the academic yearwere initially very general, and were related to how to behave in class, how to manage the students' behaviour, what to do in a meeting with parents, etc. Then, as the teacher education course progressed and certain tools coming from the field of didactics of mathematics were introduced, teacher-students became more and more able to state questions related to the knowledge to be taught. In a sense, we can say that they stopped taking the knowledge to be taught as a given and dared to state questions about their own field of expertise. For instance, they ended up asking questions such as "How to justify the need of sketching functions given their analytical expression?", or "Why do we need to measure angles in radians in addition to degrees?", etc. Following Cirade [START_REF] Chevallard | Les ressources manquantes comme problème professionnel[END_REF][START_REF] Cirade | Devenir professeur de mathématiques: entre problèmes de la profession et formation en IUFM[END_REF], we postulate that educational courses for university teachers cannot ignore the way teachers problematize their professional practice and teachers should take their concerns and difficulties as the starting point of educational processes. Besides, as researchers in mathematics education, we also agree with the importance of approaching these questions from a discipline-based level. As stated by Berthiaume (2009, p. 215):

For some time now, educational researchers have investigated the idea that, in order to be effective, higher education teaching may have to be 'discipline-specific'. In other words, teaching in higher education has to take into account the specific characteristics of the discipline being taught. This means that developing an understanding of teaching and learning is not sufficient to become an effective teacher in higher education. Rather, one must also develop an understanding of the teaching and learning requirements of one's own discipline. This has been termed 'discipline-specific pedagogical knowledge'.

We consider essential for university teachers to be able to formulate their difficulties, not only as general issues concerning students and class management, but also including the knowledge to be taught as a key element. Even if teaching problems are initially perceived as general in their manifestation, the way to address them will necessarily involve knowledge-based activities. From the perspective of the ATD, taking the knowledge to be taught into account means more than simply including it as a variable or parameter of the problem formulation. It also means considering it as an institutional construction, questioning its current shape and searching for possible new reorganisations, taking into accountwithout assumingthe epistemologies and pedagogies prevailing at the university [START_REF] Barquero | The ecological dimension in teaching of mathematical modelling at university[END_REF].

The aim of our study, which is still at an exploratory stage, is to analyse the kind of questions university teachers are able to state at the beginning of an educational courseas the ones we implementedand locate their questions at different levels of specificity/generality regarding the knowledge to be taught. We postulate that knowledge in didactics is important to provide university teachers with conceptual and methodological tools not only to improve their professional practice, but also to describe, interpret, conceive and question it in a more productive way. The first step to make progress in this direction is to start understanding how lecturers spontaneously formulate the challenges faced during their daily practice.

WHAT PROBLEMS DO LECTURERS SET FORTH?

We collected a total of 143 questions from the 4 courses, 35-40 per course, each of which was attended by 10-15 participants. In all of the cases, teachers attending the course were asked the following: "Write down two or three problems, difficulties or doubts that you find, or you think you may find during your teaching practice." There was a lot of redundancy in the questions, so we eliminated repetitions even if the phrasing was different. We are presenting this selection according to the questions' generality, using the scale of levels of didactic co-determinacy. This tool was introduced by [START_REF] Chevallard | Organiser l'étude. Écologie & regulation[END_REF] in the didactic analysis to include aspects of the institutional organisation of teaching and learning processes that are usually taken for granted [START_REF] Artigue | International Comparative Studies on Mathematics Education : a Viewpoint From the Anthropological Theory of Didactics[END_REF][START_REF] Chevallard | Anthropological Approaches in Mathematics Education, French perspectives[END_REF]. It helps distinguish the conditions and constraints affecting teaching and learning processes that are originated within the discipline, and the generic levels common to the teaching of any discipline:

Civilisation and Society

The upper levels of the scale refer to the conditions that are set up by our society or, when these are common to several societies, by the civilisation they belong to. We identified the following questions at this level:

-What to do in a culture in which effort and reward are no longer related? -How does a social situation influence the effectiveness of a course or a teaching format? For example, the students' attitude seems different in times of crisis… -To what extent should study plans be aligned with the labour market? -What is "academic freedom" and what are its limits? -Where do competencies come from? How are they established? -Clashing of (generational or social?) values: sometimes it is difficult to act as a teacher, a guide or a referent when our own values seem to be obsolete (or to strongly contrast with those of our students). For example: the value of effort, the gratification of work well done, the fact that money cannot buy everything or that not everything is on the web, the importance of culture, of thinking, that there are things that are "right" and others that are "wrong" (e.g. cheating in an exam is "wrong"), etc. -What to do with students who act as "clients"? -The application of Bologna is an adaptation of the learning process and an evolution or change: more participative students, more teacher-student interaction, etc. Adaptability is therefore considered a consequence of an evolution, but if we analyse it, we are giving the same classes, in the same environment, with the same student profile. Can we do anything to make the context change? -How is the knowledge of the different subjects selected and what criteria are used? -How far should we, as teachers, arrive in our role in and out of class? What are teachers educated for?

As we can see, all these questions refer to dimensions or difficulties related to university teaching that do not depend on the specific institution consideredmany of them can be extended to any kind of teaching and to other educational levels. What is questioned is the way our societiesmore or less explicitlyconceive, organise, and manage the dissemination of knowledge, and the general roles assigned to teachers as guides, leaders or knowledge disseminators.

School (here University)

The School level includes the conditions and constraints that depend on the specific teaching institution considered, in our case, the University with its own specificities:

-Is the number of students per class important in terms of effectiveness of the teaching? Is there an optimal number? Are there exceptions? -How are decisions regarding time-schedule, session duration, etc. taken? -To what extent are university facilities important? Are there optimal premises? How to adapt to the ones available? -How to respond to the pressure of introducing ICT in the classroom? Is it used because of real educational reasons or is it cost saving? Is it just a trend? -How to ensure a good coordination between teachers of the same subject? What happens when they have a different conception of the subject to teach? -How to establish more synergies between colleagues, sharing methodologies? -How to ensure a coherent programme? What relationships exist between subjects? -How to integrate the different subjects to obtain a more global education?

These questions also reveal the aspects teachers think can be changed and the ones they take for granted, not even considering them questionable. For instance, in the fifth question, only the coordination with teachers of the same subject is considered, according to the traditional compartmentalization of knowledge in higher education. Together with the sixth question, they reveal the lack of a professional culture that might include coordination between teachers. However, the sixth question seems to consider that this coordination only affects "methodologies", which again appears to be a vague and general dimension of teaching. The last two questions are contentrelated, but only with respect to the relationships between subjects, as these are considered to be previously determinedand, therefore, untouchable.

Pedagogy

The level of pedagogy is common to the different subjects or disciplines that can be taught. It includes all the resources, formats, and strategies teachers and students activatemany times without even noticing itfor teaching and learning processes to take place. We gathered numerous questions that can be located at this level of the scale and organised them into two main blocks: students and lecturers.

Students

-How to manage long projects, where students slack off and decrease their work intensity? How to reach a balance between establishing milestones and letting students work independently? -What to do with the students that chat, are unfocused, use their mobile phone, do other things than what is requested of them? -How to deal with student diversity? -How to arouse the students' interest in subjects that are not at the core of the degree? -How to motivate students beyond the minimum required "pass" grade? -How to encourage students to participate in a large group? -Should students be monitored closely or should they work more independently? -How to motivate students to behave in class? -How to encourage students to be more competitive through the intrinsic values of the subject that is taught? -Students are not previously taught how "to learn". How will this affect our job? -How to teach students to listen and maintain their attention? Lecturers -How to improve oral and body expression? -How to organise assessment in a fair and impartial way? -How to assess core (non-disciplinary) competencies? -Is it better to use final examinations or continuous assessment? How to measure long-term student learning outcomes? -How to design contents, planning and methodologies of the subject that take into account the student diversity? -How to reach all the students and not only those who have more knowledge, excluding the ones that got lost? How to find the balance between maximising student learning and the amount of information provided? -How to ensure an individualised methodology considering the time limitations? -How to become the best teacher for each student? -How to deal with the so-called "decline in student knowledge"? -Does the decline in student knowledge correspond to something real or is it just what each generation says about the previous one? -Does it make sense to give lectures nowadays? -How to improve teaching resources and methodologies using ICT? -What to do after the class? How to analyse what happened and what the teacher did?

How to assess teacher performance?

As can be seen from the questions above, most of them focus on specific teaching practices, but they do not refer to the difficulties of the corresponding subjects. The questions are mainly related to what the teachers can or might do, and they are very general. Only two of them refer to specific teaching formats: projects and lectures.

There is no mention of the activities organised (labs, tutorials, problem solving or case study sessions, outdoor activities, etc.) and the way to better implement them.

The need to implement new kinds of activities is not mentioned. The questions mainly have to do with the teacher and the teacher's actions. For instance: assessment is always considered as a lecturer's task; "motivation" is assumed to be generated (only) by the teacher. The questions thus reveal many features of the traditional pedagogical contract, which seems to be fully assumed by the lecturers.

Discipline

As said before, we were astonished to find so few questions at this level, which corresponds to the conditions and constraints directly linked to the content taught and learned. They can be related to the general discipline the content belongs to (Mathematics, Engineering, Economics, Management, etc.) or to the different components of the discipline, according to the way it is structured or delimitated in the considered institution. The general terms used to specify these levels are: domains, sectors, themes, tasks or questions. The divisions and boundaries established in a discipline or field of knowledge are institutional constructions. They vary from one institution to another and from one historical moment to another. The collected questions at this level remain very general; none of them specifies the difficulties related to the teaching or learning of a given piece of content. The first one, for instance, is very similar to those located at the School level: it depends on whether we interpret the question as affecting the design of an entire programme or the possible actions in one discipline:

-How to better connect the different subjects of the programme? -How to highlight the multi-disciplinary nature of the subjects? -How to select the learning goals of the subject? What content should be included? -What should the level of the learning goals be? -How to relate the subject with the real world? -How to balance learning goals between specialisation and generalization?

WHY THESE QUESTIONS? AN INTERPRETATION FROM THE ATD

The assumed educational contract between lecturers and educators

The first reason that came to our mind when trying to understand why lecturers did not ask any content-related question is the kind of implicit didactic contract that was assumed by them at the beginning of the course. Given the fact that the course was about university teaching, they might have expected to learn certain generic tools to help them in their teaching practice; not something related to their specific subject. The educators were seen as specialists in teaching processes and the questions were stated at this general level. Either way, this shows a first important phenomenon: lecturers expect to receive help with general teaching practices that are only a part of their daily practice. A lot of their teaching work (elaboration of the syllabus; choice of textbooks, reference books and other kind of resources or materials; selection, design and organisation of activities, cases or problems; decisions about the kind of in-class and out-of-class activities students should carry out; renewal of the subject matter; etc.) does not seem to have been included in the objectives of the course.

A problem of legitimacy

The second reason we put forward is related to what we call a problem of legitimacy. University teachers are often also researchers or at least experts in the subject they teach. Therefore, they may be reluctant to accept the idea that their teaching difficulties might come from problems with the subject matter they are supposed to master. Their lack of expertise can only be attributable to what is external to the discipline they teach. This reinforces the previous reason about their expectations from the educational course.

The divide between pedagogy and didactics

There is another important and more general factor that may explain the lack of content-related questions. It corresponds to the dominant interpretation of teaching and learning phenomena that has been called "pedagogical generalism" [START_REF] Gascón | La miseria del "generalismo pedagógico" ante el problema de la formación del profesorado[END_REF] or the "didactic divide" between pedagogical and subject-matter knowledge [START_REF] Bergsten | The didactic divide and the education of teachers of mathematics in Sweden[END_REF]. It tends to introduce a strict separation between instructional processes and the "content" of these processes, that is, using the scale of didactic co-determinacy, between the level of Discipline and the level of Pedagogy. The main point in crossing the boundary between the two levels is the way knowledge is conceived in the considered teaching process or, in other words, which aspects of the subject-matter are questioned and which ones are assumed as a given.

When a teacheror a lectureris asked to teach a given piece of knowledge k, the first question she will first ask herself is "what should I do to teach k?", not "what is this k I should teach?" What the theory of the didactic transposition [START_REF] Chevallard | La transposition didactique. Du savoir savant au savoir enseigné[END_REF][START_REF] Chevallard | Didactic transposition in Mathematics Education[END_REF] states is that instructional processes rely on the fiction that there is only one way to define k and that this is the k that is taught and learned. Questioning the knowledge to be taught, asking about its origin, selecting and applying a given instructional process rarely occur. This is why it is normal the participants of the course did not set forth questions of that kind. In the questions stated, knowledge always appears as a given, not as a variable.

The "pedagogical generalism" that is found in the teachers' questions is not an isolated fact. If we look at the teaching support some universities offer their (new) faculty, we see that only the Pedagogical level is addressed, and possibly some aspects of the School level. For instance, in the Teacher Training in Higher Education (FDES)1 programme proposed by the Autonomous University of Barcelona, the structure of the programme is presented as follows:

Activity 1. Teaching in the new context of learning and teaching Activity 2. Practicing oral discourse Activity 3. How to assess university students' learning? Activity 4. Experiences in educational innovation Activity 5. Observation in the classroom Activity 6. Teaching planning: from study programmes to syllabus Activity 7. Teacher's portfolio Similar programmes can be found at other universities. For example, some years ago, the Teaching Engagement Program of the University of Oregon posted a list of frequently asked questions (FAQs) organised according to the following headings: "Getting ready to teach; Presenting and facilitating information; Motivating students; Questions of respect; Assessment; Managing the classroom climate". None of the questions was content-related. It seemed as if, once certain answers were provided to the pedagogical issues, their specification to the subject-matter was considered evident or, at least, non-problematic.

CONCLUSION: A LACK OF TOOLS AND NOTIONS

One of the consequences of "pedagogical generalism", that can partially be seen in the questions stated by university teachers, is the lack of terms and concepts to go below the level of Pedagogy and start questioning the levels of Discipline. University teachers develop their professional activity at institutions where little is said about the way knowledge should be selected, arranged, updated, organised, "elementarised", put-into-practice, problematized, etc. in order to teach it or to help students to learn it. This is a crucial aspect in which the results obtained from research in Didactics of Mathematics, both practical and theoretical, can assume an important function.

From the experience of the courses here presented, we have seen how introducing certain elements of the ATD (the notions of praxeology, didactic contract, didactic moments, Herbartian schema, media-milieu dialectics, didactic ecology, etc.) provides lecturers with a productive enough framework to talk about and start questioning a larger part of their teaching activities [START_REF] Florensa | Teaching didactics to lecturers : a challenging field[END_REF]. The more is said about didactic processes, the more dimensions of these processes can be questioned and tentatively changed. Our hope is that the education of lecturers, as is the case with primary and secondary school teacher education, will have the power to make this state of things evolve. Our experience with the courses presented lets us be moderately optimistic in this respect.

Scale of levels of didactic co-determinacy

	Upper levels	Lower levels
	Humanity	
		Discipline
	Civilisation	
		Domain
	Society	
		Sector
	School	
		Theme
	Pedagogy	
		Question
	Figure 1.

http://www.uab.cat/web/personal-uab/personal-uab/personal-academic-i-investigador/formacio-i-innovaciodocent/programa-fdes/estructura-1345703511726.html

ACKNOWLEDGMENTS

Funded by project EDU2015-69865-C3-1-R (MINECO/FEDER) from the Spanish Ministry of Economy, Industry and Competitiveness and the European Regional Development Fund.


Cillian Copeland

Emma Howard
email: emma.howard@ucdconnect.ie

Maria Meehan

Andrew Parnell

Engaging with feedback: How do students remediate errors on their weekly quiz

Keywords: assessment practices in university mathematics education, feedback, remediation of errors, students' practices at university level

scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

Maths for Business is a core first-year mathematics module for non-mathematics specialists enrolled on three business programmes in University College Dublin, Ireland. Topics from one-and two-variable Calculus are covered in the module and given the cumulative nature of the content, students should (ideally) achieve the learning outcomes for a topic before proceeding to the next. To encourage mastery of learning outcomes, the module has a continuous assessment component consisting of ten weekly quizzes, worth 40% of the final mark. A week after sitting a quiz, graded quizzes are returned to students with a mark out of five, and tutors provide oral feedback to each tutorial group highlighting the most common errors made. In addition, the lecturer provides an online video entitled "Most Common Errors" and posts a pdf of the quiz solutions online. With our focus on mastery, we believe that students who do not get full marks on a quiz should engage with the feedback to identify and remediate their errors in a timely manner. However, [START_REF] Gibbs | Conditions under which assessment supports students' learning[END_REF] discuss how, even if timely and good quality feedback is provided to students, there is no guarantee they will engage with it. Therefore, to encourage this engagement, in 2016/17 we offered students who did not receive full marks on a quiz, one extra mark if they resubmitted their graded quiz one week after it was returned with error(s) identified and corrected. We refer to this process as "remediation".

Handley, [START_REF] Price | Feedback: Focusing attention on engagement[END_REF] propose a shift from examining feedback evaluation and attributes to investigating the process of students' engagement with feedback. To this end, we first wanted to explore: which students were most likely to participate in the remediation process; which feedback resources were they most likely to access; and, whether engagement in the process impacted academic achievement in the module. The analysis and findings from this part of the study are described in detail in Howard, Meehan and Parnell (under review). The main findings were that 70% of students who had the opportunity to remediate did so; the most accessed feedback resource was the pdf of the quiz solutions which were made available online; and, students who achieved an average quiz mark of 3-4 (excluding remediation marks) and who consistently engaged in the remediation process, exhibited the most learning gains as measured by their performance on the final examination. Secondly, we wanted to examine how students remediated quizzes in order to identify aspects of a "good" remediation, and from these findings, refine the instructions given to students at the start of the module on how to remediate their quizzes. We also want to identify and explore what factors might influence the nature of students' remediations. It is this second part of the study that we wish to focus on in this paper by addressing the following research questions:

1. What ways, in general, do students remediate their weekly quizzes?

2. What instructions would we give to future students to assist them in remediating their quizzes?

3. What factors may influence the nature of a student's remediation?

LITERATURE REVIEW

Assessment and feedback

There have been a number of in-depth reviews in the area of assessment and feedback [START_REF] Bennett | Formative assessment: a critical review[END_REF][START_REF] Sadler | Formative assessment and the design of instructional systems[END_REF] with some focusing specifically on higher education [START_REF] Evans | Making sense of assessment feedback in higher education[END_REF]. Assessment is generally discussed under the headings of formative assessment, where the primary objective is to provide feedback to the student and evaluation of students' knowledge is secondary; and summative assessment, where the primary role is to evaluate students' knowledge and feedback is secondary. Ramaprasad (1983, p. 4) describes feedback as "information about the gap between the actual level and the reference level of a system parameter which is used to alter the gap in some way". Building on this description of feedback in terms of its effect rather than its content, [START_REF] Sadler | Formative assessment and the design of instructional systems[END_REF] argues that the learner must:

(a) possess a concept of the standard (or goal of reference level) being aimed for, (b) compare the actual (or current) level of performance with the standard, and (c) engage in appropriate action which leads to some closure of the gap (p. 121, italics in original).

While most of the major studies on assessment and feedback relate generally to a variety of subjects, there have been calls for specific domain-focused research [START_REF] Bennett | Formative assessment: a critical review[END_REF]. Specifically, in mathematics education at the university level the area of assessment and feedback seems to be under-researched. Of those studies conducted in this area, an emphasis on summative assessment and closed-book examinations has been noted [START_REF] Iannone | The summative assessment diet: How we assess in mathematics degrees[END_REF]Iannone & Simpson, 2012;[START_REF] Trenholm | An investigation of assessment and feedback practices in fully asynchronous online undergraduate mathematics courses[END_REF]. Underpinning the need to conduct discipline specific research in this area, [START_REF] Iannone | Students' perceptions of assessment in undergraduate mathematics[END_REF] found that in contrast to the general literature on assessment, mathematics students prefer traditional closedbook examinations to more alternative assessment methods.

Engagement with feedback

There has been recognition that despite timely and informative feedback being provided to students, students may not take action on it [START_REF] Gibbs | Conditions under which assessment supports students' learning[END_REF][START_REF] Handley | Beyond 'doing time': Investigating the concept of student engagement with feedback[END_REF]. [START_REF] Handley | Beyond 'doing time': Investigating the concept of student engagement with feedback[END_REF] emphasise the difference between the student who skims and bins the feedback to one who takes "responsibility for understanding, interpreting and applying assessment feedback" (p. 557). [START_REF] Price | Feedback: Focusing attention on engagement[END_REF] discuss how a student may reject feedback "due to lack of understanding, or based on identity or self-efficacy issues" (p. 892). They further state that students may need more support in taking action on feedback. Of course, feedback needs to be of an appropriate level to help the student. Similar to the remediation process, [START_REF] Covic | Is the essay resubmission option a formative or a summative assessment and does it matter as long as the grades improve?[END_REF] provided psychology students with the opportunity to remediate corrected essay assignments. In this voluntary remediation, 48% of students opted to remediate for potentially higher marks. Students' feedback consisted of individual and group feedback on their corrected essays as well as an initial grade. We have been unable to find an equivalent study in a mathematics context.

MODULE CONTEXT AND DATA COLLECTION

For our analysis, we only considered students who were completing Maths for Business for the first time and who sat the final examination in the module (n=470).

In Maths for Business, students have the choice of completing the module through using online videos or by attending lectures or a combination of both [START_REF] Howard | Live lectures or online videos: students' resource choices in a first-year university mathematics module[END_REF]. The students are assigned to one of two lecture cohort groups and have three lectures weekly. The lectures are designed to be partly interactive with at least 15 minutes for in-class tasks. All students have access to 67 videos/screencasts which cover the entire module content and have an average length of 7 minutes each. These videos were designed and developed by one of the two module lecturers (and third author of the paper). There are no recommended textbooks for this module. Students also have access to the Maths Support Centre, and prior research has shown that students focus on using module resources with very little if any use of external resources such as websites. A student's final mark on the module consists of 40% continuous assessment and 60% for the final examination. To encourage consistent engagement with the module and mastery of learning outcomes, Maths for Business has ten fifteen-minute weekly quizzes, with each quiz usually consisting of two parts. Each quiz accounts for 5% of the final mark, directly relates to the module's content for the prior week, and is marked out of five. However, only students' best eight quizzes contribute towards their continuous assessment mark of 40%.

One week after completion of a quiz, tutors returned the graded quiz to each student with a mark for both parts of the quiz and the overall mark provided. Tutors also provided oral feedback to their tutorial class on the most common errors made in the quiz. There were approximately 50 students registered for each tutorial. On the university's Virtual Learning Environment (VLE) Blackboard, the lecturer provided a video entitled "Most Common Errors" and a pdf copy of the quiz solutions that also indicated the most relevant online videos from the module that a student may wish to revise. In Semester 1 of 2016/17, students who did not receive the full five marks on their quiz were given the opportunity to resubmit their remediated quizzes for one additional mark. The following instructions were provided by the lecturer to students:

• When your quiz is returned to you go over it and identify your errors. Write a sentence beside each error on the quiz sheet so that when you are revising the material again, you will have a note to yourself about where you went wrong.

• If it is the case that your errors were more than just a "slip", then you should write out the correct solution on the quiz sheet and write a sentence or two beside the solution summarising the method.

• You should use a different colour pen so that the tutor can clearly distinguish between what you wrote in the quiz and your remediation comments.

• Imagine you are correcting your friend's quiz and you are explaining to your friend where he/she went wrong.

The third author had difficulty in articulating these instructions, hence the second research question. To remediate a quiz, students were encouraged to use any of the resources available: "Most Common Errors" video; quiz solutions; relevant online videos; Maths Support Centre; tutor feedback; and, friends. From the VLE, we were able to record when a student accessed the first three resources listed, and from Maths Support Centre records we had information on who attended the centre for remediation purposes. We have no data for the number of students who sought help from their friends, or those who made use of the tutor's feedback comments.

Owing to the semester timetable, students could only remediate the first eight quizzes. In total 1,746 remediation marks were awarded. We collected the remediated quizzes, however, as some from the first quiz are missing, we only use remediated quizzes two to eight inclusive in our qualitative analysis (n=1,511).

QUALITATIVE METHODOLOGY

Qualitative analysis of the remediated quizzes loosely followed the stages of thematic analysis [START_REF] Braun | Using thematic analysis in psychology[END_REF]. Initially, the first author examined the remediated quizzes several times in order to familiarise himself with the data. The remediated quizzes where students only provided a full, complete solution as remediation were removed as limited information could be obtained from examining them, especially since complete solutions were available as an online resource. This left 687 remediated quizzes where students had done something other than provide only a full solution. Guided by the instructions provided to students, the first author analysed each of the remaining 687 quizzes in order to determine:

1. Has the student successfully identified each error?

2. Has the student provided a solution for each error?

3. Has the student explained their error, and if so, how?

Most quizzes consisted of two questions, therefore each question was analysed separately. In order to ascertain whether a student had identified each of their errors, the first author identified each error on a student's quiz, and noted how many of these the student identified. In terms of examining whether the student had provided a solution for each error, two approaches taken by students were identified. Some students provided a full solution to the complete question even if the error made only related to part of it, whereas other students only wrote a solution for the specific error made. Finally, the first author analysed if, and how, students explained the errors made. Students seemed to vary in their approaches based on variables such as the nature of the question asked, for example procedural or conceptual, and the quiz mark received. We will elaborate on this further in the section below.

RESULTS

To allow for comparison between students with similar levels of achievement on the quizzes, we divided students into four groups based on their average continuous assessment mark (excluding remediation marks) received for the first eight quizzes (4-5, 3-4, 2-3 and 0-2). These groups were of sizes 103, 188, 115 and 64 respectively. Of the 470 students, 47% were female. The following findings are detailed in Howard et al. (under review): students who scored less than two on a quiz were less likely to remediate; students in the two lower groups showed a limited increase in final examination mark as a result of participating in the remediation process in comparison to their peers; and, students who averaged 3-4 on their continuous assessment, particularly benefited from participating in the remediation.

Students' approaches to remediation

After coding was completed on all remediated quizzes, it was possible to compare students' remediation styles within a single quiz as well as the individual progression through all submitted remediations. With regard to error identification, there was a notable difference between students with high and low quiz marks. For students with marks of 3 or 4 in a quiz, mistakes were usually simple calculation slips and thus many students had only one or two errors to identify. This resulted in the majority of students with these marks successfully identifying all their errors. However, for those with lower marks, there were more conceptual misunderstandings to identify as well as several calculation errors. It is not surprising that these students were less successful at identifying every error. For students who received low grades on average, this pattern was clearly evident, but in addition, higher grade students exhibited this style on quizzes where they obtained a lower mark. There was a clear contrast in the level of error identification between an individual students' highest and lowest scoring quiz. This supports our hypothesis that when students achieve low marks, they are less successful at identifying their errors. In terms of provision of solutions to errors, as noted above students either provided a complete solution to the question even if the error only related to part of the solution, or they provided a solution that related to a specific error.

In relation to how students explained their errors, three codes or types of explanations were identified. Based on our knowledge of feedback, we define these explanations as: diagnostic, instructional, and objective. Some students explained errors in the context of incorrect notions or ideas that lead to them making a mistake. We refer to this approach as being diagnostic for example, "I thought brackets implied find the product but I should have used the chain rule". Others focused on providing advice or helpful tips to themselves to help prevent mistakes in any similar questions they faced in future. We refer to this approach as instructional for example, "Add powers together as they have the same base (multiply rule). Finally take away the powers from each other (division/fraction rule)"). The final, and most common approach, was an objective explanation of the error, simply describing the specific error without referring to prior knowledge or providing instructions on how they might answer future questions on the topic for example, "I compounded continuously but the question asked for quarterly".

Additionally, there was a less frequent code for whether a student provided an incorrect statement, or identified their errors incorrectly. Having addressed the first research question, we propose that in future the following instructions be given to students:

1. With a different coloured pen to the one in which the quiz was completed, put an "X" beside each error or, where relevant, indicate an omission in your work.

2. Provide an explanation for each of your errors, either describing the error or if you can, elaborate on any incorrect notions you had which may have led to the error.

Correct each of your errors by writing the correct version beside each one.

Avoid copying down the written solutions for each quiz without explicitly consulting your areas of error.

4. If you are still unsure how to remediate your quiz or need help with the questions, we suggest you visit the Maths Support Centre for help.

In addition to the above instructions, we would provide students with exemplars of "good" remediations.

In-depth analysis of specific students' remediations

We now turn to Research Question 3: What factors may influence the nature of a student's remediation? Owing to the large amount of data involved, we have chosen to address this research question by examining a specific subset of the data. It is our hope that this analysis will help us identify factors that may prove beneficial when analysing the larger data set. We consider a subset of students (n=31) who remediated at least six quizzes and achieved on average 3-4 quiz marks as these had particularly benefitted from the remediation (Howard, et al., under review) and we could investigate their style of remediating over several quizzes. These students attended between 1-26 lectures and accessed between 11-232 videos with students from the weaker mathematical backgrounds (based on the Irish State Examination Mathematics results) accessing more resources than the others. All of these students passed the end-of-semester examination achieving at least a B grade (60%) and 58% were female. Twelve students accessed the Maths Support Centre and all but one of these were female. Detailed records from centre show that at least five of these students received help from the tutors on remediating the quizzes, including quizzes where they received four out of five marks. Seeking help from the Maths Support Centre for remediating the quizzes was uncommon among the larger Maths for Business cohort. Overall, based on their module resource usage, these students seem to work consistently throughout the semester.

To investigate how these 31 students remediated, their coded, remediated quizzes were further analysed to examine any prevalent styles of remediation. Within this group some students remediated consistently in the same manner each week, while others exhibited various remediation styles. Due to the initial division made between students writing only a full solution as remediation, and those who articulated some form of further explanation, we decided to categorise a student's overall remediation style based on the amount of times a remediation consisting of just a solution was submitted. Students could be split into roughly equal groups based on whether they provided only full solution in every remediated quiz (n=10), more than half of their remediated quizzes (n=8) or in half or less of their remediated quizzes (n=13).

Overall there was no obvious difference in the resources accessed (Maths Support Centre, quiz solutions etc.) between the three groups of students. In terms of error identification, these students were, in general, successful in identifying all their errors. A prevalent remediation style consisted of students identifying their errors, providing an objective explanation of the error, and writing a solution of the specific error. With regard to the three main approaches to error explanation, the objective explanation was the most common method. Interestingly, the diagnostic and instructional approaches were rarely used consistently in remediations by individual students.

The nature of the quiz question seemed to impact the remediation approach. The quiz questions could be considered under the headings: conceptual ('The following is a graph of the first derivative f'(x) of a function f(x). Your friend is attempting this problem...He asks you: "How can you tell that f has a minimum at x=2 just by looking at the graph?"'), procedural ('Find all first and second-order partial derivatives of z = f(x,y) = ...) and economic context ('Compute price elasticity of demand, E... In one sentence, explain what your answer means'). Notably, for these students, quiz questions that were more conceptual in nature, resulted in more "solution only" remediations. As full solutions were provided as a resource to students, it was possible to copy them and submit it as remediation. Thus, an increase in the number of solution-only remediations for a given quiz question may allude to a lack of student understanding of the module content. A number of quiz questions also resulted in more diagnostic remediation responses. These questions were more application based, and focused on application of techniques to economic contexts. The two quizzes that contained these applied questions had the highest average marks of all eight quizzes, with 21/31 quizzes obtaining full marks between the two. We believe the increased use of diagnostic remediations may indicate students find it easier to locate misconceptions in mathematical application than with more abstract questions.

DISCUSSION AND CONCLUSION

Semester 1 2016/17 was the first time we implemented the remediation process. Following from the definitions of [START_REF] Ramaprasad | On the definition of feedback[END_REF] and [START_REF] Sadler | Formative assessment and the design of instructional systems[END_REF], students were provided with their actual level of understanding (mark out of five), the desired level of understanding (for example, pdf of quiz solutions), and were incentivised to engage with feedback for the intention of closing the feedback gap. [START_REF] Gibbs | Conditions under which assessment supports students' learning[END_REF] propose that even if feedback is provided, students will not necessarily use it. Despite assessment marks being provided as an incentive, on average 70% of students engaged in the remediation process. Initially, we considered providing remediation marks on a sliding scale based on a student's initial quiz mark received. This method would reflect the additional work of remediating for lower-scoring students and offer them additional incentive over high-scoring students, however, this system was not implemented as it required additional time and effort from the tutors.

Qualitative analysis of the entire set of remediated quizzes allowed for the isolation of certain properties that indicate whether a student is engaging with their assessment feedback. We note that a substantial number of students provided the full solution only for their remediation, however we advocate the three-step approachidentifying, explaining and correcting errors encourages students to recognise their own standard and utilise feedback to help close the gap between their own level and the desired standard. It is for this reason that we suggest this style as a guideline for future processes similar to remediation.

Owing to the brevity of the explanations provided in this paper on the different remediation responses, it is pertinent to mention different avenues that will be investigated as this research progresses. The in-depth analysis of the 31 students in this paper suggests that task design and a student's initial grade influences remediation style. Different task types elicited different remediation responses, based on whether the question was conceptually or application based. Perhaps, owing to a lack of understanding of the content material or the inability to transfer knowledge from one context to another, students tended to provide full solutions to conceptual questions. In addition, quizzes where students received lower grades tended to result in more solution only remediations. As these solutions are available as resources, this may suggest that some students utilise solutions when they are unable to fully identify and understand their errors on a quiz. From this, one can hypothesise that the remediation process may have less benefit to weaker students as a level of baseline knowledge may be required in order to engage with the process. In subsequent research, task design and initial quiz mark will be examined further to discern any influence on remediation response across the full set of remediated quizzes.

One limitation of our study is the lack of student perspective on the remediation process. Also, while the remediation process was beneficial to the students of this large non-specialist mathematics module, an investigation of the benefits/drawbacks of the remediation process for mathematics modules for specialists would be a constructive contrast with this research.

This study was completed with approval from the University College Dublin ethics committee in accordance with ethics applications LS-E-17-20-Copeland-Meehan and LS-16-48-Howard-Meehan.