Sandra Gaspar Martins
email: sicgm@campus.fct.unl.pt

Weekly homework quizzes as formative assessment for Engineering students are a fair and effective strategy to increase learning?

Keywords: The role of digital and other resources in university mathematics education, Assessment practices in university mathematics education, Teaching and learning of analysis and calculus

A strategy to apply online weekly homework quizzes as formative assessment for Engineering students was designed and tested in order to study if it increases student's learning. The strategy was to make optional weekly online quizzes with questions not randomly generated that students may retry over and over again until to reach the correct answer, they contribute to 10% of grade but only if students get 45% or more in usual pencil and paper assessment.

The quizzes were applied to two different mathematics courses (Single and Multivariable Calculus) of two different Engineering degrees, each one to around 100 students and during a semester. Student's adherence was very high, nearly all students refer quizzes as fair and useful to learning. Students' grades were compared with several other years.

INTRODUCTION

Frequent online quizzes have been suggested as a strategy to enhance learning by several institutions and researchers. The National Centre for Public Policy and Higher Education in the U.S.A [START_REF] Twigg | Course Redesign Improves Learning and Reduces Cost[END_REF] consider computer based continuous assessment and feedback to be a key strategy for quality improvement in learning. According to [START_REF] Gibbs | Changing student learning behaviour outside of class[END_REF], student assessment is an effective way to increase understanding and online quizzes force students to spend more time working productively outside of class. [START_REF] Tuckman | Using tests as an incentive to motivate procrastinators to study[END_REF] refers this as being especially valuable to procrastinators. One method that can be used to address the crisis in college mathematics, according to [START_REF] Thiel | Addressing the Crisis in College Mathematics: Designing Courses for Student Success[END_REF], is to 'provide regular assessment of progress' and they state that 'online homework and quizzes with online grading provide students with immediate feedback, the opportunity to correct their homework mistakes, and ongoing assessment of their success in the course'. [START_REF] Booth | Managing Mathematics with CALMAT[END_REF] considers that homework should be given out at regular times, over regular intervals, on a weekly basis; proposing that learning is work and students should develop regular work habits in order to succeed. Feedback is crucial for student success but giving adequate feedback with large class sizes is difficult and therefore automated systems are a useful solution to the large class size problem.

Quizzes are part of several successful approaches with different kinds of students, both in top universities and in other higher education institutions. Examples include: TEAL [START_REF] Dori | Improving Students' Understanding of Electromagnetism through Visualizations-A Large Scale Study[END_REF] at Massachusetts Institute of Technology (MIT); SCALE-UP [START_REF] Beichner | The SCALE-UP project[END_REF] at North Carolina State University; Peer Teaching [START_REF] Lasry | Peer instruction: From Harvard to the two-year college[END_REF] at Harvard University.

Particularly, in higher education mathematics teaching, several approaches have been raised but literature is not yet in agreement about the effectiveness of quizzes to enhance learning [START_REF] Siew | Flexible on-line assessment and feedback for teaching linear algebra[END_REF][START_REF] Varsavsky | Can online weekly quizzes contribute to learning in mathematics[END_REF][START_REF] Myers | Assessing assessment: The effects of two exam formats on course achievement and evaluation[END_REF][START_REF] Blanco | Computer assisted assessment through moodle quizzes for calculus in an engineering undergraduate course[END_REF][START_REF] Lim | Fine tuning the teaching methods used for second year university mathematics[END_REF][START_REF] Broughton | Lecturers' perspectives on the use of a mathematics-based computer-aided assessment system[END_REF][START_REF] Shorter | Comparing assessment methods as predictors of student learning in an undergrad. mathematics course[END_REF].

CONTEXT

This research took place in two mathematics' courses to Engineering students of Instituto Superior de Engenharia de Lisboa, Portugal, each during a semester. In those semesters, weekly online quizzes on Moodle (the learning management system of the institute) were made available for a week each. The AM2 course in 2013/14 was about Multivariable Calculus, the MAE course in 2015/16 was about Single Variable Calculus. Around 100 students and 3 teachers were involved in each course.

The quizzes were called 'Mini-tests' to reinforce their relevance. The 'regular' assessment involved two face-to-face tests or the First Exam or the Second Exam. For AM2, the quizzes scored up to two values proportional to the best 12 (out of 14) grades in the quizzes and it was added if the student scored more than 9.0 values (out of 20) in 'regular' assessment. For MAE, it was slightly different: the quizzes valued 10% of the grade if the student scored more than 9.0 values (out of 20) in the 'regular' assessment and if this grade was better than the 'regular' grade. In both cases the quizzes were optional.

The aim of the quizzes was not to assess students, it was to make them study more, not to postpone, not to study first the other subjects that were naturally more pleasant for them (since they belong to their study area); to make students more aware of their level of understanding (often students only realise that they cannot solve the exercises when they go to the first test, in the middle of the semester). Students are usually optimistic about their capabilities [START_REF] Wandel | Students' mathematical preparation: Differences in staff and student perceptions[END_REF]. It was written in Moodle and teachers repeatedly reminded students that the aim of the quizzes was to make students study more and be aware of their level of understanding; that students could copy all quizzes but, probably would not get the 9.0 values required in 'regular' assessment and therefore, it not be worthwhile.

THE QUIZZES

The quizzes were produced through the 'Moodle activity: test'. It allows the introduction of images and mathematical symbols using LaTeX (see Fig. 1). The possibility of creating questions with different instances for each student was considered, but it would take much more time to create questions and students also know how to solve a problem with a constant instead of a number, so it did not seem worthwhile. Whenever it was possible, we used numeric or short answers instead of multiplechoice answers since in multiple-choice answers, with a few tries, students could get the correct answer. The type of questions that we most used was 'embedded answers', because this enables a teacher to embed more than one sub-question and those sub-questions may be chosen from all the different question types: numeric, short answers, multiple-choice, true or false, etc. The 'embedded answer' question type allows the teacher to evaluate the student through their pathway and not only their final result (see Fig. 3). The feedback does not show the correct answer.

RESEARCH DESIGN, DATA AND RESULTS

This research design is a quasi-experience (not an experience since not all variables could had been controlled) where two sets of quizzes were applied to two mathematics courses. The research question of this study is: are the quizzes (applied with this strategy) a fair and effective tool to increase students' learning? The strategy for application of the quizzes is that they are weekly, online, non-mandatory, count towards grades if students achieve a certain level on traditional assessment, are not randomly generated and students may resubmit without penalty. This research question was split into four sub-questions:

• RQ1: Did the students adhere to the quizzes? • RQ2: What was students' perception of the quizzes? • RQ3: Did students felt quizzes as unfair? • RQ4: Did the quizzes increase students' grades?

The instruments utilized were: a students' survey about the quizzes; data from the answers to the quizzes; and course grades over several semesters. The quizzes were applied to two mathematics courses: AM2 with 104 subscribed students and MAE with 108.

The anonymous survey on Moodle was addressed to all students for each edition. The sample of students who answered the survey was reasonable. From the 104 students subscribed to AM2, all subscribed to Moodle, 65 answered the survey. From the 108 students subscribed to MAE, 94 in Moodle, 61 answered the survey. Moreover, by splitting the students by their grade at the first test (the survey was applied before the second test), the number of students answering the survey with a given grade reasonably correlates to the number of students in general who achieved that grade. Pearson correlation coefficients are ρ = 0.6 and ρ = 0.5 respectively.

Students of the institute do not have precedencies among courses and may be subscribed to a large number of courses, so it is usual that students subscribe to many courses where, in fact, they do not attempt to achieve success. We may verify this, for example, by noticing that from the 108 students subscribed to MAE only 94 were subscribed to Moodle, so the 14 remaining students did not access anything from the course: syllabus, slides, quizzes etc. Since there is no simple and fair way of identifying these students, in this research we always use the subscribed students to make measures. However, it is relevant to have in mind that it includes those 'ghost students'.

RQ1: DID THE STUDENTS ADHERE TO THE QUIZZES?

AM2 had 104 subscribed students, 79 attempted regular assessments and 76 students attempted at least one quiz. All but one of the approved students answered at least one quiz. The final quiz grade was the average of the best 10 out of 14 grades in quizzes, so it was natural that the last four quizzes had lower attendance (and for this reason we modified this rule for MAE, where the best 12 grades were chosen).

Chart 1. The number of students that answered AM2 quizzes split by grade.

MAE had 108 subscribed students, 103 completed regular assessment and 93 students attempted at least one quiz. All approved students answered at least one quiz. The final quiz grades were the average of the best 12 out of 14 grades in quizzes, so it is natural that the last two quizzes had a lower attendance (this rule changed from AM2). is important to note that, for example, in Q5 the number of students with a total grade was lower than in the other quizzes and the number of attempts to solve the quiz was higher than in the others (326). This shows that students were, in fact, trying to reach the correct answers (this test was particularly large and complex).

Chart 2. The number of students that answered MAE quizzes split by grade. The number of attempts to answer the quiz, registered by Moodle, is in parenthesis.

A large portion of students got a very high grade, but this was natural since students may retry without penalty and the questions were equal to all students, so it was expected that students talk to each other and reach the correct answer.

The quizzes were not mandatory and improved the grade if the student got more than 9 out of 20 values in regular assessment, so it could be expected that many students decided not to take it. However, on a regular basis, nearly half of the subscribed students answered the quizzes.

An objective result was, despite of the optional policy, that students strongly adhered to quizzes. The percentage of subscribed students that answered one quiz was 93/108=86% and 76/104=73%. All the quizzes had a high rate of attendance. Among the students that undertook 'regular' assessment, almost all took a quiz and a large percentage got high average grades on the quizzes.

RQ2: WHAT WAS STUDENTS' PERCEPTION OF THE QUIZZES?

Table 1 shows that, according to the survey, none of the students thought that the quizzes were of no interest and did not care about the quizzes, while a large percentage believed that the quizzes reminded them to study, showed them the level that they were reaching and encouraged them to learn new things; some of those things they thought they understood but in fact they did not. Summarising, more than 90% of students found quizzes useful (Chart 3); that they study more due to the quizzes. Students agree that quizzes remind them to study, show them that there were things that they thought that had understood but did not, encouraged them to learn new things and gave them a better perception of level that they were reaching.

RQ3: DID STUDENTS FELT QUIZZES AS UNFAIR?

In daily life as a teacher, teachers tell several times that one reason why they do not use online quizzes is because students may be cheating and it may generate unfairness. To avoid that problem, it was strongly emphasised to students that quizzes were much more relevant as formative assessments than summative assessments; students could resubmit the quiz without penalty to stimulate them to try to answer by themselves without fear of being penalised; and a clause was included that the quizzes only count towards grades if students get 9.0 values (out of 20) in regular assessments, as in [START_REF] Varsavsky | Can online weekly quizzes contribute to learning in mathematics[END_REF]. As result, the answers in the survey to the question 'Quizzes generate unfairness?' show that very few students perceive quizzes as unfair (see Chart 5). When questioned in the survey, no student stated that they had copied the results (see Chart 4), despite it being reinforced in that question that the survey was automatically anonymous.

Therefore, with this approach, the level of unfairness of quizzes is not considered as relevant.

RQ4: DID THE QUIZZES INCREASE STUDENTS' GRADES?

Since the goal was that all students achieve a total score in all quizzes, is was expected that quiz grades would not correlate to final grades. This did occur and it was verified using the non-parametric Spearman Rho for AM2 ( ρ = 0.34, N = 54, p = 0.01) and for MAE ( ρ = 0.28, N = 61, p = 0.03), since data were not normal (Kolmogorov-Smirnov, p < 0.01).

According to Chart 6, around 70% of students that answered the survey, believe that quizzes helped them achieve a higher grade.

Chart 6. Percentage of students' answers to 'Without quizzes, I've scored…' in both surveys.

The data of Tables 2 and3, relate to six responsible teachers/approaches and ten different teachers. The syllabus was essentially the same across the semesters but the approaches were naturally different. In the intervention semesters, the responsible teachers were also different. So, the quizzes were not the only different variable in that semester, thus we cannot attribute grade differences directly to the quizzes. For AM2, the pass rate nearly doubled in that semester, the average grade also increased significantly. The MAE course had, in some editions, five or six quizzes in class. It is curious to note that in the year that there were no quizzes, the pass rate was much lower. And the MAE pass grade and the average grade had the highest value in the experimental semester. However, it may have been a coincidence, we do not have enough data to reach any conclusions, it is just a positive indication. Summarising, as expected, quiz grades do not correlate to final grades; around 70% of respondents to the survey state that due to the quizzes they achieved a better grade.

The pass rate and the average grade increased significantly in the semesters that the quizzes were applied, which is a positive indicator but cannot be directly attributed to quizzes.

CONCLUSIONS

Two sets of 14 weekly quizzes on Moodle were available to all the engineering students on two mathematics courses (Single and Multivariable Calculus). The online quizzes were not mandatory, counted to grading if the student had more than 9 out of 20 values on traditional assessments, were not randomly generated and students could resubmit without penalty. The research question is 'Are the quizzes (applied with this strategy) a fair and effective tool to increase students' learning?'

In the answers to the survey, more than 90% of students found quizzes useful; more than 60% stated that studied more due to the quizzes; students agreed that quizzes reminded them to study; showed them that there were things that they thought they understood but did not; made them learn new things and gave them a better perception of the level that they were reaching.

The quizzes were not mandatory so students may have just ignored them. Although a large proportion of students attempted quizzes and kept answering them until the last ones.

Quiz questions were not randomly generated, so all students got the same questions and naturally, students shared the solutions with each other. To avoid unfairness, it was strongly emphasised that quizzes were important to students' formative assessment, to allow them to test themselves and get feedback on their level of understanding. Moreover, quizzes only contributed to grades if the students got more than 9 out of 20 values in 'traditional' assessments. Moreover, if a student copied many quiz results, probably would not achieve the minimum grade and it would not be worthwhile. The result was that, in the answers to the surveys, very few students stated it as being unfair. Over 70% of respondents to the surveys stated that due to the quizzes they achieved a better grade. The pass rate and the average grade increased significantly in the semesters that the quizzes were applied, which is a positive indicator, but it cannot be directly attributed to the quizzes.

This research suggests that these quizzes, with this strategy, are a fair and useful tool to increase students learning.

Figure 1 .

 1 Figure 1. Multiple-choice questions including a figure and mathematical text, MAE and AM2 example. (Translated)

Figure 2 .

 2 Figure 2. A question with multiple embedded questions along the path (including numerical answers), an AM2 and MAE example. (Translated)

Chart 4 .

 4 Percentage of students answers to 'How do you answer to quizzes?' in both surveys. Chart 5. Percentage of students answers to 'Quizzes generate unfairness?' in both surveys.

Table 1 . Students' answers to 'Select ALL the statements that you agree with' in both surveys. Chart 3. Percentage of students answers to 'The quizzes were…' in both surveys.

 1

	AM2	MAE

Table 2 . Grades of AM2 students across ten semesters, the letter representing the coordinator teacher is underlined and the experimental semester is shaded.

 2

	AM2	2010/11	2011/12	2012/13	2013/14	2014/15	2015/16
		S1	S2	S1	S2	S1	S2	S1	S2	S1	S2	S1	S2
	Subscribed students	101	200	128	153	90	123	80	104	56	66	56	108
	Approv. students	27	38	31	41	20	23	12	54	10	19	16	33
	Average appr. grade	11.7	11.8	12.3	11.7				13.9	12.4	11.5	11.7	11.5
	Pass/Subscribed	27%	19%	24%	27%	22%	19%	15%	52%	18%	29%	29%	31%
									F+G			J+K+	J+K+
	Professors	A+…	A+…	A+…	A+…	A+B	A+C	D+E		D+F	D +I
									+H			IL	I

Table 3 . Grades of MAE students across five semesters, the letter representing the coordinator teacher is underlined and the experimental semester is shaded

 3

	MAE	2011/12-SI	2012/13-SI	2013/14-SI	2014/15-SI	2015/16-SI
	Subscribed students	73	109	121	125	108
	Pass students	17	30	58	56	61
	Average pass grade	12.7	12.2	13.5	12.7	13.5
	Pass/Subscribed	23%	28%	48%	45%	56%
	Number of quizzes	0	6 in class	5 in class	5 in class	14 online
	Professors	A	A	A	A+B	B+A


Yael Fleischmann
email: yael.fleischmann@math.upb.de

Rolf Biehler
email: biehler@math.upb.de

Students' problems in the identification of subspaces in Linear Algebra

Keywords: Linear algebra, vector space, subspace, proof, tutorial groups

The goal of the study presented in this paper is the investigation of students' problems with exercises concerning central topics of linear algebra courses at university level. We present the results of our analysis of students' work on an exercise about subspaces of . We evaluated the written solutions of the task as well as transcripts based on videos taken of student groups working on the problem. We identified and classified descriptions of vector spaces and subspaces that varied widely and demonstrated highly different skills in working with geometric or formal algebraic objects. We analyzed how far students could progress in a complex reasoning process, and identified those steps in the reasoning process on which students needed support to continue.

PURPOSE AND BACKGROUND

Problems in teaching and learning of linear algebra have a long history in many countries [START_REF] Dorier | Research into the teaching and learning of linear algebra[END_REF]. Frequently, the abstract character and the formalism of mathematics that students have not been exposed to in school before is named as a central obstacle (a variety of studies are outlined and evaluated in [START_REF] Dorier | The Obstacle of Formalism in Linear Algebra[END_REF]. Since vector spaces are a central part and moreover of special importance for almost all disciplines related to mathematics at university, special attention has been paid to them [START_REF] Dorier | Epistemological Analysis of the Genesis of the Theory of Vector Spaces[END_REF][START_REF] Stewart | School Algebra to Linear Algebra: Advancing Through the Worlds of Mathematical Thinking[END_REF]. Generally, students do not develop a clear concept of vectors at school level [START_REF] Mai | A vector is a line segment between two points? -Students' concept definitions of a vector during the transition from school to university[END_REF], and the more abstract approach to this subject taught at universities is described as being "out of reach" by some students [START_REF] Stewart | School Algebra to Linear Algebra: Advancing Through the Worlds of Mathematical Thinking[END_REF]. [START_REF] Wawro | Subspace in linear algebra: investigating students' concept images and interactions with the formal definition[END_REF] investigated concept images of subspaces in interviews with students and identified recurring concept images, distinguishing between a subspace as a part of a whole, a geometric object, and an algebraic object. The introduction of first concepts in tutorial meetings in linear algebra, with a special focus on the behavior and influence by the tutor, has been studied by [START_REF] Grenier-Boley | Some issues about the introduction of first concepts in linear algebra during tutorial sessions at the beginning of university[END_REF].

CONTEXT AND DESIGN OF THE STUDY

In this study, we investigated the problems of students shortly after their first encounter with vector spaces and subspaces at university level. The participants of our study were students with major mathematics or computer science, enrolled for bachelor of science or bachelor of education (for secondary school, "Gymnasium"), most of them in their first semester. In our study, we collected data from students working on tasks in groups during their tutorial group meet-ings (1.5 hours), where the tutors were advised to answer questions but to only intervene when the students had substantial problems to continue. The students worked on exercises about the content of a recent lecture under the supervision of a tutor. In this context, we assigned special tasks that we developed ourselves together with the lecturer and his assistant, but we did not influence the course design otherwise. We will report only on one of them in this paper. The course can be considered to be typical for a beginners' lecture in linear algebra, which normally is rather abstract, and was given by an experienced lecturer. During their tutor meetings, the students worked on our exercises on separate sheets that we collected, scanned for later analysis and gave back to the students in the next meeting without any grading or corrections. We gathered between 78 and 130 written works on each exercise. Moreover, we also took video recordings of groups of 2-4 volunteering students working on these exercises. They worked on the exercise under the supervision of a student tutor who was part of the research team and familiar with our a priori analysis of the task. The experienced tutor was advised to help the students if they struggle with the exercise in the same way as she would do in an ordinary tutor group meeting. We were interested in identifying important didactic variables. The results obtained by analyzing the first implementation of the exercise about vector spaces are currently being used for designing a second implementation in the course Linear Algebra I.

The task for students in our study and preliminary research questions

In this paper, we concentrate on an exercise about subspaces and vector spaces (see figure 1) that was part of the exercise sheet during week 7 of the course, immediately after the notion of subspaces had been introduced. Students are taught analytical geometry and linear algebra at school level, where vectors are introduced as tuples (or classes of arrows), but they do not as a rule have a clear concept of a vector (cf. [START_REF] Mai | A vector is a line segment between two points? -Students' concept definitions of a vector during the transition from school to university[END_REF]. Students know equations of planes and lines in and , without considering them as subspaces, because this notion is not taught at school level.

The following exercise was designed in order to provide two different kinds of learning potentials (as described in Gravesen, Grønbaek and Winsløw, 2016):

1. Linkage potential: In part a) to e), our intension was to motivate the students to activate their school knowledge concerning the description of geometric objects using equations; we hoped that they would recognize the sets as descriptions of lines, points, parabolas etc., and connect this knowledge with the new concepts of vector spaces and subspaces. 2. Research potential: Part f) of the exercise was created in order to engage the students in a research-like activity. Even if achieving a complete solution seemed unrealistic for most of them, we were interested in how the students would approach this open question. They had to formulate a hypothesis and use abstraction to identify and construct subspaces. The exercise can be seen as a "mini research project" that differs in type from standard exercises. The parts a) to e) can be solved by a formal check whether the properties of subspaces are satisfied by the provided sets. As this was learned in the previous lecture, this is a standard task. Geometric ideas are not necessary, but we hoped that students may do geometric interpretations of the sets to develop a geometric meaning of subspaces and non-subspaces of Task f) is different, because this is the first time that this type of question is asked. Students may use the results from a) to e), that have provided examples and counterexamples of subspaces, to find the zero space, all lines through the origin, and the whole as subspaces and give reasons why they are subspaces on some level. The challenging question is whether or why these are all subspaces of . Research questions concerning f) are: How many students identify the zero space and the entire as subspaces? Are all lines through the origin identified as subspaces? Which arguments do students provide for considering a line through the origin as a subspace of ? How do they reason, when exploring, whether there are more subspaces in or whether they have already found all?

We were also interested in the sources of knowledge students used, such as their results on a) to e), parts of the lecture, or geometric interpretations related to their school knowledge, and concerning the videographed tutorial sessions, which kind of support by the tutor they can use in their reasoning process.

METHODOLOGY AND DATA COLLECTION

For the analysis of the written work of the students, we followed the method of [START_REF] Biehler | Conceptualizing and studying students' processes of solving typical problems in introductory engineering courses requiring mathematical competences[END_REF], by comparing each solution with the so-called student expert solution (SES), which is a sample solution based on the idealized actual knowledge of the students at this point of the lecture. Moreover, the student expert solution contains additional meta-information about the solution, for example, several alternative opportunities for solutions and explicitly written-out learning objectives. In relation to Brousseau's theory of didactic situations, this method can be seen as a special approach to the development of an a priori analysis. We evaluated the written work in a two-step procedure: In a first step, we categorized the solutions by correctness and collected peculiarities and mistakes. Based on this and the SES, we developed a detailed coding system for deeper analysis. The recorded videos have been transcribed in order to allow a detailed qualitative analysis.

A PRIORI ANALYSIS OF THE TASK

In the lecture, the definition of vector spaces was given in a typical traditional, abstract way. The zero space and the vector spaces (trivial vector space over ) and ℂ (the latter together with component-wise addition and multiplication) had been presented by the lecturer as first examples. Apart from this introduction, the students had only seen the following (relatively abstract) non-trivial examples for vector spaces in the lecture: (VS1):

, the "standard vector space", where is any field, with n , including the definition of addition and the scalar multiplication (component-wise), (VS2):

, the vector space of sequences over the field , with the component-wise operations. Subsequently, subspaces of vector spaces had been defined to be subsets of vectors spaces that are vector spaces themselves with respect to the same operations. Following this, they had learned that a sufficient criterion for proving that a nonempty subset of a vector space over the field is a subspace is to prove that ly and secondly . As examples of subspaces, the trivial subspaces and were nominated without proof. Moreover, for both vector spaces (VS1) and (VS2), there was an abstract example for a subspace given, and we state the first one of them here since it will be of use for our later analysis: (S1) The set is the solution space of a homogeneous linear equation system . It was shown that this set is indeed closed with respect to addition and scalar multiplication and is a subspace. Note that this example can be applied to , if we choose . The subspaces in are the lines through the origin expressed by linear equations. This interpretation could be done by students on the basis of school knowledge. The lecturer did not provide this specialization himself.

For our later analysis, the following distinction is central. All provided examples have in common that sets are characterized by equations (subspaces defined by relations). In contrast, 1-dimensional subspaces could also be defined by explicit construction: for instance for any :

. The latter way of defining subspaces was not yet a topic of the lecture, which will turn out to be an obstacle for some students. Moreover, the students had not seen any geometric interpretation or visualizations of vector spaces or subspaces, in particular no (concrete) examples of subspaces in

In the following, we will give an overview about possible approaches and steps to part f).

Step 1: Find some subspaces. With the knowledge from the lecture, the trivial subspaces (the zero space and can be named. To find nontrivial subspaces, one can identify again the set of the previous part a) of the exercise as a subspace. Starting with this set, one could generalize from numbers (like 2 and 1, as chosen in part a)) to a general form with coefficients, and give the set with not both being zero. Instead, if one abstracts from the mathematical language used in the exercise before, these sets could also be expressed constructively as Supported by [START_REF] Dorier | The Obstacle of Formalism in Linear Algebra[END_REF], we expected difficulties to translate the relational representation into the constructive representation and vice versa. Alternatively, with the knowledge from the lecture, one could apply the example (S1) given in the lecture to the space , and describe the subspaces in terms of the solutions of homogeneous linear equation systems. This reasoning can be done just algebraically. It could also happen that students use geometrical terminology concerning lines through the origin.

Step 2: Verification of the subspace properties. In order to reason why the subsets given in step 1 are subspaces, one could either refer to the solution of part a) of the exercise or (for the trivial subspaces and in case of the use of the solution spaces of homogeneous linear equation systems) to the lecture. In case of a geometric description ("lines through the origin"), either geometric or algebraic arguments have to be provided to verify the subspace properties.

Step 3: Why are these all subspaces? The final challenge is to reason if and why all subspaces of have been found. This can be done algebraically, but we did not expect our students to complete this reasoning process in the given time, since it requires a development of several successive algebraic arguments. Based on their school knowledge, the students could recognize the descriptions of geometric objects by equations in part a) to e) and abstract from the previous results, leading to the conclusion that lines through the origin are subspaces, but no other lines, single points or other collections of points. At this point, a successful reasoning based on school knowledge could be done constructively, based on geometric arguments. Trying to construct "bigger" subspaces than just the lines through the origin, a student could build the union of two different lines and check whether this set is a subspace. Alternatively, he or she could try to find the minimal subspace that includes one line through the origin and an additional point not lying on this line. He or she could come to the conclusion that this has to be the whole . A formal argumentation here is that every point can be represented as a linear combination of a point from the line and , but even if the student does not come to this conclusion at this point, he or she could have the idea to consider the line through the new point together with the original line, and therefore check this new set for the subspace conditions. He or she could check the closure of addition or come to the idea that further points have to be added to the union in order to get a subspace. Since this type of reasoning seemed to us more likely to be achieved with the previous (including school) knowledge of the students, the tutors in the normal tutor group meetings as well as the tutor in the video study were advised to guide the students along this reasoning process if they struggled in approaching the problem. Based on this sample solution process description, we tried to answer the following research questions in our analysis:

1. How far in this three-step process would the students come when they work on this exercise? Would they even be aware of the need to do step 2 and 3? 2. Would they favor one of the described approaches to the problem (geometric, algebraic), and would they use the constructive or the relational way to describe the 1-dimensional subspaces? Would they approach step 3 in a constructive way, building up subspaces starting with just one point, as described above, or would they find other ways (purely algebraic?)? 3. Finally: Would they recognize that parts of exercise f) could be solved by an application of the example (S1) given in the lecture?

Since we posed the question in part f) in a relatively weak phrasing, we could not expect the majority of students to give a fully structured, formal reasoning in this exercise, in particular for the steps 2 and 3. But we were interested if the exercise itself would stimulate the students to give reasons for their answers and, in particular, how they would argue in this case.

RESULTS

To find answers to our questions, we analyzed the written works as well as the video recordings of the students working on part f).

Work on part f): Written exercises

From the written works of 116 students on this exercise, just 48 handed in solutions for part f). This is most likely due to the fact that the time was very limited, so many students just did not come to part f). We analyzed their work with respect to the three steps of the solution as described in the a priori analysis.

Table 1: Frequency of the nominations and descriptions of the subspaces

Step 1: Which subspaces do they find? How do they describe them? Do they use previous parts of the exercise or name the set considered in part a)?

The results in table 1 were collected by counting how often the three types of subspaces were mentioned in the solutions. Hereby, each notion of a subspace Trivial subspaces 33 Zero Space 32

1dimensional subspaces

Solution with any description of the 1-dimensional subspaces (some students used more than one description)

33

-Relational description: 24

-Constructive description: 4

-Geometric descriptions: "line containing zero", "line through origin" counted, as long as it was clear enough to denote the required set. How did the students describe the 1-dimensional subspaces? We distinguished between "geometric" descriptions, using expressions like "line containing zero", "line through the origin", relational descriptions using a set like or something mathematically equivalent (see a priori analysis for a definition of this category) or constructive descriptions like . Some students used more than one description in their solution. Apart from this, it was interesting to see that only 8 students did mention any part (mostly a)) of the previous exercise in part f). It is not clear if those who could not give any (nontrivial) subspace actually never recognized that the set M 1 from part a) is a subspace (since the word "subspace" was not used in part a)), or if they just forgot about it before they started with part f). Moreover, it is interesting that the trivial subspaces, which we expected the easiest to find, were not nominated more often than the 1-dimensional subspaces. We were also surprised to see that only 2 of the students did refer to example (S1) (see a priori analysis) from the lecture, concerning the solution spaces of homogenous systems of linear equations.

No reasoning

Incorrect reasoning/ unclear approach Partial reasoning

Complete reasoning

Step 2 34 5 7 2

Step 3 35 6 6 1

Table 2: Frequency of reasoning in part f)

Step 2: Do they show that the given sets are subspaces? How do they argue?

Most students did not give reasons (see table 2 for results), but within those who did, we distinguished between approaches that did not go in the right direction (for instance students just answered by listing all properties of a subspace without proving them or claimed that it was "clear" that the spaces are subspaces), students who did give a correct approach or a partial proof (they mentioned that closure must be proved, but did not, or just checked the addition or the scalar multiplication, or just checked an example etc.) and complete solutions with full reasoning (using example (S1) from the lecture in both cases).

Step 3: How do they reason that they found all subspaces of ?

Within the 13 solutions that had some kind of reasoning (see table 2 for results), we distinguished again between unclear or vague approaches to reason the completeness of the given list of subspaces (for instance the statement "there are no other possible, because one cannot multiply vectors"), promising but incomplete approaches (some students gave reasons why lines not going through the origin cannot be subspaces, but did not consider other subsets, or just discussed the closure of one of the operations) and complete reasoning (just one case, again applying example (S1) from the lecture).

Work on part f): Video recordings

We give a summary about three groups of students that we recorded during their solution process on part f) under the perspective of our research questions. Due to limitations of space, we cannot document the method used to analyze the transcripts and the students' interaction in more detail.

Group 1: The first group tried to find subspaces by systematically going through the list of properties, and found the zero space to fulfill them. Then, they remembered the set proven to be a vector space in part a) and generalized it to a set of the form after a discussion whether the coefficients are arbitrarily exchangeable without harming the subspace conditions. They discussed the closure of the vector space operations in this set, but referring to the proof they had given in part a), they convinced themselves quickly that there was nothing else to prove. After this, they also identified the full space since there is no claim for a subspace to be a proper subset. At this point, they were asked by the tutor if and why they found all subspaces now. They had the idea to consider the set and, referring to their knowledge about groups, discussed the closure of operations on this set before they could finally rule it out to be a subspace by the fact that the scalar multiplication with elements from is not closed on . The tutor then asked them to consider the set geometrically. They start to consider the tuples of coefficients in the plane instead of the equation . With another hint from the tutor, they found out that the set whose elements are described by the equation denotes lines in the plane, and discussed the closure of the operations for these lines. The students did not develop an idea themselves to give arguments why they had found all subspaces. However, the students were able to follow the geometric constructive reasoning of the tutor (see a priori analysis).

Group 2: The second group came up with the idea to apply example (S1) from the lecture. After some discussion and a bit help from the tutor, they found that the subspaces defined there are the solutions of one or more linear equations, each having two coefficients. The central difficulty for them was to see that the number of coefficients is fixed to 2, but there could be an arbitrary number of equations in a system of linear equations that is still defined in It was a real discovery later that provided descriptions as are provided in . Up to this point, they did not consider the trivial subspaces at all. They struggled a bit to write down the concrete subspaces they could find this way in terms of algebraic expressions, but managed it with some help from the tutor. Asked whether they found all subspaces, they did not develop the idea to consider the spaces as lines in the plane on their own, but after the tutor came up with this idea, they were able to work with this concept after a short phase of orientation in which they convinced themselves that the geometric objects stand for the same subspaces they worked with before. Just at this point, they identified the trivial sub-spaces too. Step 3 (to reason that all subspaces were found) was only solved with tutorial support (similar to group 1).

Group 3: The third group used the previous parts a) to e) in their reasoning and started with the subspace found in a), but immediately identified this set to describe a "line through the origin", which gave birth to a generalization to all lines through the origin. They continued to orally communicate in geometric terms, but decided to write down the set using the relational algebraic expression . They named the trivial subspaces without further comments. The proof given in part a) sufficed for them for a reasoning of step 2, and they started to discuss step 3 quickly. They used references to part b) to e) to rule out other types of possible subspaces. The group thought they had finished at this point. It was the tutor who pointed out that step 3 was not yet satisfactorily answered. Different from the other groups, they took up the tutors input to construct other subspaces geometrically and in order to find out that such subspaces have to be equal to . With some minor help from the tutor, they finished this step quickly, needing much less time for the full task than all other groups.

CONCLUSION AND DISCUSSION

As a result, we can state that students at this point in their studies were able to find and describe (using varying descriptions) subspaces of , but the question to find all subspaces was a serious obstacle for the students. Moreover, the step to translate the algebraic description of the subspaces into a geometric view, where reasoning could be done with less formality, was a further obstacle for them, since they seemed not to connect or apply their geometric knowledge from school to the new problem.

It seems like a geometric approach to this kind of problems is not a natural, automatic behavior of students at this point of their education. This result resonates with the observations from [START_REF] Wawro | Subspace in linear algebra: investigating students' concept images and interactions with the formal definition[END_REF], who stated that intuitive geometric notions can be the preferred approach of first year students to the concepts of subspaces, but also cause problems if they their geometric intuitions are inconsistent with the formal definition. It is worth pointing out that our students did not, in opposition to the results of Wawro et al., automatically identify (often mistakenly, if there was no respect to a necessary embedding) the as a subspace of the . A possible explanation for this result is the fact that our results were obtained shortly after the introduction of subspaces in the lecture, where Wawro et al. interviewed their students when they already have had more time to develop a concept image of subspaces, including some misconceptions.

Most students did not connect the different parts of the exercise, appearing in different "languages" (like the sets in the parts a) to e) and the open question in part f)) to solve the problem in f). With some help, especially with the request to consider the sets geometrically, they were able at least to understand reasoning on this basis, and some students actually could even give proofs or approaches to proofs on their own. We came to the result that the students needed more guidance and preparation to solve this problem, and in particular support that helps them to deal with each step and even sub-step of the solution of the problem in part f). In our subsequent study in winter term 2017/2018, we are investigating if explicit indications in a) to e) to consider the sets geometrically and a rephrasing of part f), splitting it up into more explicitly described steps, have a decisive influence on the students' ability to solve task f).

Figure 1 :

 1 Figure 1: Exercise on subspaces (translated from German)


Kerstin Pettersson
email: kerstin.pettersson@mnd.su.se

Niclas Larson

Keywords: Teachers' and students' practices at university level, teaching and learning of specific topics in university mathematics, communication, small-group teaching

In this study, the learning encouraged by teaching activities in a small-group setting was investigated through the analysis of students' responses to survey and interview questions. The results indicate that the students perceived an increased ability to communicate mathematics in written form, but to a lesser extent developed their ability to discuss mathematics and build conceptual understanding.

Empirical research on mathematics teaching at tertiary level is an area of sustained growth and interest [START_REF] Biza | Research on teaching and learning mathematics at the tertiary level: State-of-the-art and looking ahead[END_REF]. Besides studies on the discourse of mathematics teaching in lectures (e.g. [START_REF] Viirman | Explanation, motivation and question posing routines in university mathematics teachers' pedagogical discourse: A commognitive analysis[END_REF], there are studies focusing on small-group tutorial sessions (e.g. [START_REF] Mali | Characteristics of university mathematics teaching: Use of generic examples in tutoring[END_REF]. Inquiry-based small-group activities were found to have reasonable effect in a project aiming to improve the conceptual understanding of engineering students [START_REF] Jaworski | An activity theory analysis of teaching goals versus student epistemological positions[END_REF]. However, there is still a need for research on the use of small-group teaching as a complement to lectures across different national and institutional practices. In the study presented here, we analysed students' experiences of learning from teaching activities in a small-group setting at a large Swedish university. Our research question is: How do the students perceive their mathematical abilities to have developed through the small-group seminars?

In the first-semester mathematics course, besides whole-class lectures, students once a week took part in so-called seminars with 10-15 students in each group. The stated aim of the seminars was "to learn to discuss mathematics, and to present mathematics orally and in written form", but also to "support the learning of algebra and calculus". Each seminar group was led by a lecturer. Before each seminar, students handed in a written solution to one or two tasks. During the seminars, they presented the solutions orally and received comments from the seminar leader on both their oral and their written presentations. Furthermore, the students discussed conceptual tasks given by the seminar leader or discussed questions posed by the students themselves.

A survey was conducted in a lecture after 10 of the 13 seminars had been held. Of the 49 students present, 42 completed the questionnaire. Students were asked to indicate on a Likert scale the extent to which the seminars had facilitated their learning of presenting and discussing mathematics, solving problems, and understanding concepts.

Seven students volunteered for individual interviews undertaken three weeks after the survey. The focus was on students' experiences from the seminars and questions were posed about their learning. In the analysis, we searched for utterances where students' perceived learning was pronounced. These utterances were then grouped together in categories due to the abilities mentioned. This was a deductive analysis drawing on the notion of competencies (e.g. Niss & Jensen, 2002). For each category we then interpreted how the students perceived that their mathematical abilities had developed.

The analysis of data is ongoing. However, the preliminary analyses indicate that students perceived that they developed their ability to communicate mathematics in written form, but to a lesser extent developed their ability to solve problems and discuss mathematics. In the survey, students marked high on the Likert scale for learning to make written presentations (mean 3.5, scale 1-4 with 4 as 'a lot'), a little lower on solving problems (3.1), understanding concepts (3.0), oral presentations (2.9) and discussions (2.9).

The categories revealed by the interviews were: understanding of concepts, communication, solving problems, procedural knowledge, and reasoning. Analyses of the interviews confirmed the results from the survey; the perceived learning from the seminars largely concerned the written presentations. While some students talked about their aim to build conceptual understanding, they mentioned also how the focus on details and technical aspects in the written communication distracted them from this aim, a finding in line with the distracting role of aspects such as examination forms found by [START_REF] Jaworski | An activity theory analysis of teaching goals versus student epistemological positions[END_REF]. In the interviews, we also met descriptions of different ways in which seminar leaders had chosen to use representations and problem solving (cf. [START_REF] Mali | Characteristics of university mathematics teaching: Use of generic examples in tutoring[END_REF]. Further data collection is planned to answer questions on the role of these characteristics and the seminar leaders' mathematical discourse (cf. [START_REF] Viirman | Explanation, motivation and question posing routines in university mathematics teachers' pedagogical discourse: A commognitive analysis[END_REF].

Seminar groups as part of first-semester mathematics teaching:

What did the students learn?


Zeger-Jan Kock
email: z.kock@fontys.nl

Birgit Pepin²

Student use of resources in Calculus and Linear Algebra

Keywords: Student use of resources, Case study, Transition from school to university, Calculus, Linear Algebra

In this study we have investigated the resources used by first year engineering students in a technical university in the Netherlands, for their learning of Calculus and Linear Algebra. Using a case study approach we have focused on how the resources and their use (a) differed from upper secondary school as compared to university, and (b) differed between the two university courses. The results indicate that, in terms of (a) students built on secondary school experiences and emulated these into their university courses, where some subsequently experienced difficulties. In terms of (b), we argue that the course organization and the alignment of curriculum materials with the learning goals had an impact on the students' choice and use of resources. Human resources played an important but varying role.

INTRODUCTION

At university level a large diversity of resources is currently made available for students learning mathematics. These include traditional curriculum resources (e.g. readers, textbooks); digital (curriculum) resources (e.g. YouTube, websites, apps); and also human resources (e.g. drop-in clinics run by tutors; setups for peer groups). The ways in which university mathematics teachers interact with various resources has been investigated by [START_REF] Gueudet | University teachers' resources systems and documents[END_REF], for example, and several studies have been conducted related to university students and their use of particular resources to learn mathematics [START_REF] Anastasakis | Links between students' goals and their choice of educational resources in undergraduate mathematics[END_REF][START_REF] Biza | Research on teaching and learning mathematics at the tertiary level: State-of-the-art and looking ahead ICME-13[END_REF][START_REF] Inglis | Individual differences in students' use of optional learning resources[END_REF]. However, relatively little is known about how students of mathematics in their first year of university "cope with" the plethora of available resources available to them, and how they organise and coordinate them for their learning.

In this paper, first, we briefly outline selected insights from the relevant literature, including our theoretical frame of "resources" (and their use) as a lens to develop a better understanding of students' mathematics learning. Second, we describe our chosen methodology and data collection strategies, before we present our findings and discuss our results in the third section. Fourth, we present our conclusions and outline implications for the practice of university mathematics learning and teaching.

THEORETICAL FRAMEWORKS Transition from secondary to tertiary education

In terms of mathematics learning, the transition from secondary school to university is challenging for many students [START_REF] Pepin | Re-sourcing curriculum materials: in search of appropriate frameworks for researching the enacted mathematics curriculum[END_REF], as discontinuities exist between secondary and tertiary mathematics education. The literature reports numerous differences between studying mathematics at school as compared to university. It is said that in comparison to secondary education, at university: (a) the mathematical content is introduced at a higher speed; (b) more mathematical autonomy is expected; (c) the levels of generalization and abstraction are higher; (d) the approach is more formal with an increased emphasis on proof; and (e) the institutional cultures at the two institutions (secondary school, university) are different [START_REF] Artigue | Mathematics education research at university level: Achievements and challenges[END_REF][START_REF] Gueudet | Investigating the secondary-tertiary transition[END_REF]. The ways the content is made available to students also differ between secondary and tertiary education (e.g. [START_REF] Corriveau | The secondary-tertiary transition viewed as a change in mathematical cultures: An exploration concerning symbolism and its use[END_REF]. University students have to autonomously manage the various resources to learn mathematics, and it is argued that secondary school does not prepare them well for this task [START_REF] Williams | Mathematics learning, identity and educational practice: the transition into Higher Education TransMaths Research Briefing: The University of Manchester[END_REF]. Thus, it can be expected that first year university students have to find new ways of working with the resources they have access to, and that are proposed to them in their courses.

Student use of resources

The use of resources by students has been the subject of relatively little research. Selected studies (e.g. [START_REF] Anastasakis | Links between students' goals and their choice of educational resources in undergraduate mathematics[END_REF] indicate that students, in their selection of resources, have been predominantly motivated by the goal to be successful in examinations (and to obtain high grades). The authors of this study made an inventory of the resources used by students when studying for mathematics modules, and explicitly related these to their learning goals. The most widely used resources were those that the university provided for the students, and their own notes. The use of particular resources, for example mathematics textbooks, was specifically linked to the study of worked examples, which were said to help students to prepare for examinations; albeit this often lead to emphasise the surface aspects of the examples [START_REF] Biza | Research on teaching and learning mathematics at the tertiary level: State-of-the-art and looking ahead ICME-13[END_REF]. In their review study [START_REF] Biza | Research on teaching and learning mathematics at the tertiary level: State-of-the-art and looking ahead ICME-13[END_REF] identified several limitations of tertiary mathematics textbooks, in particular the emphasis on formal aspects of mathematics, at the cost of opportunities to develop intuitive meanings and understandings. Relating the use of particular resources to examination grades, a study by [START_REF] Inglis | Individual differences in students' use of optional learning resources[END_REF] found that students who attended lectures or used the university's mathematics support centres had higher grades than students who often watched online lectures. The authors suggest that students might need explicit guidance on how to combine the use of various resources into an effective learning strategy. Before this guidance can be given, or be reified in a blended learning environment, more in-depth information on the actual use of resources by students is needed.

The lens of resources

In this study we use the notion of "re-source/s" that students have access to and interact with in/for their learning. We assume that the ways university students learn mathematics is influenced/shaped by their use of the various resources at their disposal. By "use of resources" we denote, for example, which resources students choose (amongst the many on offer) and for what purpose (e.g. revision); the ways they align them (e.g. first lecture then checking the textbook, etc.); which ones seem central to achieve particular learning goals (e.g. for weekly course work, examinations, for their engineering topic area). However, we do not address the specific learning of CS and LA, that is how students interact with particular (e.g. cognitive) resources to learn particular topic areas in CS and/or in LA. Gueudet and Pepin (in press) have defined student resources as anything likely to resource ("to source again or differently") students' mathematical practice, leaning on [START_REF] Adler | Conceptualising resources as a theme for teacher education[END_REF] definition of mathematics "re-sources" (in Adler's case used by teachers). In this study we distinguish between (1) material resources, and (2) human resources. (1) For material resources a further distinction has been made between (a) curriculum resources (those resources proposed to students and aligned with the course curriculum), and general resources (which students might find/access randomly on the web). Curriculum resources are developed, proposed and used by teachers and students for the learning (and teaching) of the course mathematics, inside and outside the classroom [START_REF] Pepin | Curricular resources and textbooks[END_REF]. They can include text resources, such as textbooks, readers, websites and computer software, but also feedback on written work. General resources are the non-curricular material resources mobilized by students, such as general websites (e.g. Wikipedia, YouTube). ( 2) In terms of human resources we refer to formal or casual human interactions, such as conversations with friends, peers or tutors.

METHOD Context

The study took place at a university of technology in the Netherlands, with a student body of approximately 13000 engineering students. The university offers 15 bachelor courses related to technology and engineering.

We selected two first year courses in the first term of the 2016-2017 academic year: Calculus (CS); and Linear Algebra (LA). We purposefully chose these courses, as they were different in size and target group: the CS course was obligatory for all first year engineering students, approximately 2000 students, whereas the LA course was targeted at "applied mathematics and physics" engineering students only, approximately 130 students. The CS course was organized by the mathematics department, differentiated at three levels (A, B, and C), according to perceived level of difficulty and with varying level of emphasis on formal aspects of mathematics (e.g. proof).

In CS, six hours of lectures were organised each week, and one hour of tutorials in groups of eight students. In the course catalogue, and this was supported by lecturers, the aim of CS was to give engineering students a "basis" to be able to "calculate correctly". It appeared that the aims of the CS course were to provide students with a basic set of mathematical/computational tools they could subsequently use in their engineering studies and in their future work as engineers.

In LA, four hours of lectures were organised each week, and four hours of tutorials, in groups of approximately 30 students. As in CS, the LA learning aims were described as the acquisition of mathematical skills. Moreover, aims of the course were to help students develop the skills and realize the importance of correct mathematical communication, including writing formal proofs. Completing a mathematical writing assignment was part of the course requirements to reach this aim. It appeared that the purpose of LA was to prepare students for higher mathematics (used in the mathematics and physics courses).

Participants

In total, 24 students participated in the study: 18 CS students (involved in nine different engineering programs and all taking the B level CS course); 1 CS student who dropped out of university; 5 LA students (all studying for the 'applied mathematics' engineering course). In terms of background, of the interviewed CS students 15 came from secondary schools in the Netherlands, three came from other educational systems. For the Dutch students the CS content was partly familiar, in particular for those who took "strong mathematics" courses (Wiskunde D) at secondary school. Four of the five interviewed LA students came from secondary schools in the Netherlands, one student had attended secondary school in Belgium.

Data collection strategies

Data collection strategies included the following:

(1) Student interviews: The CS students were interviewed in four focus groups, and one individual interview. During the interviews students were asked to make a drawing of the resources they used for their mathematics course (Schematic Representation of Resource System, SRRS - [START_REF] Pepin | Developing a deeper understanding of mathematics teaching expertise: an examination of three Chinese mathematics teachers' resource systems as windows into their work and expertise[END_REF]. These helped the interviewer to understand the ways the resources were used, and for which purpose. The LA students were interviewed in two groups of two, and one individual interview.

(2) Documents/curriculum resources: Relevant curriculum materials and documents (digital and text materials) were collected and analyzed. These materials were provided by the university for the students (e.g. examples of examinations, LA syllabus, LA study guide, LA assignments, CS study guide, the CS textbook, course summaries in the university's course catalogue, video clips, videos of the lectures).

(3) Teacher interviews: Interviews with two CS lecturers and one CS tutor were conducted, as well as one LA lecturer and one LA tutor.

For analysis, the interviews were transcribed and interview quotations were coded using ATLAS-ti software. The codes were based on our knowledge from the literature concerning the different curriculum resources and their use. In the next step of the analysis the findings from CS and LA were compared, and subsequently these with those from upper secondary school.

RESULTS

Resources at secondary school

In terms of curriculum materials/material resources the textbook was an important resource for most secondary school students, and so were the graphical calculator (also used in examinations) "to quickly plot graphs" (interview reference: CSS01), and past examination papers for revision and practice to prepare for the national examinations. The textbook was seen as the main source of exercises, which were done in class or at home. Regarding homework one student remarked:

At school I didn't do my homework. There was homework but yeah, if you worked on it during the class … you would get halfway and then at home I was like oh, I get it. I don't have to do the remaining exercises (LAS01).

Online general resources (e.g. YouTube; Kahn academy) were hardly mentioned in relation to secondary school.

In terms of human resources the teacher and classmates were mentioned as an important support for secondary school students. Interestingly, teachers' explanations of the mathematical concepts were not important for all students: in some schools students apparently worked largely independently (with the textbook) and the teacher was only occasionally consulted.

In short, to learn the mathematics and pass the examinations, students reported that it was sufficient to follow the teachers' explanations, do (all) the exercises in the textbook, and practice with the past examination papers. There were few resources, and the ones provided could be straightforwardly accessed and used for solving the problems posed.

Resources for CS

Figure 1 shows the typical resources used by a CS student. The lecture appeared to be an important starting point for many students, albeit not for all. They provided an orientation on the subject ("it's easier for me to revise/practice when I have already seen/heard about it"; Figure 1), and to some extent an enculturation into the world of mathematical concepts and their usages. A lecturer said: I do the historical aspect too. Some things have been known for 3000 years, so you have to know that too. So I also do applications and add historical things (..) I just want them to be excited about the subject (CSLI03).

CS was supported by an almost 1100 page general CS textbook (authored by an "external" author) chosen from commonly used university CS textbooks. It contained the essential theory and part of the homework exercises (explained by someone "outside" the students' environment), but its content was not specifically aligned with the lectures and the final examinations. Curriculum resources, such as the textbook, were mainly used for exercises, whereas lecture notes (by the teacher) seemed important for knowing about the content to be learnt. Students' own notes were often used for orientation ("I write down important stuff; also when the teacher says 'this is likely to be on your exam', I write this down."; Figure 1). Selected students used the textbook for additional/different purposes: to read the theory in the textbook to prepare in advance for the lecture.

At the same time digital resources, such as general online videos (e.g. YouTube; Kahn academy) and video-recorded lectures (whole lectures or clips of particular moments), were said to provide additional explanations. The course's online tests and the weekly coursework were used to check whether one had a good (basic) understanding of the content. Moreover, students also mentioned human resources, such as the lecturer/tutor and the roommate "to ask questions" (Figure 1). Friendship groups were important for many students, and for some they were their first line of support (before the tutor or peers in the tutorial/s), to work with on the coursework or to consult about difficult theory or exercises.

Resources for LA

In terms of material resources LA was supported by a 200 page course-specific reader, authored by the lecturer and developed and improved over the years. It contained the essential theory and the exercises required to prepare for the examinations -this was "the backbone" of the course, according to the lecturer and the tutors. Other resources, such as the lectures, the homework/coursework exercises, the lecture notes and the videos with worked examples were all aligned with the reader, to become a comprehensive and complete set of resources for the students. The importance of using these resources to individually make sense of the mathematical content, and doing lots of practice exercises "at home", was emphasised by a student: As in the CS course, the lectures appeared a starting point for many students; they provided an orientation on the subject, and an enculturation into the world of mathematics (e.g. when mathematical proof was explained as an essential mathematical thinking process). Moreover, in terms of human resources, students relied on peer groups (e.g. they collaboratively solved problems during the weekly tutorials), and on the tutors to provide help-this was an important support in the LA course. Tutors were considered more approachable than the lecturers, although students were generally positive about the possibilities to ask questions to lecturers.

Comparing resources and their use

School -university: Whilst selected resources (e.g. textbooks, past examination papers) appeared to be part of the "staple diet" for every student at university or school level, at university students tended to use more, and more varied resources than at upper secondary school, including online lecture videos, video clips (of "difficult" notions), online texts. In addition, selected resources, such as lecture notes, were not mentioned in the secondary school context, where the theory would be taught by the teacher who aligned his/her lessons with the book. Some of the additional resources, such as video lectures, teachers' lecture notes, readers on specific topics, and online tests, were part of the curriculum resources made available by the university. Other resources, such as online applets and videos, were identified by the students themselves.

In terms of human resources there were also differences: at secondary school for most students the teacher provided practically all of the necessary guidance for learning the mathematical topic. At university the lecturer provided the theory, an overview of what was important for their learning of the topic area (and also for the examinations), and selected worked examples. The practical guidance, i.e. how to solve particular mathematical problems, was mainly provided in the tutorials and the exercises/coursework accompanying the lectures. Hence, students had to find their own learning/peer groups and supports for learning, as on their own it was not possible to manage the amount of work and the pace it was taught. This situation was exacerbated in the CS course, as only one hour of tutorial group work was offered, and students had to work collaboratively outside this hour for completion of their tasks. Hence, many CS students organized and coordinated their own support to work with their peers on the coursework, or to consult about difficult theory or exercises.

The students reported that, compared to secondary school, at university: (a) the pace was faster, (b) the content was more difficult to understand, and (c) the mathematical content was offered in larger steps/sections. The interview data suggest that the role and importance of resources changed as a result of this, as students needed more support structures and feedback on their work. This was particularly pertinent with one (autistic) student, who had dropped out of university. He claimed that he had done all possible CS textbook exercises and interim tests -a practice he had succeeded with at school, but he could not make sense of the questions when he sat the final examination. He was lost in the immensity of resources on offer, which he could not possibly all trial out and use for his learning. And he clearly missed the guidance and support given by his schoolteacher, practices which had provided him with confidence for his learning, and success.

CS-LA: Amongst the university curriculum resources, the student usages of the LA reader and CS textbook differed. To come to understand the topic/s, most LA students reported reading the reader, or the lecture notes, which were aligned with the reader. CS students mentioned the textbook as one of their resources, mainly used for worked examples and exercises. In CS, lecture notes and online resources were considered practically as important as the textbook, as part of the provided resource system. This can be understood in the light of the fact that the LA reader was very different, in relation to the course, to the CS textbook: the LA reader was a "book" prepared by the lecturer to align with his lecture, hence further lecture notes or online resources became secondary/ complementary. The reader contained all information for students to pass the examinations, and all other resources were related to/in line with the reader. In contrast, the CS book was only a backup for the lecture notes (which provided the essential notions to learn and study for the examinations), and students were only expected to "dip into" it for clarification, explanation and/or further exercises. Hence, the textbook did not provide a succinct support for CS students (e.g. to pass the CS examinations).

An important difference between CS and LA was due to the different organization of the tutor hours: in LA-4 tutor hours/week, tutor groups of ca. 30 students/ tutor; in CS-1 tutor hour/week, tutor groups of ca. 9 students/tutor. This meant that CS students had to work on practically all of the problems by themselves, as there was less support from the tutor. The fact that a wider variety of resources (used) was reported by the CS students, can in part be understood in the light of the different course organisation: in the LA case they adhere to the resources provided by the lecturer; in the CS case the students had to identify and organize their own support (e.g. online resources, friendship groups).

CONCLUSION

The results of this study have shown that the students built on secondary school experiences and they took these as default positions into their courses. However, learning mathematics at university was for most students different from learning mathematics at secondary school. At secondary school the resources (text book, past examinations, teacher) were well aligned and the teacher provided guidance and support. At university more difficult content had to be understood in a shorter period of time; and students had to identify and coordinate the relevant resources, and organise their own support system (including human support such as friendship learning groups), in particular in CS.

When comparing the two courses, the results indicate that (a) the course organization and (b) the provisions and organisation of the curriculum materials (in line with the learning goals) had an impact on which resources students used, and how they used them. In the LA course, with aligned curriculum resources, four weekly tutorial hours and group work, the use of resources largely corresponded to the intentions of use by the university teachers. In the CS course resources were not clearly aligned (although selected resources were recommended); it seemed that students were provided with a "bag of tools" to choose from. Moreover, students had only one weekly tutorial hour (plus six hours of lecture). This meant that students had (a) to identify which were the relevant resources for their individual needs, and (b) to find and navigate their own path through these resources, in order to work efficiently (with regards to examinations) and effectively (with regards to the learning of the mathematics). In both courses human resource, such as lecturers and tutors, peers and friends, played important albeit changing roles for orientation and help seeking.

The results of this exploratory study indicate that in particular large courses, such as CS (> 2000 students), could become better manageable for the students, if they were supported and coached in their resource choice and organisation/management, so that they can cater for their individual needs and preferences. This finding was less visible in a smaller course, where less resources were on offer, and where resources and resource use were more prescriptive and well aligned with the learning goals (e.g. the LA course). However, when particular educational reforms are implemented (e.g. towards more blended learning, with an abundance of digital learning tools on offer), students need to be supported in their "use" of these resources, in particular at transition from school to university. Course designers would also need to take this into consideration.

Figure 1 .

 1 Figure 1. Purpose and coordination of resources by a CS student

  But now you have like a huge amount of homework and then you also have workgroups where you can work on it, but then you don't get very far. (…) And if you don't do it at home, you just won't get it and you won't make your tests really good. So you really have to do a lot at home (LAS01).