Dr Siegfried Rouvrais
email: siegfried.rouvrais@imt-atlantique.fr

Inggriani Liem

Haraldur Audunsson

Cecile Gerwel Proches

IF YOU PLEASE, DRAW ME A RESILIENT CURRICULUM!

Keywords: Higher Education, Curriculum Design, Cooperation, VUCA, CDIO Standards: 3, 10, 12

The COVID-19 pandemic significantly impacted Higher Education as a whole, and the various educational institutions. It resulted in unexpected circumstances and unavoidable trade-offs to ensure that the curricula became more agile and flexible. Resiliency is also now a cornerstone, in order to navigate the disruptive change with the high levels of volatility, uncertainty, complexity and ambiguity. Ten universities in South-East Asia and three in Europe have since 2018 been engaged in a project aimed at improving the quality of their STEM programmes. In the context of this capacity building framework, this paper outlines a curriculum design workshop to stimulate curriculum transformations for VUCA contexts. The paper shares insights into facilitating international collaboration, which enabled different perspectives and representations of an original curriculum to emerge. The value of online tools as a way of promoting international collaboration and curriculum development is also discussed. The approach is based on a serious games model, to train curriculum designers to better embrace change, to collaborate and work across cultures. It is transferable to locally support the future transformation of programmes, by sharing and challenging ideas. Target participants are University programme leaders, deans, educational quality managers, accreditation bodies, curriculum heads and council stakeholders, as well as partners from industry, and even students. The main objectives and phases of the collaborative workshop are presented, followed by implications and recommendations aimed at developing a Resilient Curriculum framework.

INTRODUCTION

In Science, Technology, Engineering and Mathematics (STEM) Higher Education (HE), Technical and Technological Universities are increasingly faced with the challenges and tensions of developing the competencies and employability of their graduates, which has been further exacerbated by the global pandemic [START_REF] García-Morales | The Transformation of Higher Education After the COVID Disruption: Emerging Challenges in an Online Learning Scenario[END_REF][START_REF] Govindarajan | What the Shift to Virtual Learning Could Mean for the Future of Higher Education[END_REF][START_REF] Mishra | Online teaching-learning in higher education during lockdown period of COVID-19 pandemic[END_REF]. During the last decade, considerable emphasis has been placed on designing educational programmes and courses to develop the core and transversal skills as capabilities of future engineers and professionals, largely within a stable context. At a more systemic level, it is critical to identify international best practices regarding curriculum transformation processes in the context of a volatile, uncertain, complex and ambiguous (VUCA) world [START_REF] Krishnamurthy | The future of business education: a commentary in the shadow of the Covid-19 pandemic[END_REF]. Krishnamurthy (2020:2) argues that the "move to emergency remote teaching due to COVID-19 provides a discontinuous disruption to business-as-usual" and that it is critical to "unbundle and re-invent teaching, learning, assessment and certification". Unexpected disruptions may be particularly relevant for engineering education. For example, several CDIO standards refer to interpersonal skills that rely on direct interaction between students (e.g. Standard 4 on Introduction to Engineering or Standard 8 on Active Learning when considered experiential) and physical workspaces and laboratories (Standard 6 on Engineering Learning Workspaces). The quality levels of these standards, as examples, can easily be impacted due to a crisis, as the recent pandemic is one example, and hence the curriculum designers need to consider and develop a contingency plan for a more resilient educational program that might not be instantly apparent. [START_REF] Marinoni | The Impact of Covid-19 on Higher Education around the World: IAU Global Survey Report[END_REF] in their study emphasised "the important degree of stress and constraint currently experienced by higher education institutions around the world. Almost all institutions that responded to the survey are affected in a way or another by the COVID-19 crisis and the crisis has affected all institutional activities… the incredible amount of pressure on higher education institutions to cope during the current crisis and at the same time their resilience and creativity." Taking into account the VUCA and multiple disruptions, STEM curricula in their structures are to become more adaptable, responsive, and resilient.

What could a resilient curriculum (RC) entail? Is it possible to reconcile HE stakeholders with cooperation design, in order to create a common understanding and develop hands-on methods? To address the question of curriculum resiliency, this paper proposes a process to facilitate STEM resiliency curriculum development with collaboration tools for curriculum design, which is largely based on a collaborative workshop, which was designed and run in 2021, with international programme designers, from diverse cultural backgrounds. As part of a European project, our aim was to bring together ad-hoc practices around agile thinking, with tools based on a serious game, from a playful and virtual perspective. The game also brings together sharing, conceptualization and collaboration, beyond the skills of the participants, around the design of agile and resilient curricula.

The proposed collaborative workshop can assist HE institutions to pave the way towards a Resilient Curriculum Framework (RCF), which can also help improve the quality and relevance of their curricula. With an international collaboration perspective, it is to facilitate innovative curriculum design activities, by exchanging or developing new practices and methods, and inspiring and learning from others. The proposed workshop is very much in line with the spirit of CDIO as reflected in its emphasis on integrated curriculum (Standard 3) where Faculty can play an active role in designing, as enhancement of faculty competence for effectively improving curriculum (Standard 10), by creating forums for sharing ideas and best practice.

CURRICULUM AGILITY AND RESILIENCY

Curriculum agility is important when relevant professional disciplines are developing rapidly, as in engineering [START_REF] Brink | Curriculum Agility: Responsive Organization, Dynamic Content, and Flexible Education[END_REF]. Brink et al. developed a shared vision on curriculum agility, where an agile curriculum is responsive and adaptable to changes in society and business, as well as student characteristics and needs, by having the capacity to change structures, learning outcomes, and learning activities in a timely manner. They introduced nine principles of curriculum agility, which defines or refines the concept of curriculum agility. Agility is perhaps now more than ever an unavoidable property of curricula to meet the transformational challenges of educational programmes in a more continuous manner, due to the various crises impacting the HE sector [START_REF] García-Morales | The Transformation of Higher Education After the COVID Disruption: Emerging Challenges in an Online Learning Scenario[END_REF][START_REF] Govindarajan | What the Shift to Virtual Learning Could Mean for the Future of Higher Education[END_REF][START_REF] Mishra | Online teaching-learning in higher education during lockdown period of COVID-19 pandemic[END_REF]. In 2022, within the CDIO community, the development of self-assessment rubrics for the agility identified principles is underway for three clusters: Curriculum Vision & Strategy, Curriculum Quality & Provision, and Curriculum Design & Research principles. These rubrics are to support programme leaders in assessing the agility of their curricula to support the change processes in a common maturity scale (cf. CDIO standard 12).

The 2020 pandemic emphasised the need for curriculum flexibility and agility at all educational levels. There is also growing interest in system-level resilience within the 2030 Agenda for Sustainable Development1 . According to IROWH (2016), the word "resilience" comes from the Latin verb "risalire", which means to rise again (to bounce back). A resilient community is able to cope with change, and retain its structures and functions after disturbances in order to keep up with continuous development. Resilience. Some HE programmes were particularly resilient for the pandemic, as for a University of Bristol programme which implemented a new curriculum in undergraduate Engineering [START_REF] Berthoud | Designing a resilient curriculum for a joint engineering first year[END_REF]. With a process of developing programme-level intended learning outcomes, followed by a process of linking the content and assessment of the programmes to focus on these learning outcomes, it resulted in a simplification of the structure of their programme along the pandemic. Flexibility and diversity of content delivery methods allowed teaching to large cohorts in a variety of situations due to changing restrictions thanks to the constructive alignment approach.

Curriculum development and collaboration in curriculum design require a change in mindset in VUCA contexts. HE curriculum designers and Faculty need to be able to embrace change, and be immersed in and able to cope with perturbations. It is important that they are more adaptable, cooperative, innovative, able to collaborate and work across cultures.

SOURCES OF INSPIRATION

The curriculum design workshop was mostly inspired by two sources. These are relevant examples of serious games, which facilitate stakeholder collaboration and cooperation to help identify alternative strategies and structural solutions. First, University of Utopia is a serious collaborative game intended for HE teachers [START_REF] Laplanche | French : University of Utopia : un jeu sérieux collaboratif pour utiliser des concepts en pédagogie universitaire[END_REF]. It allows pedagogical concepts presented via concept cards to be transferred to teaching situations defined collectively in order to improve the quality of learning activities. The participants develop a mono and multidisciplinary educational activity project on a poster. Second, the Climate Fresk (CF) offers workshops to raise awareness of climate change issues (www.fresqueduclimat.org). It promotes individual and/or collective awareness and facilitates

Background

Since 2018, ten universities in South East Asia and three European universities have been partners in a project aimed at improving the employability of students after graduation (www.eastemproject.eu) and industry collaborations [START_REF] Rouvrais | University-Industry collaboration themes in STEM Higher Education: An Euro-ASEAN perspective[END_REF]. This European project also aims to develop the skills of teachers, to promote the creation of networked educational services (cf. network of STEM centers 2 ). The collaborative approach stimulates the sharing of good practices (Bennedsen & Rouvrais, 2016) and is a source of inspiration.

Cooperation kick-off

Some project members participated in a first workshop to stimulate and reinforce cooperation skills between partners involved in programme transformations. To commence with, as a cooperative design, participants first experienced an online CF on an international scale in 2021 (see Figure 1.b). It lasted approximately four hours, and was held online on Zoom in October 2021. Mural was used for the CF, and interaction was ensured through the use of Zoom breakout rooms, which facilitated group interactions. Eleven participants from diverse countries, including France, Iceland, Indonesia, South Africa, Vietnam and Thailand, participated. A qualitative and quantitative questionnaire allowed for analysis. The feedback primarily related to duration, group size, rhythm, STEM topic alignments, pros and cons, and the use of the Mural tool. Overall, it was found to be a pedagogically fruitful workshop to engage in systems thinking and intercultural cooperation. As a cooperative design, overall 60% of the participants liked the CF workshop (strongly agreed), 30% agreed, 20% neither agreed or disagreed (Likert scale). Twenty-five percent appreciated (strongly agreed) that collaboratively using cards in a design and system thinking workshop can be beneficial for a future RC workshop, 50% agreed, 12.5% neither agreed or disagreed, and 12,5% disagreed.

Best practices from the CF gameplay were further explored for the novel workshop. One RC workshop was then held online in November 2021 on Zoom, with five participants from different institutions, participants collectively exploring how first to design curricula.

CURRICULUM DESIGN WORKSHOP

2 https://www.fsf.vu.lt/en/eastem-centers-platform

Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15, 2022.

"When the mystery is too impressive, we dare not disobey… I took out of my pocket a sheet of paper and a pen. But then I remembered that I had mainly studied geography, history, arithmetic and grammar and I told the little guy (with a little bad humour) that I did not know how to draw" (Antoine de Saint Exupéry, The little prince, chapter 2, 1943).

The RC workshop was designed with the aim of developing collaboration between training managers, program directors, teachers, students and industry. In order to "get out of its own walls" and exchange in complete neutrality, the groups are to be mixed between stakeholders and institutional cultures to "draw" the main lines of an imaginary training program, resilient to the unforeseen. Under a serious game schema, and informally following some design thinking methods, the RC workshop stimulates a collaborative innovation in curriculum transformation with a view to analyse VUCA readiness of programmes.

In curriculum modelling, arrays including course blocks and semesters columns are often used, with learning pathway constraints between core, broadening or elective courses. It often takes very long to develop new programme architectures, and other challenges arise, such as deep managerial perspectives, resistance to change or administrative duties, which could limit the innovation and freedom for redesigning existing structures. A curriculum architecture representing a set of study program curricula of a faculty cluster helps to reach resilience principles when it already supports the basics of agility at more systemic levels. Participants reflected on the fact that there was a lot of paperwork, administration, and sometimes little freedom on structures, with respect to the university curriculum framework, accreditation standards and other requirements. Among the six drawings, as an example, in Figure 2, the revamped burger-styled curriculum inspiration came from a meeting in Paris in June 2014 with members from UpCERG group from Uppsala University, Aalto University, and IMT Atlantique. It was recognized as attractive and flexible, but with few temporal links between too abstract courses. As a metaphor, it offers a kind of menu for curriculum blocks. The workshop highlighted how architectural representations of the curriculum suggested by the international participants varied considerably, both from a structural point of view and from the informal artefacts used. Five participants were asked to cooperatively sketch a resilient curriculum, echoing arbitrarily a BSc in cybersecurity. The participants however experienced challenges in designing a common understanding and reframing the problem in a human-centred and graphical way. They were questioning the main aim, problem, assumptions and implications. As a future activity, VUCA scenario could then be introduced to reflect on the resilience of the various participant's structures.

The RC workshop concluded with a short debriefing session and dialogue. Table 1 indicates preliminary elements of analysis, based on feedback.

Table 1. SWOT of the RC trial workshop.

Strengths Weaknesses

REFLECTIONS AND IMPLICATIONS

Intercultural encounter

The RC workshop allowed participants to identify many intercultural, professional and personal skills and aptitudes, which are mobilised by the participants. Problems of cross-cultural adaptation however remain. As noted by [START_REF] Kennedy | An Aboriginal way towards curriculum reconciliation[END_REF], design and implementation of a programme may be influenced by cultural methods for conducting business and maintaining knowledge integrity, by taking interdisciplinary teams of academics on a journey towards 'curriculum reconciliation'.

As an example, for the Indonesian author, the "gotong royong" principle is anchored as related to its ideological basis. Gotong royong in literal meaning is "mutual assistance", a kind of collaboration with empathy, compassion in order to share burdens. During the 2019pandemic, the Indonesian government encourage "gotong royong" to overcome the crisis. For the South African author, the African philosophy or concept of Ubuntu arose. This could be translated as "I am because we are" or "I am because you are", linked to Zulu, and in Xhosa, the meaning is also broadening to "the belief in a universal bond of sharing that connects all humanity". For the Icelandic author, "þetta reddast" is described as the country's motto. Þetta reddast can be translated to "it will all work out okay". Life could often be difficult in this barren, Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15, 2022.

harsh country and over time Icelanders have developed a mentality which can sometimes seem a bit carefree. When faced with difficulties Icelanders always maintain a belief that things will work out in the end; no matter how big the problem, a solution will always present itself"3 .

Cooperation to stimulate thinking out of the box

It is important to note that the development of resilience takes time, especially in systems that are characterised by tight control and little room for risk-taking and innovation, as may be found in many HEIs. This workshop was thus exploratory in nature, with the purpose of primarily focusing on how best to support the development of resiliency and agility. It is important to bring together multiple, diverse stakeholders, who can journey through a process whereby their mindsets and traditional ways of working are challenged. The diversity of the group further led to participants being exposed to multiple perspectives and fresh thinking to attempt to address common problems plaguing HEIs.

It is important that members of HEIs, especially those on the ground, have the necessary skills and capabilities to drive change. The fixed organisational culture of HEIs which can be quite focused on policies and plans, can ensure efficiency, but can lead to staff being inhibited and unable to respond and recover quickly in the face of adversity. If the individuals in HEIs are unable to bounce back quickly, then how much more challenging is it going to be when examining the curriculum, especially when considering that it is the individuals in the system who are responsible for achieving outcomes. We thus argue that resilience-building of the curriculum is dependent on the extent to which individuals in the system themselves are capable of embracing change and being responsive and proactive.

Diversity of viewpoints

Participants highlighted the value of being able to hear the experiences of others, especially international perspectives. There was some value in the metaphors-direction, to help people think about curriculum as something other than just a timeline of courses planned over three or five years. The hamburger abstract design, in particular, was especially received well and led to the other participants thinking more creatively, as well during the second collaborative design from scratch phase; thus, emphasizing different characteristics of a curriculum. Increasingly, more and more criteria on HE programs are offered in quality procedures, sometimes even imposed by accreditation agencies.

To date, there is no global and unified framework for visualizing and discussing curriculum designs. The workshop presented was without any real constraints in terms of modeling, able to guide actors in HE to collaborate more effectively in their curriculum transformations. This informal and cooperative workshop makes it possible to start to act together in the construction of action plans in connection with strategies set by Universities, Ministries or accreditation systems. In our VUCA times, the present impact of the "turbulence" on the curriculum is being experienced worldwide.

Many HEs suffer because they have a rigid curriculum, with specific courses which are fixed.

It is critical that the curriculum is both agile and resilient. As an example, in Indonesia, the Ministry of Education started to run the MBKM (Independent Learning-Independent Campus), formed at the beginning of 2020. Students of the 4-year Bachelor program are given the freedom to take the opportunity to spend one semester outside the study program and two semesters of carrying out learning activities outside the university. The credits of those semesters are taken into account in the academic transcript. It means that the curriculum must support a "big room" for MBKM activities and its credits. If the curriculum is not agile or is not resilient, it creates problems for the study programme. Students learn and gain experience from real world professionals, by doing the activities outside the university, especially through internships (in industry, government offices, research centres or in the community).

CONCLUSION

The main purpose of the RC workshop is to train curriculum designers to share a common understanding with intercultural colleagues and team members, to later transform (redesign/reform) their own curricula according to local challenges, traditions, culture, etc. What HE in VUCA times will entail, still remains a question for our future, even more in the context of international programme interoperability as to be seen in the ongoing European Universities movement. They are opportunities to allow organisations to increase the quality and relevance of curriculum design activities and to increase capacity to operate jointly at transnational level for programme interoperability. The RC workshop, held as a trial in Fall 2021, aims to identify alternative curriculum design strategies as curriculum solutions which may not be instantly apparent. It questions the problem and investigates whether it is possible to reconcile HE stakeholders with cooperative design, in order to create a common understanding and develop a collection of hands-on methods for curriculum transformation in VUCA contexts.

The workshop can assist curriculum developers and facilitate resilience of educational systems facing VUCA events. The tool paves the way to resilience investigations within a curriculum framework, to include models, methods, and processes transferable to local institutional contexts according to the local needs and challenges, for curriculum renewal, transformation and reconciliation. The tool challenges assumptions, strategies, alternatives, implications and foresight solutions. The RC workshop, which was presented in this paper, is a preliminary activity, to highlight why and how collaboration is needed between different stakeholders to open mindsets and innovation. On an international scale, the RC workshop demonstrated that collaboration of all stakeholders in curriculum design can be done effectively with serious games using various online tools. Sustaining all the CDIO standards during VUCA times is a challenge, for example because it relies on interpersonal skills, active learning and dedicated workspaces, all of which can easily be disrupted by unexpected VUCA events. An open minded cooperative workshop as outlined here may indeed be fruitful when developing strategies for more resilient and interoperable CDIO spirited engineering programs.

Globally, HE has to contemplate and start to develop resilient curricula, and this paper is a very first step in that direction. In 2022, in the context of reconfiguration of HE and the quality assurance landscape in Europe and worldwide, roles and challenges for STEM schools and accreditation bodies have been questioned. As a first recommendation, a RC Framework could be investigated, to include models, methods, processes and tools for guiding transformations of STEM programmes. As a starting point, graphic languages for the representation are now to be studied, maybe with a system modeling approach which makes sense to meet such challenges of exchange and interoperability [START_REF] Rouvrais | Architecting the CDIO Educational Framework Pursuant to Constructive Alignment Principles[END_REF]. As a second recommendation, the RC workshop may be complemented with concept cards and strategic canvas to guide participants to identify more specific strategies and solutions under VUCA constraints.

  Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15, 2022. a constructive discussion, creating a will to act in the face of the challenge of an ecological transition (cf. Figure1.a).

Figure 1 .

 1 Figure 1.a Climate Fresk sample in 2021 with students at IMT Atlantique, and 1.b online with EASTEM curriculum designers

  Six drawings were proposed, e.g. (i) curriculum architecture of all study programs in ITD (with four Faculty and three domains), (ii) a postgraduate diploma in Business Administration at UKZN with course structure, (iii) pillars and structure of a biomedical engineering BSc. at RU, (iv) an industrial design engineering bachelor with flexible choice-based curriculum.

Figure 2 .

 2 Figure 2. Curriculum drawing sample.

https://sdgs.un.org/2030agenda

Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15, 2022.

Iceland Magazine, by Sara McMahon, June 19/2014. https://icelandmag.is/article/what-does-thetta-reddast-mean Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15, 2022.

ACKNOWLEDGMENTS

The authors would like to warmly thank Hakara Tea, Climate Fresk international facilitator, for his valuable input and advice on collaborative design, as well as Assoc. Prof. Suzanne Brink from Umea University for sharing her thoughts on the presented workshop. The authors also acknowledge their colleagues from the EASTEM project, co-funded by the European Union (598915-EPP-1-2018-1-SE-EPPKA2-CBHE-JP).

BIOGRAPHICAL INFORMATION

Siegfried Rouvrais-Delahaie is an Associate Professor at the CS Department of IMT Atlantique (formerly Télécom Bretagne). His educational interests are in methods and processes for higher education transformations.

Inggriani Liem is affiliated with Institut Teknologi Del, and is a former lecturer of ITB School of Electrical Engineering and Informatics. She is head of Bebras Indonesia NBO, an international initiative aiming to promote computational thinking. She is an expert on curriculum design in Informatics and experiential-based learning in Software Engineering.

Haraldur Audunsson PhD is an Associate Professor of Physics in the Department of Engineering at Reykjavik University. His interests include applied physics, physics and engineering education, and experiential learning. He has been involved in curriculum design for several years.

Cecile Gerwel Proches PhD is an academic in the Graduate School of Business and Leadership at the University of KwaZulu-Natal (UKZN). Her teaching and research interests include leadership, change management, and organisational culture.


Dominique Guégan
email: dguegan@univ-paris1.fr..

Florian Ielpo
email: ielpo@ces.ens-cachan.fr

Keywords: Subjective Distribution, Autoregressive Conditional Density, Generalized Hyperbolic Distribution, Fed Funds futures contracts. JEL Codes: C51, E44

In this paper, we introduce a new approach to estimate the subjective distribution of the future short rate from the historical dynamics of futures, based on a model generated by a Normal Inverse Gaussian distribution, with dynamical parameters. The model displays time varying conditional volatility, skewness and kurtosis and provides a flexible framework to recover the conditional distribution of the future rates. For the estimation, we use maximum likelihood method. Then, we apply the model to Fed Fund futures and discuss its performance in recovering the expected stance of monetary policy in the US.

Flexible Time Series Models for Subjective Distribution Estimation with Monetary Policy in

View *

Introduction

Risk neutral and subjective distributions are linked through the aggregated risk aversion that is implicit in financial markets. To detect the term structure of the subjective distribution, most of literature use the latent information in option prices while making assumption on the shape of risk aversion. We propose here to use a time series model for the historical distribution of the futures on interest rates in order to estimate the subjective distribution. This model is based on a GARCH volatility structure and is conditionally distributed as a Normal Inverse Gaussian distribution. The model thus encompasses time varying volatility, skewness and kurtosis. We estimate the model on a dataset made of several Fed Fund futures contracts and we discuss the empirical results obtained.

The subjective distribution is the distribution that reflects the market participants' perception of the future value of a financial asset. This distribution carries important information regarding the market participants' risk perception over states and maturities. Contrary to the subjective one, the risk neutral distribution1 is supposed to make market participant neutral toward risk. This latter distribution should therefore carry no risk aversion component. Since these distributions are equivalent in the probabilistic sense, they are solely related through a risk aversion correction. Following [START_REF] Leland | Why Should Buy Portfolio Insurance[END_REF], this relation can be roughly stated as follow:

Risk Neutral Probability = Subjective Distribution × Risk Aversion Adjustment.

(1)

This relation is well-known for empirical finance applications: see the results presented in [START_REF] Ait-Sahalia | Nonparametric Risk Management and Implied Risk Aversion[END_REF], [START_REF] Jackwerth | Recovering Risk Aversion from Option Prices and Realized Returns[END_REF] and [START_REF] Rosenberg | Empirical Pricing Kernel[END_REF]. However, it has scarcely been used to analyze the link between monetary policy and the bond market. For most of the developed countries the Central Bank directly controls the short rate. In this perspective, the subjective distribution of the short rate process yields pieces of information on the market participants' perception of the future stance of monetary policy. Monetary policy makers need to measure the perception of the Central Bank policy by financial markets. For example, the information disclosure following the regular meetings of the Federal Reserve Board is known to be followed by a reduction of the bond market volatility. However, the financial market assessment of the monetary policy goes far beyond the sole changes in conditional variance: the estimation of the conditional distribution is thus essential to Central Bankers. On the market participants side, monitoring the changes in "market beliefs" is also important for asset management and risk control purposes: monetary policy is known to be the main risk factor in the government bond market. For all these reasons, the design of a model of the kind that is to be discussed here is one of the cornerstone of empirical monetary policy.

Until now, several estimation strategies for the subjective distribution have been proposed. On the one hand, several articles proposed to use the risk neutral distribution estimated from option prices as a proxy for the subjective one: see [START_REF] Mandler | Comparing Risk Neutral Probability Density Functions Implied by Option Prices -Market Uncertainty and ECB-Council Meetings[END_REF], [START_REF] Brière | Market Reactions to Central Bank Communication Policies: Reading Interest Rate Options Smiles[END_REF] and the survey presented in [START_REF] Mandler | Market Expectations and Option Prices[END_REF]. On the other hand, using the relation presented in equation (1), [START_REF] Jackwerth | Recovering Risk Aversion from Option Prices and Realized Returns[END_REF], [START_REF] Ait-Sahalia | Nonparametric Estimation of State-Price Densities in Financial Asset Prices[END_REF] and [START_REF] Rosenberg | Empirical Pricing Kernel[END_REF] used the historical distribution of index returns as a proxy for the subjective distribution. The conclusion of the last stream of literature is that the risk neutral and the subjective distribution proxy are very different.

However, these attempts were mainly designed for equity assets, and little attention has been devoted to fixed income securities. For such markets, the additional problem is the term structure aspect of the subjective distribution. We thus need a kind of financial asset whose historical dynamic carries information about this term structure.

We propose to use the existing future contracts to do so: they are actively traded and used to make bets on the future stance of monetary policy. In order to have a reactive estimate of the subjective distribution, we use here a dynamical Normal Inverse Gaussian distribution, building a flexible time series model. This methodology allows for a direct mapping from the past observations space into the parameters' space, without any a priori knowledge about the dependence between them. The estimation of this model can be performed by maximum likelihood. When confronted to the data, this model is accepted. An event study shows that the results obtained with this model are close to what is expected.

The paper is organized as follow. Section 2 presents several theoretical and empirical arguments to use the historical dynamic of future rates for the estimation of the subjective distribution. Section 3 presents a new time series model using a dynamic NIG distribution for the estimations. Finally, the Section 4 discusses the empirical results. Section 5 concludes.

Methodology and framework

One of the main novelties of this paper is to show how to use the dynamics of the futures contract prices to estimate the subjective distribution of the futures instantaneous rate for various maturities. In this section we discuss the reasons why we propose such a strategy, both from an empirical and theoretical point of view. We first present the main hypothesis used in this article. Then we provide details regarding the Fed fund contracts and the datasets used in this paper. Finally, we present empirical facts regarding the dataset and supporting the model presented in the next section.

Main assumption and notations

In this section, we review the main notations and hypothesis that are used in this paper. We provide essential insights within the theoretical link between monetary policy and the bond market, through future rates.

Let R(t, T ) be the spot rate of maturity T at time t. The short or instantaneous spot rate r t on date t is given by:

r t = lim T →t R(t, T ). ( 2
)
When t is different from today, this rate becomes the future instantaneous spot rate, that is thus unknown at time t and assumed to be a random variable. We denote F (t, T ) the instantaneous future rate known at time t for a maturity T from the future contracts. Under no arbitrage restrictions, the link between spot rates and instantaneous forward rates is given by:

R(t, T ) = T τ =t F (t, τ )dτ T -t (3)
The spot yield curve being an average of the forward rates can be used to recover the instantaneous forward rates (using a spline model for example, see e.g. [START_REF] Svensson | Estimating and Interpreting Forward Interest Rates: Sweden 1992-1994[END_REF]). Now, we know that 2 :

E S [r T |F t ] = F (t, T ), (4)
where E S [.] denotes the expectation under the subjective distribution S, and F t being the filtration produced by the information set at time t. We will define define this filtration in the next section: it will basically result from the past evolutions of the futures. This relation states that the forward rates can be considered as a market forecast of the future short rate. In many countries, this rate is directed by the Central Banks: in this perspective, the forward rates can be used to investigate the financial markets' understanding of monetary policy and the market perception of the risk associated to the upcoming Central Bankers meetings 3 . This link between the bond market and 2 This is one of the presentation of the expectation hypothesis. See e.g. [START_REF] Jarrow | Modelling fixed income securities and interest rate options[END_REF] for more details.

3 Recent papers nonetheless empirically showed that these forecasts may be biased: [START_REF] Piazzesi | Futures Prices as Risk-Adjusted Forecasts of Monetary Policy[END_REF] showed that the difference between the realized target rate and the market forecast implicit in the yield curve is statistically significant and even linked to the economic momentum. This bias in the forecast is often referred to as term premium. Even though this premium can be very well explained in sample, the out of sample performances are really poor (and biased again). This is why we choose to discard this problem for the time being, considering that given the information on the current date t, the market forecast for the date T > t is conditionally unbiased. We hope being able to circumvent in a more documented fashion this difficulty in a future paper.

monetary policy has already been used in e.g. [START_REF] Svensson | Estimating and Interpreting Forward Interest Rates: Sweden 1992-1994[END_REF], in order to measure the market forecast over several central bank decision meetings.

One of the main novelty of this paper is to develop a time series model to recover the distribution of r T conditionally upon F (t, T ) = {F (t, T ), F (t -1, T ), ...}, for each maturity T i of interest (those of the different future contracts or those of the Central Bank decision meetings). Here, we assume that the conditional historical distribution of F (t, T ) is close enough to the conditional subjective distribution of r T |F t , hence they can be treated as equal. The main hypothesis of our work is then:

Hypothesis 1. Let F S t (.) be the cumulative distribution function associated to the subjective distribution of r T conditionally upon the information available at time t. Let F H t (.) be the cumulative distribution function associated to the historical distribution of the future rate F (t, T ), conditionally upon the information available at time t. Then, the two distributions are equal on any point of their common support, i.e. F S t = F H t . Most of the existing literature assumes that the historical distribution of equity returns is a consistent proxy for the subjective distribution: see e.g. [START_REF] Jackwerth | Recovering Risk Aversion from Option Prices and Realized Returns[END_REF], [START_REF] Ait-Sahalia | Nonparametric Risk Management and Implied Risk Aversion[END_REF] and [START_REF] Rosenberg | Empirical Pricing Kernel[END_REF]. This hypothesis is valid as long as the working purpose is to recover the short term subjective distribution for equity indexes. Here, we are interested in recovering the term structure of the subjective distribution from the yield curve: this is why we propose to use the futures instead of the short rate. We document this hypothesis in the following subsection.

Dataset description

In this section, we present the Fed fund futures contracts that are used in this article. We first review the motivations to use these future contracts, before we detail the building of our own dataset.

There exist several types of future contracts that may be used to recover market probabilities of future rates hikes and cuts. Most of them are three-months contracts, which is not convenient for monetary policy analysis: Central Bank meetings are closer to a monthly frequency. For this main reason, we focus on the the Federal Fund Future contracts, that are monthly contracts. This contract has a price that is equal to

P (t, T ) = 100 -100 × 1 n T i=T -n+1 r i , (5)
that is 100 minus the one-month average of the future refinancing rate r i -that is the future target rate plus a daily cash premium -over a one month period (with n days).

The Fed Fund futures are a useful and now quite liquid asset to extract monetary policy stance expectations from the bond market. It has already been widely used in empirical research: see e.g. [START_REF] Krueger | The Fed funds Futures Rate as a Predictor of the Federal Reserve Policy[END_REF], [START_REF] Robertson | Using Federal Funds Futures Rates to Predict Federal Reserve Actions[END_REF], [START_REF] Kuttner | Monetary Policy Surprises and Interest Rates: Evidence from the Fed Fund Futures Market[END_REF], and [START_REF] Carlson | An Option for Anticipating Fed Action[END_REF]. These contracts are widely used by practitioners to measure the current market feeling about the future monetary policy stance. The extraction of market based monetary policy expectations requires some preliminary calculations that are detailed e.g. in [START_REF] Kuttner | Monetary Policy Surprises and Interest Rates: Evidence from the Fed Fund Futures Market[END_REF]. These are not reported here.

The global dataset used in this paper is made of 77 Fed Fund future contracts over their whole lifespan. The first contract's maturity is November 2000 and the final contract's is March 2007. The future prices time series are observed on a daily basis. The prices are converted into rates using the relationship mentioned before. This dataset includes rate hikes and cuts periods, which is important for the purpose of this paper, and especially for the examination of the information implicit in these future rates.

Nevertheless, for the ease of the presentation of the empirical results in Section 4, we chose to focus on 4 particular future contracts. These contracts have the following maturities: December 2006, January 2007, February 2007 and March 2007. We retained these contracts because of the period covered. Over the lifespan of these contracts, monetary policy started to change, with the end of a rate hikes period and the starting of an upcoming rate cuts period. This period should result in a dramatic change in the shape of the subjective distribution across the sample that is used here. At least, we should note a progressive change in the skew of the distribution over the period that is considered here: moving from a positive one -which is the sign of upcoming rate hikes -to a negative one, as financial markets will start believing in upcoming rate cuts.

This sub-sample starts on the 1 st of August 2006 and ends on the 30 th of October 2006. It is necessary to suppress at least the last two months of the lifespan of the future contracts for each of them in so far the level of uncertainty for these months is quickly decreasing: thus, during these last two months the volatility is likely to decrease quickly as expectations about the future stance of monetary policy are getting more and more accurate. This decreasing conditional volatility usually leads to diagnose non stationarity.

In the remaining of the paper, we generally refer to the first sample of 77 contracts as the "full-sample" and to the latter restricted sample as the "sub-sample". The fullsample is mainly used in this section and the sub-sample will be used for the empirical results presented in Section 4.

Stylized facts and informative content of the dataset

From a practical point of view, the future contracts are actively used by traders to take bets over the upcoming Central Bank decisions. On this point see the references presented in the previous subsection. Thus, their historical dynamics should reflect the changes in the market perception of the future monetary policy.

We propose to test this hypothesis by the following statistical analysis. Building on the previous notations, r T i is the spot rate on date T i , that should be forecast -accordingly to our assumption -by F (t, T i ). If the dynamics of the corresponding future rate is linked to the realized rates4 , then it should be possible to relate the distribution -and thus the moments -of

F (t, T i ) to r T i . We denote ∆f (t, T i ) = log F (t, T i ) F (t -1, T i ) , (6)
the log increment of the future rate, for a given maturity. This transformation ensures the second order stationarity of ∆f (t, T i ). Now, the descriptive statistics of ∆f (t, T i ) are denoted as m 1 (T i ) for the expectation, m 2 (T i ) for the volatility, m 3 (T i ) for the skewness and m 4 (T i ) for the kurtosis. Their estimation is done using the corresponding sample statistics.

We propose to check whether the historical distribution of the futures, proxied by the previous moments -expectation, volatility, skewness and kurtosis -can be explained by any combination of the realized corresponding spot rates. It would support the fact that the historical dynamic of the future rates carries information regarding the market view of the future realized rates.

We adopt the following notations: L(T i ) = log r T i is the log of the realized rate: it is meant to capture a level effect. ∆L(T i ) = L(T i ) -L(τ i ) is the variation of the spot rate over the windows defined by the lifespan of the future contracts, i.e. the time elapsed between the issuance τ i and the delivery

T i dates. |∆L(T i )| = |L(T i ) -L(τ i )| represents
the total variation over the duration of the future contract. Its absolute value accounts for the realized volatility over the future lifetime. Finally, S(T i ) = L(T i ) -L captures the effect of turning points of the rates' dynamics, when L T is way above or below its historical average L. It is bound to have a particular effect regarding the historical dynamic of the futures, given the well documented mean reverting property of interest rates.

We adjust on the sample moment m j (.) the following linear model:

mj (T i ) = α 0 + α 1 L(T i ) + α 2 ∆L(T i ) + α 3 |∆L(T i )| + α 4 S(T i ) + ǫ(T i ), ∀i (7)
where ǫ(T i ) is a centered white noise and mj the sample moment. The estimation has been performed on the full sample dataset, using ordinary least squares. The estimated R 2 are presented on figure 1. The R 2 obtained are very high for a model with 4 explanatory variables, indicating that the moments of the historical distribution of the futures are statistically linked to the realized rates. It seems now natural to relate this historical distribution to market expectations: the subjective distribution is by construction a forecasting density and should verify empirical properties of this kind.

Second, we briefly discuss the stylized facts of the futures dataset used in this subsection, for time series modeling purposes.

Financial datasets and especially financial returns are known to display time varying volatility and higher order moments. The log increments of future rates display the same characteristics. On the previous dataset, we computed the first four moments estimator for each contract. Figure 6 presents the results obtained: each of the moments dramatically vary across the contracts. Thus, we performed T × R 2 tests for autoregressive moments of order 1 to 4 for each contract in the dataset (for this test, see [START_REF] Engle | Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation[END_REF] and for its application to higher order modeling see [START_REF] Jondeau | Financial Modeling Under Non-Gaussian Distributions[END_REF]). Figure 6 present the test statistics along with the χ 2 quantile for testing first order autoregressive patterns for each moment across the contracts: for most moments and contracts, there is an autoregressive component that is statistically significant. This supports the idea that the first four moments are time varying.

Finally, we propose here to test the adequacy of four distributions to the data at hand. These distributions are the skewed t-distribution, developed in [START_REF] Hansen | Autoregressive Conditional Density Estimation[END_REF], the skewed Laplace distribution (see [START_REF] Kotz | The Laplace Distribution and Generalizations: a Revisit with Applications to Communications[END_REF] and the references within), the Normal Inverse Gaussian distribution, introduced by Barndorff-Nielsen (1997) and the Gaussian distribution, used as a benchmark. Before testing the adequation of these distributions, the log-returns were first filtrated using a GARCH(1,1) model introduced in [START_REF] Bollerslev | Generalized Autoregressive Conditional Heteroscedasticity[END_REF], so as to take into account the possible second order dependency in the data at hand. We performed Kolmogoroff-Smirnoff tests on the GARCH residuals.

The results obtained are presented in the table 1. The NIG distribution yields the best results, once compared to the other distributions, up to a 5% risk level. Thus, in the remaining of the paper, we will only retain the NIG distribution to model the distribution of the future rates. The adequation tests lead to accept this distribution for 95% of the samples: the regularity of the acceptation is a very interesting feature for our purposes. We want to find a distribution that is likely to fit any future sample. The fact that this distribution has at least one more parameter that the other competitors may explain the quality of the fit. However, this is not a drawback for our approach: what we favor here is the flexibility of the distribution for estimation purposes. For all these reasons, we will only consider the NIG distribution for the models that are to be presented in the following section.

Flexible time series models

On the basis of the previous preliminary empirical work, we propose a series of NIGbased new time series models allowing for time varying volatility, skewness and kurtosis by letting the more sophisticated version of the model have dynamic parameters controlling the skewness and the kurtosis. We first present the general settings of the models and then discuss the estimation methodology.

General settings of the model

The Normal Inverse Gaussian distribution (NIG hereafter) has been introduced by Barndorff-Nielsen (1997) and used successfully in many financial applications (see e.g. [START_REF] Jensen | The NIG-S&ARCH Model: a Fat-Tailed, Stochastic, and Autoregressive Conditional Heteroskedastic Volatility Model[END_REF]). This distribution accommodates basic stylized fact of financial time series such as asymmetry and excess kurtosis greater than 0. In this section, we build nested time series models based on this distribution to provide estimates of the subjective distribution.

A stochastic variable X is said to be normal inverse Gaussian distributed -X ∼ N IG(α, β, δ, µ) -if it has a probability density function of the following form:

f X (x) = αδ π exp[p(x)] q(x) K 1 [αq(x)], (8)
with K 1 (x) the modified Bessel function of the second kind, with index 1;

p(x) = δ α 2 -β 2 + β(x -µ), q(x) = (x -µ) 2 + δ 2 with α > 0, |β| < α and δ > 0.
The expectation, variance, skewness and kurtosis of X have a closed form expression and are:

E[X] = µ - δβ α 2 -β 2 V [X] = δα 2 α 2 -β 2 (9) S k [X] = 3β α 1 δ α 2 -β 2 K u [X] = 3 1 + 4 β α 2 1 δ α 2 -β 2 . ( 10
)
The characteristic function of X is given by:

φ(ω) = exp iωµ + δ α 2 -β 2 -α 2 -(β + iω) 2 , ( 11
)
and the moment generating function of X is:

ψ(ω) = exp ωµ + δ α 2 -β 2 -α 2 -(β + ω) 2 . ( 12
)
The Laplace transform is defined whenever ω ∈ ]βα : β + α[.

Here, we use this distribution to model the conditional distribution of the log-increments of the future rates ∆f (t, T ) introduced in equation ( 6). It is noteworthy that by doing so we have positive rates at any time with probability 1, which is an essential feature of any interest rates model.

In the following, we work with a fixed maturity, thus we use the simplified notation ∆f t , henceforth. Now, the general settings are:

∆f t = σ t ǫ t ,
and we consider the following models:

Model 1: σ t = σ and ǫ t |σ t ∼ N IG(α, β, δ, µ).

Model 2:

σ t = ω 0 + ω 1 σ 2 t-1 + ω 2 ∆f 2 t-1 and ǫ t |σ t ∼ N IG(α, β, δ, µ) Model 3: σ t = ω 0 + ω 1 σ 2 t-1 + ω 2 ∆f 2 t-1 and ǫ t |σ t ∼ N IG(α t , β t , δ, µ)
, with:

α t = κ 0 + κ 1 α t-1 + κ 2 exp{χǫ t-1 } β t = γ 0 + γ 1 β t-1 + γ 2 exp{χǫ t-1 } Model 4: σ t = ω 0 + ω 1 σ 2 t-1 + ω 2 ∆f 2
t-1 and ǫ t |σ t ∼ N IG(α t , β t , δ, µ), with:

α t = κ 0 + κ 1 α t-1 + κ 2 exp{χ 1 ǫ t-1 } β t = γ 0 + γ 1 β t-1 + γ 2 exp{χ 2 ǫ t-1 },
where ǫ t |σ t means that ǫ t is taken conditionally upon σ t . ǫ t conditionally on ǫ t-1 = {ǫ t-1 , ǫ t-2 , ...} is assumed to be a second order stationary process. These models are thus nested, displaying richer and richer dynamics. Model 1 is an homoscedastic NIG model. Model 2 is GARCH(1,1) model with NIG innovations. Model 3 has a GARCH(1,1) variance dynamics and innovations with varying coefficients with the same parameter χ relating the dataset to the parameters' space. Finally, model 4 has χ 1 = χ 2 . In both these models, the parameters of the innovations work conditionally upon σ t . This way, we are able to focus the modeling work on the third and fourth moments, given that the α parameter of the NIG controls the kurtosis and β the skewness.

In an approach with dynamic parameters, the main problem is to choose which power (or functional form) of ǫ t-1 is related to each parameter. In model 3 and 4, we propose to relate the parameters to the conditional spectral moments of ǫ t-1 , i.e. exp{χ . ǫ t-1 }. This allows a direct mapping of the past information ǫ t-1 into the parameters space and requires no a priori knowledge on the proper moment to match. What is more, it may highlight the fact that each parameter of the distribution that is modeled may depend on a different spectral moment, when testing between model 3 and 4 whether it is possible to impose χ 1 = χ 2 = χ. In the NIG case, this dependence of the parameters on different values can be documented. For example, [START_REF] Hanssen | The Normal Inverse Gaussian Distribution: a Versatile Model For Heavy-Tailed Stochastic Processes[END_REF] showed how the NIG parameters are differently related to the cumulant generating function -and thus to the moment generating function. It should be possible to make this dependency over past observation richer, relating α t and β t to ǫ t-1 and ǫ t-2 for example. Due to numerical complexity and to the number of parameters to estimate, we discarded this possibility. Our work is far from the approach used by [START_REF] Jondeau | Conditional Volatility, Skewness, and Kurtosis: Existence, Persistence, and Comovements[END_REF]. They propose several ways to map ǫ t-1 into the parameter space for the Skewed student t distribution of [START_REF] Hansen | Autoregressive Conditional Density Estimation[END_REF] in a GARCH framework again, using several moments or truncated moments; but their approach requires to have preliminary intuitions about the way to truncate series or about the power of the series to consider, which is hardly the case with NIG distribution.

Let us quickly discuss stationarity conditions. Model 1 and 2 yield second order stationary processes whenever the usual conditions on the existence and stationarity of NIG and GARCH processes are fulfilled. In particular, we need to impose that ω 0 , ω 1 , ω 2 are positive and that |ω 1 + ω 2 | < 1. The conditions for model 3 and 4 are derived using the conditional moments of order 1 and 2 of each dynamic parameter. We thus need to impose

|E[β t ]| < E[α t ]. E[β]
does not need to be positive. More precisely5 , for model 3, the conditions are:

|ω 1 + ω 2 | < 1, ω 0 , ω 1 , ω 2 > 0, E[e χǫ t-1 ] exists 6 , κ 1 = 1, γ 1 = 1, sgn(κ 0 + κ 2 ψ(χ)) = sgn(1 -κ 1 ) and |E[β t ]| < E[α t ]
. Similar conditions can be obtained for model 4:

|ω 1 + ω 2 | < 1, ω 0 , ω 1 , ω 2 > 0, E[e χ 1 ǫ t-1 ] and E[e χ 2 ǫ t-1 ] exists, κ 1 = 1, γ 1 = 1, sgn (κ 0 + κ 2 ψ(χ 1 )) = sgn(1 -κ 1 ), and |E[β t ]| < E[α t ].
These conditions are obtained when computing the unconditional expectation of α t and β t . We chose to let κ 0 , κ 1 , κ 2 , γ 0 , γ 1 , γ 2 to be either positive or negative, provided that the above constraints are satisfied, in so far as it is hard to have a precise idea of the domain of definition of these parameters. Constraining these parameters to be positive may be of no harm for the final results, but some of the estimated parameters may be negative without threatening both the conditional and unconditional existence of the process.

Estimation methodology

In this section, we present the estimation methodology used to fit the model on a real dataset. We propose to use a sequential maximum likelihood approach to estimate the parameters of the dynamic models presented in the previous section. We first consistently estimate the GARCH parameters using a pseudo maximum likelihood approach, assuming that the innovations are Gaussian. Then, once the GARCH parameters have been estimated, we estimate the remaining parameters using the conditional maximum likelihood method, using the filtrated GARCH innovations estimated before. The likelihood function is obtained by assuming that the residuals are NIG distributed. The adequation tests performed in Section 2 clearly point toward the use of this distribution.

The conditional log likelihood function associated to the NIG distribution is:

ln L =(T -1) ln δ + T t=2 ln α t + T t=1 δ α 2 t -β 2 t + T t=2 K 1 α t δ 2 + (ǫ t -µ) 2 + T t=2 β t ǫ t σ t -µ -(T -1)log(π) - 1 2 T t=2 ln δσ 2 t + (ǫ t -µσ t ) 2 ,
with K 1 (.) being the previously mentioned Bessel function of the second kind with index 1 and π being approximatively equal to 3.14159. We assume that the usual regularity conditions for the maximum likelihood method to be both consistent and asymptotically Gaussian are verified. The latter expression is maximized using a simulated annealing method, as presented in [START_REF] Belisle | Convergence Theorems for a Class of Simulated Annealing Algorithms[END_REF], given the possible existence of several local optima.

With models 3 and 4, the derivatives of the log likelihood with respect to the parameters are rather involved, because of the Bessel function. We thus propose to estimate the estimates' variance covariance matrix Σ using bootstrap. This matrix will be estimated using the block bootstrap approach. This bootstrap procedure is described in [START_REF] Efron | An Introduction to the Bootstrap[END_REF].

Results

This section is devoted to the analysis of the empirical results: first, we discuss the bulk results: the significativity of the estimates and the log-likelihood ratio tests that were performed for model selection purposes. Then, we present an event study so as to test the ability of the model to capture known features of the subjective distribution.

Main estimation results

The estimation results are presented in table 2 for model 1, in table 3 for model 2, in table 4 for model 3 and in table 5 for model 4. Most of the parameters estimated are significant up to a 5% risk level. The parameters relating α t and β t to past observations are significant, either for models 3 and 4. This is a corner stone for any model with time varying parameters of the kind that is presented here: should it be 0, the parameters α t and β t would be degenerated.

The tables 6 and 7 present Likelihood ratio tests, for model selection purposes. We briefly recall the methodology: for example, let model 3 be the constrained model, with log likelihood denoted lnL c and model 4 be the unconstrained model, with a loglikelihood denoted lnL u . The null hypothesis H 0 χ 1 = χ 2 assumes that the constraint in model 3 can statistically be imposed. The test statistics is then:

LR = 2(lnL c -lnL u ), (13)
with the previous notations. Under the null hypothesis, this statistic has a Chi-square distribution, with a degree of freedom equal to the number of constraints imposed in the constraint model. For model 3 vs. model 4, there is only one constraint:

χ 1 = χ 2 (= χ).
For model 2 vs. model 3, there are 5 constraints:

κ 1 = κ 2 = γ 1 = γ 2 = ξ = 0.
Globally, model 4 is always the one that is favored by the test, bringing support to our approach. As expected, the parameters driving α t and β t are not linked to same spectral moments, in so far as χ 1 is statistically different from χ 2 . What is more, the spectral moment to which they are related is close to 0. This finding is not totally surprising: around 0, the derivatives of the moment generating function are known to deliver information regarding any moments of the distribution.

Like when selecting the distribution, we favor again the model that contains the greater number of parameters. This is however not a real problem since we are not interested in the forecasting ability of the model. Our problem is basically a filtration problem: the conditional distribution of the log increments of the futures is unobservable and we propose to estimate it from the time series at hand. Thus, our concern is the in-sample quality of the fit. In this perspective, the result that the more parameter we have, the better the obtained fit is rather intuitive. However, we cannot increase the number of parameters at will for the numerical feasibility of the estimation strategy: this is why we do not propose a more complex model.

Finally, we propose to use the fourth model to perform an event study so as to assess the global ability of the model to capture key facts about the market perception of monetary policy and back the aforementioned empirical results.

Event study

We propose here to observe the changes in probabilities for the maturities available in the sub-sample dataset during two particular Central Bank decision meetings. These meetings are the ones that occurred on January, 31 st 2006 and August, 8 th 2006. During these days, the Fed announced the new target rate applying until the next Central Bank meeting, along with an economic justification of this decision. In the meantime, Mr. Bernanke delivered speeches concerned with the future evolution of the macroeconomic figures and thus of the stance of the future American monetary policy. These two decision meetings took place in an interesting period, that matches the end of a tightening period, i.e. of a rising target rate period. During the meeting of January, no rate cut were forecast by the bond market, and progressively, over the year, a rate cut forecast progressively appeared. It is very important to assess the ability of our model to capture these changes in the market perception of monetary policy.

The change in the Fed's target rate process is very well documented (see [START_REF] Goodfriend | Interest Rates and the Conduct of Monetary Policy[END_REF] for a detailed analysis stylized facts related to the Fed target rate). The target rate is made of multiple of 25 basis points, i.e. 25/100 percents, and is thus a discrete process. The discreteness of the process is an important feature of monetary policy (see e.g. [START_REF] Guégan | An Econometric Specification of Monetary Policy Dark Art[END_REF] and the references within). We propose here to compute the estimated probabilities associated to these discrete outcomes implied by the continuous and time varying distributions. The computation of these discrete probabilities is made by integrating numerically the continuous distribution over 25 bps sets, using the conditional parameters estimated with model 4. Thus if R T is the future target rate for the maturity date T , then its support S T is such that: .25, 0.5, . . . , 4, 4.25, . . .}. (14) Thus P (R T = k), with k ∈ S T can be computed as:

S T = {0
P (R T = k) = k+0.125 k-0.125 f (x)dx, (15)
were f (.) is the subjective distribution density estimated though the historical distribution of the futures rates of maturity T , following our hypothesis 1. Given that the NIG cumulative distribution function has no closed form expression, the expression in equation ( 15) must be numerically computed (we used trapezoidal integration).

The probabilities for these different meetings are presented in figures 4 and 5. Each figure presents the implicit discrete probabilities before and after the meetings, along with the changes in the probabilities over these decision meetings. Several observations can be made on the basis of this event study:

-First, it appears that the uncertainty is reduced by the meetings, which is a very classical result of the literature dedicated to the impact of "Central Bank transparency". On this point see e.g. [START_REF] Brière | Market Reactions to Central Bank Communication Policies: Reading Interest Rate Options Smiles[END_REF] and the references within. The announcement of the Central Bank decision, along with the chairman's speech brings about a reduction of the volatility in the market. This can be observed by noting that some events close to the mode of the distribution are getting higher probabilities whereas others have lower ones, after the meeting. The subjective distribution is thus getting more concentrated around its expectation. This is true for the two decision meetings that are observed.

-Second, over the two meetings, the skew of the distributions changed progressively. In January, the skew of the subjective distributions is positive, underlining the fact that the market is still believing in the increase of the Central Bank target rate over the futures' maturities that are considered. On the contrary, on the August decision meeting, the skew turned out to be negative. At this time, the Central Bank communication made the market believe that no more rate increases were to be feared. Rate cuts were now forecast by the market, underlining the change in economic conditions in the US economy: a slowing economic growth and a slowing inflation.

Beyond this event study, an economic interpretation of several parameters is also possible. First, the parameter controlling the level of persistence of the α t process (that is κ 1 ) is higher than that of the β t (that is γ 1 ) process over every sample: the kurtosis is a more persistent process than the skewness one. More, for both α t and β t the estimated value of κ 1 and γ 1 are remarkably stable across samples. The signs associated to the estimated γ 0 and κ 0 yield interesting pieces of information: κ 0 (resp. γ 0 ) is a function of the unconditional expectation of α t (resp. β t ). Over each dataset, κ 0 remains positive and it is increasing from December to March, revealing a growing level of uncertainty in the financial market perception of the upcoming Fed's decisions: α t is explicitly related to the thickness of the tails of the subjective distribution and thus to the probability of appearance of extreme values, that is extreme monetary policy decisions. On the contrary, γ 0 changes sign when comparing the December-January estimations to the February-March period, moving from a positive sign to a negative one. This is consistent with the change in the skewness sign remarked during the event study: over months, upcoming rate cuts seemed to be more and more likely to happen. These results are globally confirmed by the observation of the time varying conditional moments obtained with model 4. The computation of these moments can be made using the moments closed form expressions previously mentioned. We present it on figure 3: the red mark signals the two Central Bank decision meetings studied earlier.

The previous observations still holds when observing the figures: the evolution of the skewness changes after the August meeting, confirming what has been presented ear-lier. Thus, by looking at these results, the approach developed here seems to be likely to provide an accurate measure of the financial markets' view of the future stance of monetary policy. A tool of this kind should be useful to both monetary policy makers and to market analysts.

Conclusion

In this paper, we developed a time series model to infer the subjective distribution from the time series of the fed fund future contracts. To do so, we use the link that we introduced between the dynamic parameters and the moment generating function.

The empirical results obtained confirm the interest of this approach. What is more, the assumption that the historical distribution of futures should be close to the subjective distribution of the future short rate is empirically realistic and the model proposed presents empirical performances that validate it. This assumption can thus be used for more general study around the estimation of the subjective distribution, and even extended to the econometrics of stochastic discount factors, given that most of the futures are backed to American options.

Appendices

α t = κ 0 + κ 1 α t-1 + κ 2 exp{χǫ t-1 } β t = γ 0 + γ 1 β t-1 + γ 2 exp{χǫ t-1 }.
The standard deviation of the estimates are reported between brackets under the estimates. Standard deviations have been computed by block bootstrap.

α t = κ 0 + κ 1 α t-1 + κ 2 exp{χ 1 ǫ t-1 } β t = γ 0 + γ 1 β t-1 + γ 2 exp{χ 2 ǫ t-1 }.
The standard deviation of the estimates are reported between brackets under the estimates. Standard deviations have been computed by block bootstrap. The R 2 is obtained when estimating the linear model defined by equation ( 7) by Ordinary Least Squares with the following explanatory variables:

L(T i ) = log R(T i ) (16) ∆L(T i ) = L(T i ) -L(τ i ) (17) |∆L(T i )| = |L(T i ) -L(τ i )| (18) S(T i ) = L(T i ) -L . ( 19
)
Global R2 presents the R 2 obtained when using the four variables defined in equations ( 16), ( 17), ( 18) and ( 19). Level presents the R 2 obtained when using only the explanatory variable presented in (16). Delta R2 presents the R 2 obtained when using only the explanatory variable presented in (17). |Delta R2| presents the R 2 obtained when using only the explanatory variable presented in (18). Spread R2 presents the R 2 obtained when using only the explanatory variable presented in (19). This figure presents the T × R 2 test for autoregressive higher order moments. It presents the test statistics obtained for each Fed fund futures contracts, with maturity ranging from November 2000 until March 2007. The test statistics were computed over the whole lifespan of the future contracts but the last two months of it, due to stationarity concerns. The plain black line indicates the χ 2 quantile used to test the null hypothesis.

Figure 1 :

 1 Figure 1: R 2 study for Fed fund futures contracts, using 77 contracts of maturities ranging from November 2000 until March 2007.

Figure 2 :

 2 Figure 2: Average (annualized), volatility (annualized), skewness and kurtosis of the log-returns of full sample.

Figure 6 :

 6 Figure 6: Engle (1982)'s T × R 2 test for autoregressive moments.

Table 1 :

 1 P-value associated to a Kolmogorov Smirnoff test. The Kolmogorov Smirnoff tests were performed over the Fed fund future contracts for the Skewed Student, the Skewed Laplace, the NIG and the Normal distributions, for Fed fund future log daily increments with maturities ranging from November 2000 until June 2007.

		Skewed Student Skewed Laplace NIG Gaussian
	# of test acceptation	39	65	73	36
	# of test rejection	38	12	4	41
	Total	77	77	77	77

Table 4 :

 4 Subjective distribution: Estimation Result for model 3. The model 3 is defined by σ t = ω 0 + ω 1 σ 2 t-1 + ω 2 ∆f 2 t-1 and ǫ t |σ

t ∼ N IG(α t , β t , δ, µ), with:

Table 5 :

 5 Subjective distribution: Estimation Result for model 4. The model 4 is defined by σ = ω 0 + ω 1 σ 2 t-1 + ω 2 ∆f 2 t-1 and ǫ t |σ t ∼ N IG(α t , β t , δ, µ), with:

The risk neutral distribution is the distribution used to give a price to financial assets.

The subjective distribution is a forecasting density of r Ti .

sgn(x) is the sign function: -1 if x < 0 and 1 if not.

The conditions of existence for the moment generating function can be found in e.g.[START_REF] Hanssen | The Normal Inverse Gaussian Distribution: a Versatile Model For Heavy-Tailed Stochastic Processes[END_REF].

and to the participants of the French

4.0 4.5 5.0 5.5 6.0 6.5 0.0 0.3

Prob. after meeting

Janurary 2007

Target Rates

Prob.

4.0 4.5 5.0 5.5 6.0 6.5 -0.005 0.020

Prob. variation

Janurary 2007

Target Rates

Prob.

4.0 4.5 5.0 5.5 6.0 6.5 4.0 4.5 5.0 5.5 6.0 6.5 -0.002 0.008

Prob. variation

March 2007

Target Rates

Prob.


Haraldur Audunsson

Siegfried Rouvrais

Ralph Rudd

Ragnar Kristjánsson

Matthieu S Olivier

Moschetta

DOES A MASTER'S PROGRAM IN ENGINEERING REQUIRE A FINAL PROJECT?

Keywords: Final project course, dissertation, thesis, learning outcomes, rubric, Master of Science. Standards: 2, 5, 8

Within engineering education frameworks worldwide, requirements for a master's degree are diverse and very few graduate-level engineering courses are recommended for accredited programs. To ascertain how common the requirement for a final project at the master's level is, an ad hoc review of international master's programs was conducted. This review included several of the highest-ranking universities internationally and selected universities in Europe. From this review, it is established that the standard practice is to require students attempting a master's degree in engineering to complete what we term a final project course, which may or may not be research-focused, and typically corresponding to one semester of work. This paper summarizes how the considered universities integrate a final project course into their programs and distinguishes how these might differ from traditional research-focused master's dissertations. We discuss the practical difficulties of managing such projects. We conclude by providing a rubric for self-assessment and reform that aligns with the criteria for continuous improvement in a graduate program quality framework.

INTRODUCTION

An inherent goal of engineering education is to prepare graduates for the challenges they may face as professional engineers in the workplace. Educational programs will prepare students differently, depending on the needs, traditions and cultures in the relevant country as well as the values of the specific university. Therefore, engineering programs vary (usually within accreditation constraints) and thus the graduating students will have distinct nuances to their list of graduate outcomes.

One relevant skill is the student's ability to complete large, challenging and complex projects, where the student is required to incorporate diverse discipline-specific skills, as well as both personal and interpersonal skills. Traditionally, this particular attribute has been trained and Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15, 2022. evaluated using a final project course (FPC) positioned in the educational program towards the end of the master's qualification.

As far as possible, the learning outcomes of the FPC in engineering should reflect the main areas of the future engineer's transversal practical skills, as emphasized by [START_REF] Kamp | Engineering Education in the Rapidly Changing World[END_REF] and outlined in Figure 1. These capabilities should also be inherent in the learning outcomes and in any potential rubric for engineering departments' self-assessment of the FPC.

Final project course (FPC)

In this paper we use the term final project course, as a general term to capture all the different formulations of the large, challenging and complex project a master's student may be required to complete. It is often also referred to as a final research project, final-year project, final design project, capstone project, terminal project or final internship. Typically, this FPC corresponds to one semester (approximately 30 ECTS) and is most often placed near the end of the educational program at the master's level.

We distinguish between two classes of FPC: research focused and non-research focused. The research-focused class contains the master's dissertation1 , traditionally found in the sciences, where the non-research focused class contains all formulations of the FPC which do not explicitly develop the engineering student's ability as a researcher and are typically designfocused.

The CDIO framework and the FPC

In accreditation reference and orientation guides, and in educational frameworks like the CDIO, there exist few guidelines about structural requirements of an engineering education at graduate level, and, as such, the requirements for an FPC worldwide vary significantly.

This can be contrasted with the guidelines available for undergraduate engineering education.

The CDIO international framework suggests, in one of its standards (number 4), the course "Introduction to Engineering". As a good practice, it is recommended that this course is placed early in the curriculum structure thus engaging freshman students in the practice of engineering through problem solving and simple design exercises, preferably in teams. The course also includes personal and interpersonal knowledge, skills, and attitudes that are essential at the start of a course [START_REF] Rouvrais | A Starter Activity Design Process to Deepen Students Understanding of Outcome-related Project Learning Objectives[END_REF] or program to prepare students for more advanced product, process, system, and service building experiences.

In the Design-Implement Experiences standard (number 5), opportunities to conceive, design, implement and operate products, processes, systems and services are suggested for inclusion in required co-curricular activities. For example, these opportunities should be made available in undergraduate research projects and internships at the end of the program.

Regardless of these skills being introduced and developed at undergraduate level, many leading international institutions require a further FPC at the master's level. Furthermore, there is a significant variation in the offered FPCs learning outcomes, focus and content.

In this paper we review a dozen engineering programs at international institutions and universities that culminate in a professional engineering degree at the master level, in an effort to ascertain how common a FPC is, its size (as measured in ECTS credits) and its formal intention. After summarizing the results, we discuss various aspects of the FPC, and make recommendations on its learning outcomes. Furthermore, we suggest a maturity rubric [START_REF] Rouvrais | An Assessment Framework for Engineering Education Systems[END_REF], as exists in the CDIO framework, to formalize the evaluation of the implementation quality of the FPC. In practice, engineering programs consider FPC a necessity and therefore the CDIO consortium may want to extend its framework to harmonize FPCs at participating universities and institutions.

SURVEY OF FPC REQUIREMENTS

At the beginning, we set out to ascertain how common a FPC is at the senior or master's level and map the similarities between the requirements where possible. The survey was conducted as an ad hoc review including some of the top-tier universities internationally and selected universities in Europe. The data was inferred from webpages of the programs, curriculum handbooks and sometimes by private communications. There are variations on how FPC is organized and therefore some details in Table 1 are institutionally dependent and prone to ambiguities. The criteria was that students graduating from the program were able to apply to become chartered engineers or professional engineers. This typically meant that the students had completed a BSc degree in engineering, or related field, and then one to two years at the MSc level.

The results are summarized in Table 1.

Based on the ad hoc survey summarized in Table 1, we may conclude that the norm at many leading international institutions and universities offering engineering degrees is that students at the master's level complete a FPC. This course can be a research or design activity, and is most often 30 ECTS but 24 and 60 ECTS variations were also observed. There is a diversity of its intended learning outcomes and style, but the vast majority of the surveyed institutions require a research-focused dissertation. Details on the FPC variations across the different surveyed institutions are provided below.

Variations in the FPC

When considering some of the top-ranking engineering institutions worldwide, MIT, Oxford and Cambridge all require a research-focused dissertation in their master's degrees. At Stanford and NTU, there are both research and non-research focused FPCs, but the non-research focused variants can only result in the award of a Master of Science (MSc) degree. Thus, all five of the top ranking institutions considered require the student to complete a research dissertation to obtain a Master of Engineering (MEng) degree or Engineer's degree (ED).

All engineering programs in Iceland require a master's dissertation, which is most often 30 ECTS in size. In the occasions where the dissertation constitutes 60 ECTS of the degree additional emphasis is placed on making an original research contribution, which is not typical at the master's level.

Throughout the rest of Scandinavia, almost all the surveyed programs require a 30 ECTS master's dissertation, with a 60 ECTS variant available at Chalmers. The exception is an option at Aalto which offers an "Aalto Thesis" FPC where "2-4 students from different fields form a team for a 6-month project to solve a work-life partner's real and complex challenge through their master's thesis", but currently this option is on a break.

In France [START_REF] Rouvrais | Work-based Learning Models in French Engineering Curricula: Insight from the French experience[END_REF] the FPC is a structured internship in the industry, lasting 4 to 6 months, resulting in a final report. The student writes a report, evaluated by the company advisor, a faculty member and an external evaluator, and then there is a formal defense. This internship is the last course in the program (e.g. min 24 ECTS), other shorter internship periods exist from freshman level.

The emerging Skolkovo Institute of Science and Technology was founded in 2011 in a partnership with MIT and is based on the CDIO vision. At Skolkovo a significant part of the MSc program in engineering is devoted to a "Research and MSc thesis project" (36 ECTS out of 120 ECTS total for the MSc program, see www.skoltech.ru). The emphasis at Skolkovo is very much in alignment with the programs listed in Table 1.

Table 2. Rubric for self-assessment for a master's-level engineering final project course (FPC).

Maturity Scale Criterion 5

The final project course is regularly monitored, evaluated and revised with respect to curriculum integration, learning outcomes, supervision and professional experience, based on feedback from students, instructors and other stakeholders. 4

There is documented evidence of the impact of the implementation of the final project course according to the integrated curriculum plan and constructive alignment principles.

3

FPC is being implemented across the curriculum according to the integrated curriculum plan and supervision requirements. 2 FPC has been approved by stakeholders, implemented as a research lab work, industry partnership, design or research project, with learning outcomes that train professional activities that integrate personal, interpersonal, conceiving, designing, implementing and operating skills and competencies with disciplinary knowledge. 1

A curriculum analysis has been conducted to identify the need for a FPC to synthesize competence in discipline and personal skills benchmarked with the integrated curriculum. 0

There is no evidence of a final large scale project course at the MSc level engineering program.

CONCLUSION

The review presented in this paper shows that the norm at several leading universities is that students complete a final project course (FPC) near the completion of the engineering program at the master's level, being a substantial part of their program, typically equivalent to one semester of work or 30 ECTS, and in some cases even 60 ECTS. This is inferred from an informal ad hoc survey of a dozen universities in several countries, including five top-ranking engineering institutions.

All five of the top ranking institutions considered, see Table 1a, require the student to complete a research-focussed dissertation to obtain a Master of Engineering (MEng) degree or Engineer's degree (ED). In most of the engineering programs in Scandinavia, Table 1b, students have to complete a research-focused dissertation to obtain a Master of Science degree. In France, full collaboration with industry is a must for the FPC. Within the CDIO educational framework, there is no obvious requirement for a final project course, but rather a group of courses that involve conceive, design, implement and operate.

The main purpose of the FPC is to synthesize competence in discipline and personal skills as benchmarked with the integrated curriculum and prepare the student for engineering professional activities. The FPC can be implemented as a research lab work, industry partnership, design or an applied research project. Because the project is the student´s signature work, the assemblage of several such projects is one of many gauges on the department's output and provides significant contribution when reviewing engineering programs.

The suggested rubric (Table 2) for quality maturity of a master level engineering FPC is based on the CDIO educational framework, the placement of the FPC in the program and stakeholders interest, and the learning outcomes. The rubric is for programme level selfassessment, including mapping the process maturity level and state of adoption, as well as for continuous improvement. The proposed FPC rubric is in line with the ubrics used for evaluating the twelve CDIO standards.

It is evident that several leading universities consider FPC a necessity and in an effort to harmonize its contribution to engineering education the CDIO consortium may want to consider recognizing the FPC and include its contribution in the CDIO framework.

Figure 1 .

 1 Figure 1. Five main areas of the future engineer's transversal capabilities (Adapted from Institut Mines-Telecom: Portrait de l'ingénieur 2030 (In French, 2014), www.imt.fr/limt-presente-le-portrait-de-lingenieur-2030/).

We further distinguish between a dissertation, which occurs at master's level and does not require an original research contribution (instead it disserts a specific topic), and a thesis, which occurs at PhD level and does have an originality requirement (by definition).Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15,

Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15, 2022.

SUPERVISION AND MENTORING PROCESSES FOR FPC

The role of the supervisor(s) for the FPC is to guide the student throughout the whole project and be supportive when needed, with the learning outcomes serving as the guideposts. The supervision should focus on the student's expertise and discipline, stimulate the student's ingenuity and agility, and in an effort to prepare the student for his professional life. The supervisor should, at least implicitly, make the student aware of his responsibility as an engineer and the influence he or she may have as an engineer in the future [START_REF] Kamp | Engineering Education in the Rapidly Changing World[END_REF].

Workload

The FPC is typically a significant part of the engineering program (30 to 60 ECTS), and may therefore require advising and/or supervision from faculty members. In such courses the program-level leaders of engineering departments are concerned with how to balance the workload on the faculty and external stakeholders while maintaining training and supervision quality and the autonomy of the learner.

The supervision is multi-faced and can be done either by an individual or by a small team, and the supervisor has to be aligned with the type of setting the student is working in, be it within the university or in an internship. Due to the many facets of the supervision and mentoring, the university may want to complement the advising, as for example outlined by [START_REF] Saalman | Lessons learned from developing and operating a large-scale project course[END_REF]. This may include pedagogical tutors, writing workshops and facilitating collaboration teams to make the students journey through this often challenging final course more fruitful (Audunsson et al., 2018) and as a discussion forum on different modes of how to approach the report writing [START_REF] Hakkala | Refining engineering MSc theses with a focus enhancing structure model[END_REF].

To formalize and streamline the advisory process the department may set up a formal checklist with the learning outcomes and a sequence of milestones to facilitate time management. Wellprepared learning outcomes facilitate the assessment activities [START_REF] Rouvrais | Architecting the CDIO Educational Framework Pursuant to Constructive Alignment Principles[END_REF][START_REF] Valderrama | Guidelines for the final year project assessment in engineering[END_REF] and may aid the advisor and inform the student of the expectations during the thesis work.

Quality assurance

In addition to general quality quality assurance systems within engineering departments and institutional and external qualification framework, departments may want to consider additional requirements. The final project is a signature work by the student and reflects the quality of the educational program. Therefore one option is to mandate that the final report is open to the public and other institutions. For example, in Iceland all final reports at the MSc-level are placed in a web-based depository open to all, and the only exception is if the report contains confidential information, including market or industrial advantages. In this case, the public release of the report will be delayed for an appropriate time period. Another quality-assurance check worth considering is to have an open presentation of the project work when completed, sometimes referred to as a dissertation defense, although the term defense may not be appropriate at this level. A view from the student's side was discussed by [START_REF] Kindgren | Using students' reflections on program goals after master´s thesis as a tool for program evaluation[END_REF]. In their paper they outlined how reflection documents submitted by students after completing a master's thesis could be used as a tool for program evaluation.

Proceedings of the 18 th International CDIO Conference, hosted by Reykjavik University, Reykjavik Iceland, June 13-15, 2022.

FPC LEARNING OUTCOMES

The main purpose of the FPC is to synthesize competence in discipline-specific and personal skills as benchmarked with the integrated curriculum plan. The different forms of the FPC have been highlighted, with this paper emphasizing the distinction between those that are research focused (and thus require the student to develop capacity as a researcher) and those that are not.

The learning outcomes of this final course should focus on training engineering professional activities that integrate personal, interpersonal, conceiving, designing, implementing and operating skills and competencies with disciplinary knowledge. Thus, in effect, the learning outcomes should reflect that this is the final training effort by the program to prepare the student for the workplace. The specific learning outcomes may be country specific, university or discipline specific and reflect the needs of the society in addition to the values and vision of the university. Furthermore the learning outcomes should be aligned with the CDIO framework, i.e. Standard 2, and be the culmination and synthesizing of previous courses that involve conceive, design, implement and operate.

Should Masters of Engineering be trained researchers? Not necessarily, but they should be capable of leading, managing and reporting on large, complex projects. Therefore, the learning outcomes for the final project should reflect the difference between a degree in engineering and traditional research-led master's dissertation for science degrees or future PhD students.

These objectives and learning outcomes are integrated in the rubric in Table 2, and are the cornerstone to constructive alignment with FPC activities and assessment modes [START_REF] Rouvrais | Architecting the CDIO Educational Framework Pursuant to Constructive Alignment Principles[END_REF].

REFERENCE MODEL AND RUBRIC FOR MSC FINAL PROJECT COURSE

In alignment with the CDIO principles and best practice at the master's level in engineering, we present in Table 2 a rubric for self-assessment of master's-level FPC. It includes process maturity levels to meet the coherent adoption and continuous improvement [START_REF] Rouvrais | An Assessment Framework for Engineering Education Systems[END_REF].

BIOGRAPHICAL INFORMATION

Haraldur Audunsson