Laure Berti-Équille
email: berti@irisa.fr

Measuring and Constraining Data Quality with Analytic Workflows

One challenging aspects of data quality modeling and management is to provide flexible, declarative and appropriate ways to express requirements on the quality of data. The paper presents a framework for specifying and checking constraints on data quality in RDBMS. The evaluation of the quality of data (QoD) is based on the declaration of data quality metrics that are computed and combined into so-called QoD analytic workflows. These workflows are designed as a composition of statistical methods and data mining techniques used to detect patterns of anomalies in the data sets. As metadata they are used to characterize various quantifiable dimensions of data quality (e.g., completeness, freshness, consistency, accuracy). The paper proposes a query language extension for constraining data quality when querying both data and its associated QoD metadata. Probabilistic approximate constraints are checked to determine if the quality of data is (or not) acceptable to build quality-constrained query results.

INTRODUCTION

. Maintaining a high level of data quality in a database is challenging and cannot be limited to one-shot approaches addressing simpler and more abstract versions of the wide

range of data quality problems. Technically, data quality management mainly refers to the detection and elimination of various data quality problems, such as:

-Duplicate and redundant data. A wide range of techniques have been proposed for record linkage [START_REF] Winkler | Overview of Record Linkage and Current Research Directions[END_REF] and entity resolution since the "merge/purge problem" identified by Hernández and Stolfo [START_REF] Hernández | Real-world Data is Dirty: Data Cleansing and The Merge/Purge Problem[END_REF],

-Imperfect data. Inconsistency, imprecision, and uncertainty are some of the problems associated with data imperfection [START_REF] Parsons | Current Approaches to Handling Imperfect Information in Data and Knowledge Bases[END_REF]. A significant amount of work has been proposed in the areas of integrity constraint checking, imprecise data management, and uncertainty in DBMSs, e.g., [START_REF] Andritsos | Clean Answers over Dirty Databases: A Probabilistic Approach[END_REF][START_REF] Benjelloun | Databases with uncertainty and lineage[END_REF][START_REF] Dalvi | Management of Probabilistic Data: Foundations and Challenges[END_REF],

-Missing values and incomplete database. The problem of handling incomplete information has also been addressed in information and database systems [START_REF] Naumann | Completeness of Integrated Information Sources[END_REF],

-Stale data. Various refreshment techniques and synchronization policies have been proposed for ensuring data freshness depending on the type of system architecture [START_REF] Bouzeghoub | A Framework for Analysis of Data Freshness[END_REF][START_REF] Peralta | Data Quality Evaluation in Data Integration Systems[END_REF]: e.g., data warehousing systems check the recentness of materialized views; caching systems estimate the time-to-live of cached data before expiration and tune the caching policy for ensuring data currency.

QoD dimensions are numerous with various definitions, interpretations and measurement methods depending on the considered application domains. One of our goals is to cope with this diversity in proposing a flexible way to express the dimensions of data quality and to associate relevant constraints. Among other challenging research directions that have been recently identified in Data Quality Research [START_REF] Berti-Équille | Quality Awareness for Data Management and Mining[END_REF] (e.g., pattern design for quality-aware IS engineering [START_REF] Akoka | A Framework for Quality Evaluation in Data Integration Systems[END_REF], new perspectives of methodological approaches for data quality management [START_REF] Batini | Data Quality: Concepts, Methodologies and Techniques[END_REF] or benchmarks for comparing the research contributions to specific data quality problems [START_REF] Weis | A Duplicate Detection Benchmark for XML (and Relational) Data[END_REF]), QoD metadata modeling and management and QoD-aware query languages are indeed of particular interest for the community. There is currently no model that captures in a precise and easy way the semantics of all static and dynamic dimensions of data quality, and allows carrying out relevant automatic checking based on QoD metadata management. The existing propositions focus on one or two "hard-coded" QoD dimensions (often considered separately) [START_REF] Mendès-Sampaio | Incorporating the Timeliness Quality Dimension in Internet Query Systems[END_REF][START_REF] Guo | Caching with 'Good Enough' Currency, Consistency, and Completeness[END_REF][START_REF] Lembo | Source Inconsistency and Incompleteness in Data Integration[END_REF][START_REF] Peralta | Data Quality Evaluation in Data Integration Systems[END_REF]. The approach which consists in defining default QoD dimensions and in basing the analysis on the Cartesian product or on the ad-hoc composition of some data quality dimensions do not make it possible to model, in a faithful way, the interdependencies between these QoD dimensions [START_REF] Akoka | A Framework for Quality Evaluation in Data Integration Systems[END_REF]. This led to vagueness of analysis results, difficult to conciliate and whose practical utility is very low for a complete data quality diagnostic. It is thus necessary to develop metadata models and declarative data manipulation languages which make it possible to represent combinations and interdependencies of user-defined QoD dimensions with relevant measurement techniques and constraints. As a first step in this direction, the paper presents a framework and a QoD-constrained language for integrating "natively" the specification, evaluation and checking of data quality in the data management system [START_REF] Berti-Équille | Quality Awareness for Data Management and Mining[END_REF]. The emphasis of the paper is put on:

• QoD Evaluation. We define and implement QoD analytic workflows for computing of a set of statistical and data mining functions that evaluate various dimensions of data quality. The results of these functions are stored in a metadata repository as QoD measures characterizing data distribution properties, data quality problems, anomaly patterns, and any useful information reflecting QoD evaluation for different granularity levels of a RDBMS (i.e., for the values, tuples, attribute domains, tables or the database),

• QoD-aware data management. We present a query language extension that allows the declaration and assignment of contracts on the quality of data as sets of constraints on the QoD measures resulting from the analytic workflows.

The paper is organized as follows: Section 2 presents the main steps of our approach based on the design of analytic workflows for QoD evaluation. Section 3 presents the syntax of the language extension we propose for measuring and checking constraints on data quality. Section 4 briefly presents the prototype we've developed to implement the approach. Section 5 presents related work on declarative languages dedicated to data quality control and management. Section 6 concludes the paper and presents our current research perspectives.

A FRAMEWORK FOR EVALUATING

AND CHECKING QOD

Overview

Our approach can be summarized through the following steps:

1. Definition and computation of analytical functions for QoD evaluation. First, the design of analytic workflows consists of the definition (or eventual reuse) of functions that measure objectively various dimensions of the quality of data (e.g., freshness, consistency, completeness, accuracy); each dimension may characterize an aspect of the quality of a database object instance (i.e., value, tuple, attribute domain, table or database). The computed measures are stored and managed as QoD metadata in a repository. One QoD dimension of a DB object instance can be characterized by many QoD measures in the metadata repository.

2. Definition of probabilistic and approximate constraints on QoD measures. To establish a QoD diagnostic, measured QoD values have to be compared to expected QoD values with a certain degree of tolerance. To do so, probabilistic and approximate constraints are specified and checked to determine if the quality is acceptable or not for each QoD dimension. The computed probabilities are stored in the metadata repository and refreshed by the QoD metadata management system.

3. Quality-constrained query declaration and processing. Probabilities assigned to the QoD dimensions are then used in the query processing to build quality-aware query results (or QoD diagnostics) in conformance with the QoD constraints predefined in the analytic workflows.

From the system-centric perspective, our approach has been translated into three main components that pragmatically integrate data quality awareness in the system: i) an extensible library of functions and statistical methods for measuring various aspects of QoD, ii) a quality metadata manager that stores, indexes, searches, and refreshes QoD measures and related descriptive metadata, and iii) an extended query engine that allows declaration and manipulation of data with probabilistic approximate constraints on QoD metadata.

Designing QoD Analytic Workflows

In order to analyze and evaluate the quality of data, a relevant sequence of tasks are specified and planned depending on the underlying goal of the QoD evaluation. For this purpose, we define QoD analytic workflows as sets of parameters interacting with each other. These include: a goal (G), a set of tasks (T ), a set of resources (R), a set of resource allocations (A), and a set of limitations (L). A QoD analytic workflow, W , is then a function of all the sets interacting with each other, as: W = {G, T, R, A, L}. The goal G of an analytic workflow is defined for a DB object instance resource in order to characterize it and to analyze a variety of data quality issues for a specific purpose. A goal has three components: i) the data instance resource (e.g., a set of rows or values), ii) the data quality issue (e.g., data consistency, freshness or completeness), and iii) the purpose (e.g., detect outliers or correct anomalies). The set of tasks (T ) are the building blocks of the analytic workflow. Tasks can be broken down into smaller tasks through task refinement. This activity continues until a satisfied level of abstraction has been achieved for that particular QoD analytic workflow being modeled. The set of resources (R) include the set of inputs of the analytic workflow, such as the set of DB object instances to analyze (e.g., attribute domain, tuple, value), the set of analytical functions to be applied for QoD evaluation, and the set of output QoD metadata including QoD measures (i.e., outputs of functions) and descriptive metadata (i.e., detailed description of the settings and parameters of the functions).

Table 1 gives a classification of the analytical functions that are used for QoD evaluation; the mention from level I to level IV indicates the increasing range of complexity of the methods. It also provides examples of the functions used for computing measures related to the following data quality dimensions: completeness (CP), consistency (CT), accu- These techniques have the same goal as the previous methods, in the sense that they can be used to detect data glitches (e.g., duplicates, anomaly patterns, and dubious data), but their computation and maintenance costs are much higher.

U

DupDetectionProb returns a decimal number in [0,1] that represents the probability of the DB object instance to be a duplicate after clustering and association rule mining on a combination of attributes whose values are identical or similar.

R

Table 1: Categories of Functions for QoD Evaluation racy (AC), freshness (F) and uniqueness (U) (i.e., absence of duplicates). The computed measures are respectively associated to the instances with the level of granularity which the measure is computed from, namely the value (V ), record (R), attribute domain (A), table (T ) and database (D) (see the last column of Table 1). The granularity levels correspond to the main structural elements of the relational DB; metadata are thus associated to each DB object instance depending on its granularity level.

The reader is invited to read the chapter 2 of [START_REF] Berti-Équille | Quality Awareness for Data Management and Mining[END_REF] to have a more detailed description of the functions that can be used in the QoD analytic workflows.

The set of resource allocations (A) defines the relationship of the tasks and the relationship of the resources, as well as the task/resource allocations. These relationships are given at the time the resources and tasks are defined. The set of limitations (L) defines any limitations or restrictions imposed on the tasks and resources. Such restrictions may include scheduling restrictions (with precedence constraints in task planning), resource restrictions (e.g., resources A and B are mutually exclusive), resource allocation restrictions, (e.g., a function f cannot be applied for task t), and so on. Once sets of resources and their relations are defined in the QoD analytic workflow model, the data and functions become inputs to the tasks and QoD metadata become output. Output from one analytical function could serve as input to other functions; the same resource can serve either as input or output, or both depending on which task it applies to. Figure 1 presents the example of a QoD analytic workflow designed for evaluating the quality of data of the database named CRM DB composed of two tables: PROD-UCT and CUSTOMER. This includes several tasks of evaluation on three QoD dimensions, namely freshness, accuracy, and completeness at different granularity levels (e.g., cell, row, column, table). Data object instances are the inputs of the tasks. Each task may be composed of several subtasks with allocated functions for the computation of the QoD measures. Summary statistics for numerical values are computed, out-of-range data values may also be detected with univariate statistics and percentiles (IQR) e.g., 'sas func iqr outlierProb.sas' of the subtask outlierProb composing the task ACCURACY evaluates the accuracy at the CELL granularity level of CRM DB. The execution of each allocated function generates QoD measures that are stored in the metadata repository as QoD matrices represented in Figure 1 as {Q}. The detailed description of each function is stored as descriptive metadata in the repository using PMML1 represented as [D] in Figure 1.

QoD analytic workflows combine multiple analytical functions as the ones illustrated in the example of Table and

DECLARATION OF CONSTRAINTS ON DATA QUALITY

In our approach, constraints on data quality are grouped and expressed by means of quality contract types and quality contracts instances:

1. A quality contract type defines a set of quality dimensions, measures and functions associated to a particular database object instance. It corresponds to the computation tasks of a particular QoD analytic workflow.

2. A quality contract instance is a set of one-sided range constraints defined on the QoD dimensions declared in the contract type.

Declaration of QoD Contract Types

The syntax of creation of a quality contract type is given in Figure 2. A contract type is named (ct name) and assigned to a given schema of the database. Each contract type is composed of a list of named measurable dimensions, the datatype of the output, the granularity level which the measure is associated to (e.g., ON CELL, ROW, COLUMN, TABLE, DATABASE) and the identifier or name of the function that computes the measure following the BY FUNCTION statement. QoD measures are stored in the metadata repository and they are assigned either to: i) a global granularity level, i.e.

ON CELL, ROW, COLUMN, TABLE, or DATABASE, or to: ii) a specific DB object instance, i.e., an existing table, column, record, or cell. For each row in the database, the ROWID pseudo column returns a row's address.

The analytical function may be a PL/SQL procedure or the call specification of a Java, C or SAS program. The creation of a task in the QoD analytic workflow can be associated to the creation of a contract type on a database object instance. This leads to the execution of the declared analytical functions, the computation and storage of the measured QoD values and the associated PMML descriptions of the functions in the metadata repository. As an example, Table 2 gives the syntax for creating the quality contract types resulting from the tasks of the QoD analytic workflow illustrated in Figure 1. For instance, in FRESHNESS contract type, updateFreq is a float returned by a JAVA function named 'java func updateFreq.java' that indicates how frequent the rows and tables have been updated since their creation time. In ACCURACY contract type, outlierProb is a float returned by a SAS function named 'sas func iqr prob.sas' that returns the probability of being outlier for each value of the database. In COMPLETENESS contract type, the PL/SQL procedure plsql func nullValues returns a percentage of missing values

CREATE

Declaration of QoD Contract Instances

The syntax of creation of a quality contract is given in Figure 3. Each contract declaration refers to an existing contract type (ct name). It defines the constraints on each contract type dimension with simple or composed expressions using basic binary operators. Consider again the CRM DB database and the QoD analytic workflow given in Figure 1, three quality contract types have been defined for characterizing freshness, accuracy, and completeness on CRM DB.

Table 3 gives examples of quality contract instances, named fresh, accurate, and complete whose definition is based on their respective contract types: FRESHNESS, ACCURACY, and COMPLETENESS given previously. Contract instances describe the one-sided range constraints to be checked in conformance with the expected values on each dimension declared in the corresponding quality contract type. The declaration of contract types and instances is a part of the QoD analytic workflow specification. Our objective is to incorporate and use a set of analytical functions for computing QoD measures that can be easily extended by other user-defined functions. The call and execution of functions is triggered immediately after the validation of the contract type declaration in the workflow design. Constraints are checked on the defined granularity levels or on the particular database object instances immediately after the validation of the contracts declaration when the workflow is executed.

QoD Acceptability

The constraints are based on the comparison between measured and expected values for each QoD dimension. The degrees to which these constraints are satisfied are aggregated in order to compute a value called acceptability assigned to each considered QoD dimension associated to a particular DB object instance. This value represents the probability that the QoD dimension is acceptable considering that the values of its associated QoD measures are in conformance with expected values given in the constraints. More formally, given a particular QoD dimension Q i , the QoD acceptability of a DB object instance indicates the likelihood δ i that the QoD dimension measured for the database object instance o is acceptable considering the constraints c ij on its associated QoD measures m ij being satisfied with respect to an interval of expected values M ij with a tolerance ij . Acceptability of QoD dimension Q i on the DB object instance o, noted Acc i (o) is then defined as:

Acc i (o) = P r(Q i (o)| j c ij : m ij ∈ [M ij ± ij ]) = δ i . ( 1
)
In our current implementation, δ i is defined as the weighted sum of all distances between measured and expected values used to characterize the QoD dimension Q i , as:

δ i = j w ij .d ij with j w ij = 1 and
d ij = ⎧ ⎨ ⎩ 0 i f c ij is satisfied wit ij = 0 Normdist(m ij , M ij ) if c ij is satisfied with ij > 0 1 i f c ij is not satisfied with ij > 0
d ij is null when the measured QoD value m ij fully satisfies the constraint c ij with respect to the expected interval M ij without ij . If the constraint is not satisfied even within the interval of the tolerance noted [M ij ± ij ], then d ij equals 1; otherwise d ij is computed as the normalized distance between the measured QoD value and the expected value for the object instance o.

Quality acceptability of a DB object instance o, noted Acceptability(o) is defined as the vector of probabilities over the k QoD dimensions in the set of declared contracts as:

Acceptability(o) = ⎛ ⎝ Acc 1 (o) • • • Acc k (o) ⎞ ⎠ (2)
Suppose the quality contracts given in Table 3 are applied to CRM DB. For each CRM DB object instance, the constraints declared in the quality contracts are checked. For each granularity level, Table 4 gives the computed probabilities for the DB object instances of the PRODUCT table of CRM DB. Figure 1 also indicates non null probabilities with nuances of red. Again, null probability means acceptable QoD dimension; 1 means unacceptable. We suppose that the probabilities of the other CRM DB object instances are null.

Constraining Data Quality in the query

We have designed a query language extension named XQuaL for manipulating and checking constraints on the quality of data in RDBMS. The syntax of a quality-extended query is given in Figure 4. Once declared, one (or several) contract(s) or constraints may be used in the QWITH part of the XQuaL queries.

Figure 4: Syntax of QWITH queries

'sfw query' represents the classical SELECT-FROM-WHERE statement of the query. In the QWITH statement, declared contract instances are identified and the constraints defined in the declared contracts are checked on ROW or CELL granularity levels specified after the statement ON. In the QWITH declaration, two alternatives are possible for checking constraints:

• Exact checking (default). Only DB object instances that exactly satisfy (with = 0) the constraints declared in the contracts are used in the query processing for building the query result,

• Approximate checking. DB object rows and cells that satisfy approximately (with non null ) the constraints defined in the contracts of the QWITH query will be considered for elaborating the query result.

Consider the query that retrieves all the products whose PRICE is greater than $10 in CRM DB. QoD metadata of PROD-UCT table involved in this query have been defined by means of the quality contract types and instances previously declared in Tables 2 and3. A "quality-blind" query would return three rows: P 1, P 2 and P 3. Different quality-aware queries are given in Table 5 based on the declared quality contracts that check data freshness, accuracy, and completeness at the ROW and CELL granularity levels. Results are presented in both EXACT and APPROXIMATE modes along with the acceptability values of the result for each QoD dimension considered in the QWITH part of the query. These queries lead to different results depending on the constraints required on the chosen QoD dimensions.

In the EXACT mode, only DB object rows and cells that satisfy exactly (with = 0) all the constraints defined in the contracts invoked in the QWITH query will be considered for elaborating the query result. For Q1 query, the contract fresh is defined on two granularity levels (ROW, TABLE) as given in Table 2. The constraints are checked on data freshness for each row involved in the SFW query processing. Probabilities resulting from the checking are given in the fourth column of Table 5. In the EXACT constraint checking mode, only the rows involved in the query with null probability will be considered for building the query result. Consequently, P 1 row will be excluded (P r f resh (ROW ( P 1 )) = .42 < .50 in Table 4) whereas in the APPROXIMATE mode, P 1 row will be included in the result (.42 + .15 > .50 see the fresh contract instance in Table 3).

Q2 is the same query as Q1 but with constraining the accuracy as it has been defined in Table 3. For Q2, the query result will not include product P 3 in the EXACT mode and also in the APPROXIMATE mode as well because its PRICE value ('7777') has a high probability of being inaccurate with respect to the contract accurate applied to CELL granularity level (P raccurate(CELL( 7777 )) = .35 > .06). The result presentation includes the probabilities of every cell returned in the result.

Similarly, for Q3 constraining freshness and completeness on rows and accuracy on cells, only P 2 satisfies the constraints and the query result will exclude P 1 and P 3 in the EXACT mode but it will include P 1 in the APPROXIMATE mode. Now consider a query that retrieves the list of product identifiers and the city of their purchaser, PROD ID and CUST CITY joining PRODUCT and CUSTOMER tables on CUST ID field. A "quality-blind" query would return P 1, P 2, and P 3 PROD IDs. Different join quality-aware queries may be formulated with respect to the previous quality contracts on freshness, accuracy, and completeness.

Suppose that all the acceptability values of CUSTOMER table are null except for cell C2 and its associated row such as P f resh (ROW ( C2 )) = .6, Paccurate(CELL( C2 )) = .2 and P complete (ROW ( C2 )) = .5.

For Q4 in the EXACT mode, the join of PRODUCT and CUSTOMER tables only consider the joining cells that have a null probability of being inaccurate with respect to the contract accurate, then P 2 -C2 and P 3 -C2 joins will be excluded from the final join result because the probability of C2 cell from CUSTOMER table is P raccurate(CELL( C2 ) = .2 > .06 with respect to contract accurate. But in the APPROXIMATE mode (.2 -.15 < .06), this cell will be considered for joining the tables and leads to a result including P 2 -C2 and P 3 -C2 rows.

For Q5 in the EXACT mode, the join of PRODUCT and CUSTOMER tables only consider the rows of both tables having null probability with respect to the contracts fresh and complete. P 1 row will then be rejected in the EXACT mode (P r f resh (ROW ( P 1 ) = .42 < .50) but included in the AP-PROXIMATE mode (P r f resh (ROW ( P 1 ) = .42 + .15 > .50).

Since QWITH query optimization is our ongoing work, the interesting issues concerning: i) the heuristics we've defined for building the algebraic annotated QWITH query trees, ii) the techniques for query rewriting, iii) the cost model for QWITH query processing, and iv) the algorithms we propose for relaxing the constraints of QWITH queries, will remain out of the scope of this paper. These omportant aspects constitute our immediate perspectives of research and development for improving the QWITH query engine.

XQUAL PROTOTYPE

XQuaL prototype is currently being implemented on Eclipse Platform 3.3.2 (JRE1.5.0 12) using SQLExplorer Plug-in 3.5.0 and based on Kepler 2 [START_REF] Bowers | Actor-Oriented Design of Scientific Workflows[END_REF] that is built upon Ptolemy II framework 3 developed at the University of California, Berkeley. Kepler is a scientific workflow management system that allows specifying and executing data-intensive analysis. A workflow in Kepler can be graphically designed by chaining together tasks, where each task may take input data from previous tasks, parameter settings, and data coming from external data sources.

The contribution of the XQuaL project in terms of development concerns: i) the implementation of the extended QWITH query engine, ii) the development of the library of functions for QoD evaluation, iii) a QoD contract editor invoking C, SAS or PL/SQL functions extending the design of Kepler workflows for QoD analysis, iv) the coupling of the QWITH query engine with the workflows and contracts specifications, and v) the development of a QoD metadata manager for storing, indexing and retrieving QoD measures.

RELATED WORK

In Data Quality Research, several contributions have been proposed for expressing in a simply and declarative way constraints on data quality with extending the query languages. Different approaches have used data quality indicators and metadata for selecting the best source [START_REF] Mihaila | Source Selection and Ranking in the WebSemantics Architecture: Using Quality of Data Metadata[END_REF] or the best query plans to execute [START_REF] Naumann | Quality-Driven Integration of Heterogenous Information Systems[END_REF] in a distributed environment. As the first prototype, Q-Data [START_REF] Sheth | Q-Data: Using Deductive Database Technology to Improve Data Quality[END_REF] checks if the existing data is correct and ensures data validation and cleanup by using a logical database language (LDL++). More recently, Guo et al. [START_REF] Guo | Caching with 'Good Enough' Currency, Consistency, and Completeness[END_REF] have proposed a model for express-2 Kepler project: http://www.kepler-project.org/ 3 Ptolemy II: http://ptolemy.eecs.berkeley.edu/ptolemyII/ ing currency and consistency constraints (C&C) in the queries on replicated and cached data by the means of a new clause on data currency that extends SQL. DQ 2 L (Data Quality Query Language) [START_REF] Mendès-Sampaio | Incorporating the Timeliness Quality Dimension in Internet Query Systems[END_REF] was designed to query relational data supporting a data quality aware query processing framework. In the context of quality-driven query processing, Braumandl et al. [START_REF] Braumandl | ObjectGlobe: Open Distributed Query Processing Services on the Internet[END_REF] propose to take into account data quality estimates when evaluating the user's query and deciding the best manner of carrying out the query (which sources to reach, which server to use, etc). TriQL4 , the query language of the Trio project [START_REF] Mutsuzaki | Trio-One: Layering Uncertainty and Lineage on a Conventional DBMS (Demo)[END_REF] is an extension of SQL including three built-in predicates for checking confidence, accuracy and lineage of queried data and extending traditional data management.

All these approaches have proposed an extension of SQL query language in order to include in different ways constraints on specific and fixed dimensions of the quality of data mainly for query answering. They may suffer from the drawback that they are neither flexible nor modular from the user perspective in the way that QoD is evaluated by hard-coded functions. In our approach, an extendable library of functions can be used to define and compute QoD measures. The composition of functions is required through the design of analytic workflows that can be set up by the user to automatically check and enforce various aspects of data quality particularly important for his/her specific needs and goals.

CONCLUSION AND PERSPECTIVES

Since, realistically, it is difficult to evaluate with certainty the quality of data in a large database, we propose an analytical and probabilistic approach that allows the evaluation of the quality of database object instances and takes into account this evaluation for elaborating and presenting alternative query results depending on constraints on data quality. Since it is also important to accommodate ad-hoc queries with taking into account possible data quality requirements, which of course, a priori are unknown, our objective is to evaluate data quality in the context of known uses (e.g., at the query time). In this paper, we address this by proposing a complete approach based on the use of various functions, statistics, and data mining techniques that are combined into analytic workflows dedicated to QoD evaluation for quality-aware query processing and QoD diagnostics. Analytical functions generate measures that are intended to characterize user-defined dimensions of QoD. These measures are stored as metadata in a repository. They are checked with respect to user-defined constraints that are associated to the database object instances and defined by means of contracts. Several QoD measures may be associated to one QoD dimension. A scoring function is used to compute the probability that for a given DB object instance, a QoD dimension is acceptable with respect to a quality contract that defines the set of constraints on its associated QoD measures.

QoD evaluation can be based on the results of typical datacentric analysis. They require a sequence of activities and components for data retrieval computation and presentation, assembled together into a single executable data analysis pipeline. The components may be part of the data management system, part of another application invoked through system calls (e.g., executing SQL or SAS scripts, etc.). In addition to providing users with a mechanism to define and compose themselves QoD analysis tasks, our long-term research perspectives are to design and support end-to-end analytic workflows for QoD evaluation, e.g., through tools for accessing external resources (data sources or functions), archival of metadata (QoD measures and descriptive metadata), optimizing and monitoring of QoD analytic workflow execution. In this context, we also want to propose various alternatives for assisting the user in the design of QoD analytic workflows depending on resource limitations.

Figure 1 .

 1 Figure1presents the example of a QoD analytic workflow designed for evaluating the quality of data of the database named CRM DB composed of two tables: PROD-UCT and CUSTOMER. This includes several tasks of evaluation on three QoD dimensions, namely freshness, accuracy, and completeness at different granularity levels (e.g., cell, row, column, table). Data object instances are the inputs of the tasks. Each task may be composed of several subtasks with allocated functions for the computation of the QoD measures. Summary statistics for numerical values are computed, out-of-range data values may also be detected with univariate statistics and percentiles (IQR) e.g., 'sas func iqr outlierProb.sas' of the subtask outlierProb composing the task ACCURACY evaluates the accuracy at the CELL granularity level of CRM DB. The execution of each allocated function generates QoD measures that are stored in the metadata repository as QoD matrices represented in Figure1as {Q}. The detailed description of each function is stored as descriptive metadata in the repository using PMML 1 represented as [D] in Figure1.QoD analytic workflows combine multiple analytical functions as the ones illustrated in the example of Table and Figure1. Functions may be implemented in different ways depending on the set of limitations specified in the workflow. Of course, many other analytical functions may be added to the library. The panel of analytical functions used for com-

Figure 1 :

 1 Figure 1: Example of QoD Analytic Workflow for CRM DB

Figure 2 :

 2 Figure 2: Syntax of Quality Contract Type Creation

Figure 3 :

 3 Figure 3: Syntax of Quality Contract

  P accurate (CELL(77)) = 0 P accurate (CELL(77)) = 0 Q3 SELECT PROD ID, PRICE P2,77 P fresh (ROW (P2))) = 0 P1,77 P fresh (ROW (P1)) = .42 FROM PRODUCT P accurate (CELL(P2)) = 0 P accurate (CELL(P1)) = 0 WHERE PRICE > 10 P accurate (CELL(77)) = 0 P accurate (CELL(77)) = 0 QWITH fresh ON ROW P complete (ROW (P2)) = 0 P complete (ROW (P1)) = 0 AND accurate ON CELL; P2,77 P fresh (ROW (P2))) = 0 AND complete ON ROW; P accurate (CELL(P2)) = 0 P accurate (CELL(77)) = 0 P complete (ROW (P2)) = 0 Q4 SELECT PROD ID, P1,Camden UK P accurate (CELL(P1)) = 0 P1,Camden UK P accurate (CELL(P1)) = 0 CUST CITY P accurate (CELL(Camden UK)) = 0 P accurate (CELL(Camden UK)) = 0 FROM PRODUCT P, P2,Camden NJ US P accurate (CELL(P2)) = 0 CUSTOMER C P accurate (CELL(Camden NJ US)) = 0 WHERE P3,Camden NJ US P accurate (CELL(P3)) = 0 P.CUST ID=C.CUST ID P accurate (CELL(Camden NJ US)) = 0 QWITH accurate ON CELL; Q5 SELECT PROD ID, P2,Camden NJ US P fresh (ROW (P2)) = 0 P1,Camden UK P fresh (ROW ( P1 )) = .42 CUST CITY P complete (ROW (P2)) = 0 P complete (ROW (P1)) = 0 FROM PRODUCT P, P3,Camden NJ US P fresh (ROW (P3)) = 0 P2,Camden NJ US P fresh (ROW (P2)) = 0 CUSTOMER C P complete (ROW (P3)) = 0 P complete (ROW (P2)) = 0 WHERE P3,Camden NJ US P fresh (ROW (P3)) = 0 P.CUST ID=C.CUST ID P complete (ROW (P3)) = 0 QWITH fresh ON ROW AND complete ON ROW;

  Sets of global or application-specific integrity rules, consistency constraints or inferred rules from statistical techniques that characterize the most plausible relationships between data instances or that compute the deviations from the rules. These constraints are verified at runtime. Constraint violations indicate errors or dubious data.

	Level	Category	Description		Examples
					QoD Functions		DB object
	I	QoD	Profiling	Simple counts computed from the database dictionary,	CP	nullValues% returns a	D,T,R,A
		Functions	look-up tables, control, log or trace files, usually with		percentage that repre-
				single-pass algorithms. These metadata can be used to		sents the quantity of null
				characterize some aspects of database completeness		values.
				and freshness depending on the level of granularity of the	F	updateFreq returns a	D,T,R,A
				database object instances (i.e., value, record, column, ta-		decimal in [0,1] that rep-
				ble or database in the relational context)		resents how frequent the
						DB object instance has
						been updated since its
						creation time.
	II	QoD			CT	SyntacticCorrectness	D,T,A,R,V
		Constraint-			returns	a	decimal
		Based Functions			number in [0,1] that rep-
						resents the percentage
						of format discordances,
						syntactical errors and
						misspellings.
	III	QoD Synopses	Statistical summaries, aggregates or parametric es-	AC	outlierProb returns a	V
		Functions	timations computed as approximate answers with		decimal number in [0,1]
				deterministic error bounds or probabilistic guarantees		based on IQR that rep-
				that the approximate answer is the actual one. Ba-		resents the probability of
				sic synopses are samples, equi-depth histograms, quan-		being an outlier in the
				tile computation. Advanced synopses are computed from		attribute domain.
				sketch-based computation techniques (e.g., V-optimal his-
				tograms, wavelets). They are useful to quickly reveal un-
				likely values that are artifacts or inconsistent patterns
				from samples of very large data sets before going into
				deeper and more expensive analysis.
	IV	QoD		Data mining results obtained from techniques such as
		Exploratory	clustering, association rule discovery, and decision trees.
		Mining	Func-
		tions

Table 2 : Example of Quality Contract Type Decla- ration

 2

Table 3 : Example of Contract Declaration

 3

Table 4 : Acceptability Values per QoD Dimension in PRODUCT table of CRM DB

 4

			CELL		ROW		COLUMN	TABLE
		Acceptability	7777	P 1	P 4	CUST ID	ORDER DATE	PRODUCT
		P r fresh	-	.42	0	0	-	.56
		P r accurate	.35	-	-	.06	-	-
		P r complete	-	0	.286	.25	.25	.071
	Query#	Query					Results
					Exact			Approximate ( = .15)
	Q1	SELECT PROD ID, PRICE FROM PRODUCT	P2,77 P3,7777		P fresh (ROW (P2)) = 0 P fresh (ROW (P3)) = 0	P1,77 P2,77	P fresh (ROW (P1)) = .42 P fresh (ROW (P2)) = 0
		WHERE PRICE > 10					P3,7777	P fresh (ROW (P3)) = 0
		QWITH fresh ON ROW;
	Q2	SELECT PROD ID, PRICE FROM PRODUCT WHERE PRICE > 10 QWITH accurate ON CELL;	P1,77 P2,77:		P accurate (CELL(P1)) = 0 P accurate (CELL(77)) = 0 P accurate (CELL(P2)) = 0	P1,77 P2,77:	P accurate (CELL(P1)) = 0 P accurate (CELL(77)) = 0 P accurate (CELL(P2)) = 0

Table 5 : Examples of QWITH Queries

 5

PMML -Predictive Model Markup Language, Version 3.1: http://www.dmg.org/

TriQL : http://infolab.stanford.edu/widom/triql.html

* Currently visiting researcher at supported by the European Commission (Grant FP6-MOIF-CT-2006-041000).


Karsten Schmidt

Carl Winsløw²
email: winslow@ind.ku.dk

Task design for Engineering Mathematics: process, principles and products

Keywords:

We present and analyse principles

INTRODUCTION

Mathematics is as important in most branches of engineering, as engineering is to the prosperity and development of contemporary society. Thus, it is of great importance to investigate exactly what mathematics is needed by (future) engineers, and how it could be effectively taught to them; such research is only emerging (see e.g. Winsløw et al.,to appear,sec. 2.5). In the common case where mathematics is taught in separate "service" courses which cater to several different study programmes, these questions may be considered in entire separation: a syllabus for the mathematics course is decided based on needs in the different study programmes of engineering which include the course, and subsequently the syllabus is delivered by mathematics faculty. This amounts to a complete separation of external and internal didactic transposition, in the sense of Chevallard (1992), where the selection of mathematical contents to be taught may be based on needs and priorities from the engineering disciplines, while the actual teaching is carried out according to generic standards and methods for teaching mathematics. The aims of the overall study programme (in engineering) are only considered in the external transposition of the mathematical knowledge (see Fig. 1). We can call this model a parallel model for teaching mathematics to engineers, as the internal didactic transposition runs in parallel to the rest of the programme and does not interact with it (while it is certainly intended that the students' learning serves in other courses, later on). In the literature on university mathematics education, it is widely agreed that the parallel model has drawbacks:

-students may experience the mathematics teaching as unmotivated and difficult, which is reflected in relatively high failure and attrition rates for some engineering programs (e.g. Baillie & Fitzgerald, 2010). -the knowledge they acquire in the mathematics course may not transfer readily to engineering contexts, in the sense that students are able to invest the knowledge acquired in mathematics courses when they need to do so in other courses of the programme (e.g. [START_REF] Britton | A case study of the transfer of mathematics skills by university students[END_REF].

Motivated by these well-known problems, the model in Fig. 1 has been modified, in many universities, by various attempts to relate the internal didactic transposition of mathematics more closely to the rest of the engineering programs (e.g. [START_REF] Kumar | Teaching mathematics from an application perspective[END_REF].

One of the most common ideas to further such an interaction is that to include, in the mathematics course, more or less simple examples and student assignments where engineering problems are solved based on mathematical methods and theoretical notions (see, for instance, [START_REF] Härterich | MathePraxis -connecting first-year mathematics with engineering applications[END_REF]. A main challenge here is that university mathematics teachers usually have no in-depth knowledge of any engineering discipline, let alone of all the specialties which the course they teach caters to. Of course they may then ask engineering specialists for help to identify authentic problems from Engineering which can be solved using the mathematics to be taught in their course (we abbreviate this kind of problems as APE). In that way, "Scholarly Engineering" may exercise a more direct influence on the internal didactic transposition of mathematical knowledge (cf. Fig. 1). In this paper, we investigate some general questions related to the implementation of this (quite common) idea at the level of the internal didactic transposition:

RQ1. How could the identification and transposition of APEs be organised, given the academic and institutional separation of university mathematics teachers from their colleagues in engineering?

RQ2. What didactic variables (cf. eg. [START_REF] Gravesen | Task design for students' work with basic theory in analysis: the cases of multidimensional differentiability and curve integrals[END_REF] are relevant to the construction of assignments based on APEs?

It is clear that answers to these questions will depend on institutional conditions and that even when such conditions are given, one will at most obtain very partial answers in the sense of reasonably validated examples of organisations (RQ1) and didactic variables (RQ2). As always in education, transfer of "answers" from one context to another will require some adaptation and interpretation, but this limitation may in fact be less important for the above questions, given the relatively high similarity of mathematics courses, the engineering programs they serve and the institution which deliver them. For these reasons, and given the importance of the matter already argued, it appears worthwhile to present such locally and partially validated answers. Concretely we will present and analyse the process, principles and products of APE design done at the Technical University of Denmark since 2000. Founded by H. C. Ørsted in 1829, this is one of the most prestigious Schools of Engineering in Europe, and by far the largest in Denmark.

A TASK DESIGN PROCESS

Mathematics 1 (hereafter Mat1) is the basic mathematics course for 17 different B.Sc.Eng.-programmes at the university, catering to about 1100 students per year.

The course occupies 1/3 of the students' time during the first year, and covers complex numbers, basics of Linear Algebra, Ordinary Differential Equations including linear systems, and multivariate and vector Calculus up to Gauss' theorem.

Besides ensuring a technical foundation for later work, the university also considers a common course on mathematics as important to the formation of an engineer identity.

Most of the course is quite traditional, however with intense use of the computer algebra system Maple. Exercises with easy applications to engineering occur. However, during the last four weeks of the course, the students work on a "project". This is an assignment containing about 20-30 more or less challenging tasks, related to a mathematical model related to an APE. The model is usually given in the assignment, and while some new mathematics may be introduced, the starting point is Mat1. Each project assignment is presented in a text of varying extent (ranging from 4 to 29 pages, averaging 11); it is those texts which we aim to analyse in this paper.

The students do the projects in groups, hand in a report of about 20-50 pages, and defend their work during an oral exam, which accounts for 25% of their grade.

The groups can choose their assignment from a list of 4-5 projects, in part depending on the study programme, with titles like those shown shown in Fig. 4. As the titles suggest, the project problems come from many different areas of Engineering. Every year, new projects are added and some are dropped; and the details of retained projects are updated based on teachers' experiences. The elaboration of new projects is a particularly delicate undertaking. When the first projects were done from 2000-2006, a systematic effort was deployed to engage researchers from the universityboth applied mathematicians and researchers from Engineering at large -to propose project topics; they were then, mostly, drafted or adjusted by the course responsible. Some are still used in revised form.

It is the task of the course responsible to organise the production and revision of projects. The initiative can come from teachers at the course or other mathematicians, who identify a more or less classical APE which can form material for a project; then, the motivation is often that some specific parts of Mat1 can be worked on in new ways. But the initiative also frequently come from colleagues from other departments. In some cases, their motivation is personal fascination with mathematics in a more or less current APE, and possibly ongoing collaboration with mathematics colleagues in this relation (reflecting an interaction between Scholarly Mathematics and Engineering, which could be added to Fig. 1). In other cases, they propose an attractive current APE for a Mat1 project, in order to attract students to their specialty later on -these colleagues then, sometimes participate for free as supervisors on the students' project work.

Summing up what the process involved in Mat1 could contribute to RQ1, at least two venues can be identified in relation to Fig. 1: -Scholarly (applied) Mathematics and other basic sciences such as chemistry and physics, where the main source of motivation is mathematical contents related to Mat1; but work done here can still involve or lead to APE. Project proposals from this source are typically mathematically "rich" but are not often related to current research. -Scholarly Engineering, often with current APE's; the elaboration of a project typically necessitates considerable adaption to fit Mat1, and is often tailored to the interest of students from a small range of study programmes.

Finally, the genesis of a project may involve a mixture of both sources, when the APE is identified by scholars with a deep involvement in both areas.

DIDACTIC VARIABLES AND PRINCIPLES

To present and analyse the project assignments which have appeared over the last 15 years, we have defined 10 didactic variables (DV) which are relevant to classify them according to the aims which have, explicitly or implicitly, been pursued (Fig. 2). Each variable has, in principle, a non-numerical range, but can be determined with relatively high objectivity for each assignment, based on the text. The detailed presentation of any project in terms of the variables will, naturally, be difficult to compare with others when given in this form. So when considering all projects it appears useful to assign indicatory numerical values to the DVs on a scale from 0 to 2: for instance, to assess the breadth of Mat1 contents which a given project requires the students to work with, 0 indicates that only one topic (such as systems of linear equations) is involved, 1 that a few topics from both Calculus and Linear Algebra are involved, and 2 that the project combines many topics. Naturally, this "grading" is not absolute but relative to other projects (cf. also Fig. 2). In the next section, we outline a concrete assignment and explain, at the same time, how the numerical values of the other DV's are set. The variables were initially formulated in by the authors (based on the first authors' many years of involvement in the design) and subsequently validated and adapted during the actual analysis of assignments. The variables thus constitute a concrete answer to RQ2, which is of course a partial answer based on experiences from context we described. In the rest of this paper, we provide more explanation on how the variables can be used to analyse concrete projects and, potentially, to direct and systematize the design of student assignments.

For each DV, Fig. 2 also includes a brief description of the more or less explicit aims which have been pursued in the construction of projects over the past 17 years. The brevity required in the Table format does not allow for much nuance. We note that what is ideal use of Maple is not subject to complete consensus among the teachers of the course, or in relation to the rest of the institution. On the one hand, some course teachers consider that students should use Maple whenever it is useful; while others, including colleagues from other departments, often insist on the value of students' mastery of basic manual computation (as reflected in the aims for DV3 given in Fig. 2). The variables DV4-10 all describe aspects of the relationship between the internal transposition represented by the assignment, and Scholarly Engineering (cf. Fig. 1).

Their values are thus of specific importance to go beyond the parallel transposition.

Didactic variable (DV):

Aim of designers:

DV1

PRODUCTS

A total of 37 projects have been proposed during the past 10 years. Not all projects are used every year, and all are revised before use, in the light of past experience, new needs in the course, and in a few cases, updates to the APE and its solution from Scholarly Engineering. We first give a relatively detailed presentation and analysis (based on the DVs) of one project; at the same time, we describe how each of the 10 DVs is assigned a value as described above, for a rough analysis of the projects. Then we present an overview and rough analysis of the whole inventory of projects.

In-depth presentation of one project

We now take a closer look at one of the projects, entitled: Heat flow in a housesimulation and dimensioning. The assignment is relatively long, 18 pages, including about 5 pages of data. The first paragraph outlines the underlying APE:

The building sector accounts for about 40% of the total energy consumption in Denmark.

It is a common assumption that there is a large unrealized potential for reducing the consumption (…) in a financially sound way. This requires knowledge of the physical processes which affect the energy consumption of buildings, the financial aspects of the construction and maintenance of buildings, as well as the mathematical methods used to compute these.

It turns out that the energy flow in the building is modelled as an analogy of currents in electric circuits (cf. Fig. 3). The project is based on a genuine APE, and bibliography of the assignment includes a reference to [START_REF] Nielsen | Simple tool to evaluate energy demand and indoor environment in the early stages of building design[END_REF] which is the essential source (DV4 = 2), along with a "pricelist" from the construction industry, and the last part of the assignment draws on a simple model of investments and interest. Relative to other projects, this assignment involves a relatively broad area of Engineering fields, and DV5 is set to 2. The introduction acknowledges that the model proposed in assignment is "a bit simplified", but in fact it still gives similar results; we assign DV7 to 2, in spite of some problems (see end of this section). where 𝑇𝑇 𝑠𝑠𝑒𝑒𝑑𝑑 is the desired internal temperature (constant), and 𝑃𝑃 is the performance of the internal heat source. While (*) is just a first order ODE, it still gives rise to interesting Engineering tasks: the investigation of stationary solutions, the performance needed to ensure an average temperature of 19.8 0 C, and the thickness required to respect given limits on the oscillation of 𝑇𝑇 𝑖𝑖 . The full model consists of a system of three differential equations which are similar to (*) but with an added complexity due to the heat contribution from sunlight which, moreover, is investigated with two different models. The students must also take into account a model of the walls involving layers of materials to be computed using authentic data. Finally, to take into account the cost of construction, the students are given a simple mathematical model for the total economy involving investment, interest, and operation costs; the mathematics is very simple but still gives rise to interesting questions regarding how to optimize, for instance, insulation thickness. Throughout, the students use real data (DV8=2) but these are all given, so DV9=0. Throughout the models are given to the students, and while students are given full and extensive explanations, they are not really asked to do more than apply them; thus DV6 = 1.

The project draws on a broad range of Mat1-topics: harmonic oscillations and complex exponential function, single and coupled differential equations, solved using advanced matrix algebra, involving both eigenvalue problems and quadratic forms. Thus DV1=2, while DV2=0 as almost no new mathematics is introduced (the exception being the argument required to justify the stationary solution to the system of differential equations, which involves an extended eigenvalue problem).

While most of the tasks can in principle be solved manually, the visualizations of temperature variations corresponding to different parameter values decisively require a tool like Maple. The assignment moreover invites to numeric experimentations, possibly based on graphical representations, and standard use for tedious operations like inversion of matrices, make the overall potential of Maple-use relatively average for projects; we thus assign DV3=1, even if the realized use by some students in some cases goes beyond a mere use of techniques known from the rest of the course.

In real practice, the project also suffers from some flaws. Some of the questions lead to less interesting results (like tedious computations leading to a requirement of a four-doubling of the wall thickness in order to reduce an already negligible oscillation of 0.12 0 C for 𝑇𝑇 𝑖𝑖 ). More serious is the breakdown of the model when taking into account the contribution of sunshine at low temperatures such as 8 0 C, where the stationary answer cannot be found. It can be argued that such problems often arise with simplified models, but it still leaves a negative impression on many students, which might be avoided by future revisions. Altogether, we consider DV10=1.

Figure 1 :

 1 Figure 1: The parallel model for didactic transposition in engineering education

Figure 2 .

 2 Figure 2. Didactic variables for the analysis of project assignments.

Figure 3 .

 3 Figure 3. A figure from the project assignment "Heat flow in house".The central model, illustrated in Fig.3, concerns a house with three rooms, called "climate zones". Here 𝑄𝑄 𝑘𝑘 ′ are the internal and external heat sources (heaters and sunlight), while 𝐶𝐶 𝑘𝑘 are the heat capacities of the rooms and 𝐾𝐾 * are the heat transmission coefficient of the walls of the house, reflecting that these walls involve a variety of layers. Before introducing the final model house, the students' work with the simpler case of a one-room house, and an external temperature 𝑇𝑇 𝑒𝑒 which is a

Inventory of project assignments

Fig. 4 lists the inventory of projects used in the last 10 years, analysed using the DVs.

DV (cf. Fig. 2)

Project name (shortened in a few cases) 1 2 3 4 5 6 7 8 9 10

Oscillations in Axle-bearing Systems 2 0 1 2 1 1 2 2 1 2 Micro/Nano Cantilever Based Mass Sensor 1 2 1 2 1 1 2 1 1 1 Enzymatic Hydrolysis of Cellulose 1 1 1 0 1 2 1 1 0 1

Modelling 2D Halbach permanent Magnets 1 2 2 2 1 2 2 1 0 2 Factorization of Integers 0 2 2 0 1 0 1 0 0 1

Oscillations in Grid Constructions 2 2 1 2 1 2 1 0 1 1 Groundwater Flow in the Forest Vestskoven 1 2 2 2 1 1 2 2 0 1

Error Correcting Codes 0 2 2 1 1 1 1 0 0 2 Phononic bandgaps 2 0 2 2 2 2 1 0 0 2 A number of interesting tendencies can be identified in the above table, including apparent dependencies of some variables, potentials which appear relatively unexplored (like DV9), etc.; some of these are still not fully analysed. We stress that a simple sum of the values of didactic variables, for a specific project, cannot be construed as a measure of the "didactic quality" of the assignment. One reason is that the variables are not of equal importance (in particular, DV1, 4 and 10 are essential).

But more importantly, one cannot always construe the number two as being objectively "the best possible value" of the DV; the aims listed in Figure 2 are open to debate and the viewpoint of teachers and designers may differ. A good example is DV4, where we have given "2" for projects with a clear APE, "1" for projects with an authentic problem from basic science (e.g. Chemistry) and "0" for projects which are not based on an APE but on a (prima facie) purely mathematical problem, such as the project "Geodesic curves". One can argue that a project of the "0" type can also be of high quality as a project for engineers, in view of the importance in several branches of the mathematical problem (in the example, DV2=2 and indeed, geodesic curves have multiple applications in many branches of engineering, see e.g. Patrikalaksis and Maekawa, 2010, 265-291). A similar uncertainty must also be pointed out for other variables such as DV3, where the further graduation suggested in Fig. 2 could be useful to provide a more nuanced picture than in the analysis in Fig. 4, where "2" merely means that Maple is indispensable for large parts of the project.

SUMMARY AND OUTLOOK

We have presented the principles, process and products of a relatively longstanding effort to integrate elements of scholarly Engineering (APE's) in the internal didactic transposition of basic mathematics in a course catering to a wide range of Engineering programmes, going well beyond isolated "applications" of a Calculus or Linear Algebra. We have emphasised the multiple dimensions which such an effort needs to consider, in order to maintain the link with the mathematical knowledge to be taught within the module in question, and to establish non-trivial links with Scholarly Engineering (cf. Figure 1). Certainly, the concrete inventory of variables can be developed and adapted further, and we believe it can eventually become a valuable explicit basis for the discussion of aims (right column in Fig. 3) of projects in our and other similar contexts. More importantly, considering such explicit variables could be an important tool for systematizing the design process, both as a check list for constructing new projects and (in combination with the analyses behind Figure 4) to identify potentials for enriching existing projects. We expect that the variables will also become useful guidelines for investigating the effects of the project work in this course as a means to facilitate the transition to later courses where mathematics is so fully integrated into the Engineering knowledge to be taught that the latter is in practice as inseparable from mathematics as music is from sound.