Pramudita Satria Palar

Lavi Rizki Zuhal

Yohanes Bimo Dwianto

Koji Shimoyama

Joseph Morlier

Shapley Additive Explanations for Knowledge Discovery via Surrogate Models

It is sometimes desirable to delve further into how the inputs

affect the output in design optimization and uncertainty analysis. Surrogate models such as Gaussian Process Regression and support vector regression are useful for such tasks and can be further enhanced by introducing advanced post-processing methods. This paper investigates Shapley Additive Explanation (SHAP) as a tool to aid surrogate-assisted data-driven analysis. In particular, surrogate-enabled SHAP analysis allows visualization of the input-output relationship in a meaningful way using summary SHAP plots and SHAP dependence plots. Some important information that can be extracted and visualized from SHAP includes the importance of input variables (i.e., global sensitivity analysis), nonlinearity level, and level of interactions. The benefits of Shapley values for engineering analysis using surrogate models are demonstrated in three engineering test problems.

I. Introduction

Global sensitivity analysis (GSA) is an important field in which the objective is to quantify the relative impact of input variables and their interactions on the output [START_REF] Iooss | Uncertainty management in simulationoptimization of complex systems[END_REF][START_REF] Borgonovo | Sensitivity analysis: a review of recent advances[END_REF]. GSA is widely used in uncertainty analysis, although other domains, such as design optimization and exploration, also take advantage of GSA. For example, designers can obtain better information regarding which variables should be prioritized when optimizing a system with GSA. Similarly, in uncertainty analysis, GSA helps engineers determine which variables should be controlled to reduce uncertainties in the output. Variance-based sensitivity analysis techniques (primarily Sobol indices) are arguably the most widely used methods due to their intuitiveness [START_REF] Sobol | Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates[END_REF][START_REF] Piano | Variance-based sensitivity analysis: The quest for better estimators and designs between explorativity and economy[END_REF]. In modern practice, GSA is frequently performed using surrogate models to reduce the computation cost for estimating sensitivity indices. Polynomial chaos expansion (PCE) [START_REF] Xiu | The Wiener-Askey polynomial chaos for stochastic differential equations[END_REF] is one of the most widely used surrogate models for handling a GSA task [START_REF] Sudret | Global sensitivity analysis using polynomial chaos expansions[END_REF]. Besides PCE, other popular models for GSA include Kriging and support vector regression [START_REF] Cheng | Global sensitivity analysis using support vector regression[END_REF][START_REF] Lu | Improved Kriging with extremum response surface method for structural dynamic reliability and sensitivity analyses[END_REF][START_REF] Cheng | Surrogate-assisted global sensitivity analysis: an overview[END_REF]. Beyond GSA, sometimes there is a need to take a peek inside the constructed surrogate models, aiming to understand the inside of the input-output relationship better. For example, one might want to know the nonlinearity level or the nature of the interaction between variables in such a relationship.

Analyzing the importance of variables is also central to one of the current subjects of interest in machine learning: explainability [START_REF] Roscher | Explainable machine learning for scientific insights and discoveries[END_REF][START_REF] Burkart | A survey on the explainability of supervised machine learning[END_REF]. In this regard, explainability is particularly important for black-box machine learning models, which expressions are too complex to decipher. Such complexity is in contrast to simple but interpretable models such as linear regression or logistic regression. Interpretable models, as the name suggests, provide a form that helps gain insight and is also easily understood by a human. The main drawback of simple models is their limited predictive power, which is why complex models are desirable in many applications. It is then common to apply post hoc explanations to interpret complex black-box models.

Several tools to help interpret black-box models have been developed, such as partial dependence plot [START_REF] Friedman | Greedy function approximation: a gradient boosting machine[END_REF], individual conditional expectation (ICE) [START_REF] Goldstein | Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation[END_REF], and local interpretable model-agnostic explanations (LIME) [START_REF] Ribeiro | Why should i trust you?" Explaining the predictions of any classifier[END_REF]. The techniques mentioned above work by dissecting the black-box model into a post hoc explanation that works at either a global or local level (i.e., to a single prediction level). For example, PDP depicts the marginal effect of an input on the prediction of a model to show the relationship between the input and the output. Of interest in this paper is the SHapley Additive exPlanation (SHAP) [START_REF] Lundberg | A unified approach to interpreting model predictions[END_REF], which is based on the Shapley values from game theory [START_REF] Shapley | A value for n-person games[END_REF]. SHAP is convenient since it reveals several essential pieces of information at once. Moreover, some fast methods exist for calculating SHAP for several machine learning models, such as random forest and gradient boosting. However, despite the potential of SHAP for engineering design or uncertainty analysis, to the best of our knowledge, there are only a few papers that deploy SHAP for engineering problems (e.g., see [START_REF] Mangalathu | Failure mode and effects analysis of RC members based on machine-learningbased SHapley Additive exPlanations (SHAP) approach[END_REF][START_REF] Feng | Interpretable XGBoost-SHAP machine-learning model for shear strength prediction of squat RC walls[END_REF]). In this regard, SHAP can be utilized with commonly used surrogate/machine learning models in engineering, e.g., Kriging and PCE.

Often, there are situations when it is desirable to visualize the input-output relationship in either design optimization or uncertainty analysis. For example, in design exploration, one wants to investigate how specifically changing an input variable would impact the performance of the design. Unfortunately, direct visualization is impossible for high-dimensional input spaces, which is why specialized techniques are required for visual analysis. Examples of statistical machine learning and data mining techniques that have been applied to handle such tasks include self-organizing map [START_REF] Obayashi | Multi-objective design exploration and its applications[END_REF][START_REF] Nagar | Visualization and analysis of Pareto-optimal fronts using interpretable self-organizing map (iSOM)[END_REF], active subspace [START_REF] Constantine | Active subspace methods in theory and practice: applications to kriging surfaces[END_REF][START_REF] Zhou | A unified approach for global sensitivity analysis based on active subspace and Kriging[END_REF], and proper orthogonal decomposition [START_REF] Oyama | Data mining of Pareto-optimal transonic airfoil shapes using proper orthogonal decomposition[END_REF]. Typical questions include: how strong is the relationship between inputs and output? How nonlinear is the input-output relationship? How does a single variable interact with other variables? GSA metrics such as Sobol indices can answer some of these questions. However, users can be better informed if these questions are answered visually. For example, if a function is nonlinear, we want to know how specifically the inputs affect the output. Tools from the interpretable ML field can then be of great advantage to aid such an endeavor.

In this paper, we investigate the use of SHAP to aid data-driven analysis using surrogate models, with applications toward design optimization and uncertainty analysis. Due to its model-agnostic nature, SHAP can work with any supervised machine learning model. A surrogate model is essentially a supervised machine learning, either as an interpolation or regression technique, in which SHAP can be conveniently applied to reveal important insight. This paper aims to introduce SHAP as an additional instrument for helping design exploration and uncertainty analysis.

II. Surrogate modeling

A surrogate model is essentially a supervised machine learning model. That is, given a vector of input variables 𝒙 = {𝑥 1 , 𝑥 2 , . . . , 𝑥 𝑚 } 𝑇 , where 𝑚 is the input dimensionality, and a label 𝑦 = 𝑓 (𝒙), a surrogate model works by approximating the relationship between the input and the output. The first step in building a surrogate model is to provide the experimental design X = {𝒙 (1) , 𝒙 (2) , . . . , 𝒙 (𝑛) } 𝑇 , where 𝑛 is the sample size, and the responses 𝒚 = {𝑦 (1) , 𝑦 (2) , . . . , 𝑦 (𝑛) } = { 𝑓 (𝒙 (1) ), 𝑓 (𝒙 (1) ), . . . , 𝒙 (𝑛) }. The black-box function 𝑓 (𝒙) is evaluated through either a numerical model or physical experiment. In this paper, we focus on data from numerical models to create a surrogate model. The input domain is defined as 𝛀, where 𝛀 = 𝑚 𝑖=1 Ω 𝑖 and Ω 𝑖 is the marginal input for the 𝑖-th variable. This paper only briefly explains GPR and SVR since we focus on the interpretability of a surrogate model. Readers are referred to more specific literature on GPR (e.g., [START_REF] Rasmussen | Gaussian processes in machine learning[END_REF][START_REF] Sacks | Design and analysis of computer experiments[END_REF]) and SVR (e.g., [START_REF] Drucker | Support vector regression machines[END_REF][START_REF] Smola | A tutorial on support vector regression[END_REF][START_REF] Schölkopf | Learning with kernels: support vector machines, regularization, optimization, and beyond[END_REF]) for more details. We use KADAL, an in-house surrogate modeling code, to build the GPR and SVR model.

A. Gaussian Process Regression

The main ingredient of GPR is the assumption that the black-box function is a realization of the following stochastic process:

𝑌 (𝒙) = 𝜇 𝐺 𝑃 (𝒙) + 𝑍 (𝒙), (1)
where 𝜇 𝐺 𝑃 (𝑥) is the mean function and 𝑍 (𝒙) is a zero-mean Gaussian process. For simplicity, we assume that 𝜇 𝐺 𝑃 (𝑥)

is constant in the in the entire input space, i.e., 𝜇 𝐺 𝑃 (𝑥) = 𝜇 𝐺 𝑃 .

GPR assumes that the responses are correlated with each other. Let us denote two points in the input space, 𝒙 and 𝒙 ′ , which responses are correlated to a degree. The correlation is modeled by the kernel function 𝑘 (𝒙, 𝒙 ′ ; 𝜽) = corr(𝑦, 𝑦 ′ ; 𝜽),

where 𝜽 = {𝜃 1 , . . . , 𝜃 𝑚 } is the length scales. In this regard, the length scale indicates the degree of correlation between the responses. This paper uses the squared-exponential kernel function to model the correlation between input points.

Our implementation uses the squared-exponential kernel to model the correlation between responses at input points.

Construction of a GPR model requires several information. First, an 𝑛 × 𝑛 correlation matrix 𝑹 is constructed, with its (𝑖, 𝑗)-th component is as follows: 𝑅 𝑖 𝑗 = 𝑘 (𝒙 (𝑖) , 𝒙 ( 𝑗 ) ; 𝜽), where 𝒙 (𝑖) , 𝒙 ( 𝑗 ) ∈ X. It is also necessary to build a correlation vector 𝒓 (𝒙) = {𝑘 (𝒙, 𝒙 (1) ; 𝜽), . . . , 𝑘 (𝒙, 𝒙 (𝑛) ; 𝜽)} 𝑇 , especially for making a prediction. GPR then predicts the response at an arbitrary location using the following expression:

ŷ(𝒙) = 𝜇 𝐺 𝑃 + 𝒓 𝑇 𝑹 -1 ( 𝒚 -1𝜇 𝐺 𝑃 ), (2)
Hyperparameter optimization aims to find the best set of hyperparameters (including length scales) that optimizes a certain objective function. The most common approach for hyperparameter optimization is to maximize the log-likelihood function, written as:

ln L(𝜽) = - 𝑛 2 ln(2𝜋) - 𝑛 2 ln(𝜎 2 𝐺 𝑃 ) - 1 2 ln(| 𝑹|) - ( 𝒚 -1𝜇 𝐺 𝑃 ) 𝑇 𝑹 -1 ( 𝒚 -1𝜇 𝐺 𝑃 ) 2𝜎 2 𝐺 𝑃 , (3)
where 𝜎 2 𝐺 𝑃 is the process variance. Let us define the vector of hyperparameters 𝜸 = {𝜽, 𝜇 𝐺 𝑃 , 𝜎 2 𝐺 𝑃 }. First, 𝜇 𝐺 𝑃 is estimated as

μ𝐺𝑃 = (1 𝑇 𝑹 -1 1) -1 1 𝑇 𝑹 -1 𝒚. (4)
Notice that an analytical formulation exists for estimating 𝜎 2 𝐺 𝑃 if the GPR exactly interpolates the model. However, no such formulation exists for regressing GP, which is why the process variance should be tuned together with the lengthscales using a numerical optimization procedure. Our implementation uses the covariance matrix adaptation evolution strategy [START_REF] Hansen | Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES)[END_REF] and local hill climbing to optimize the hyperparameters.

B. Support Vector Regression

In contrast to GPR, in which a kernel defines the correlation, SVR uses a kernel function to perform mapping into high-dimensional space. To grasp the concept of SVR, let us begin with the linear SVR that makes a prediction using the vector of coefficients 𝒘 𝑆𝑉 and the bias term 𝑏, reads as

f (𝒙) = 𝒘 𝑇 𝑆𝑉 𝒙 + 𝑏. (5)
SVR trains the coefficients and the bias by minimizing the so-called 𝜀-insensitive loss function. The definition 𝜀 is such that the loss function penalizes points outside the 𝜀 boundary.

There are various versions of SVR depending on the penalization type. In this paper, we use the 𝐿 2 -SVR which reads as

L 𝜀 2 =            0 if f 𝑆𝑉 𝑅 (𝒙) -𝑦) < 𝜀 f 𝑆𝑉 𝑅 (𝒙) -𝑦) -𝜀 2 otherwise (6)
As one can see from Eq. ( 6), the 𝐿 2 -SVR uses a quadratic penalty in the formulation. After defining the penalization type, an SVR model is constructed by solving the following minimization problem:

min 1 2 ||𝒘|| 2 + 𝐶 2 𝑛 ∑ 𝑖=1 (𝜉 𝑖 + 𝜉 * 𝑖 ) 2 , (7)
subject to 𝑦 -𝒘 𝑇 𝒙 𝑖 -𝑏 ≤ 𝜀 + 𝜉 𝑖 , (8)
𝒘 𝑇 𝒙 𝑖 + 𝑏 -𝑦 𝑖 ≤ 𝜀 + 𝜉 * 𝑖 , (9)
𝜉 𝑖 , 𝜉 * 𝑖 ≤ 0, [START_REF] Roscher | Explainable machine learning for scientific insights and discoveries[END_REF] where 𝜉 𝑖 and 𝜉 * 𝑖 are slack-variables and 𝐶 is the regularization parameter. Central to nonlinear SVR is the "kernel trick," which allows an SVR to construct a nonlinear prediction using the mapping function Φ(.). The prediction then reads as

f (𝒙) = 𝒘 𝑇 𝑆𝑉 Φ(𝒙) + 𝑏 = 𝑛 ∑ 𝑖=1 (𝛼 𝑖 -𝛼 * 𝑖 )Φ(𝒙 𝑖 ) 𝑇 Φ(𝒙 𝑖 ) + 𝑏. (11)
The high-dimensional product Φ(𝒙 𝑖 ) 𝑇 Φ(𝒙 𝑖 ) is accomplished by the kernel function 𝑘 (𝒙, 𝒙 ′ ; 𝜽). We can then write the prediction of a nonlinear SVR as

f (𝒙) = 𝑁 ∑ 𝑖=1 (𝛼 𝑖 -𝛼 * 𝑖 )𝑘 (𝒙 (𝑖) , 𝒙) + 𝑏. (12)
The training process in SVR is accomplished by solving the following dual form:

min 1 2         𝜶 𝜶 *         𝑇         K - K -K K                 𝜶 𝜶 *         +         𝜺 -𝒚 𝜺 + 𝒚         𝑇         𝜶 𝜶 *         (13)
subject to:

        1 -1 *                 𝜶 𝜶 *         =         0 0         , 𝛼, 𝛼 * ≤ 0. ( 14
)
where 𝛼 and 𝛼 * are Lagrange multipliers, K = 𝑲 + (1/𝐶) 𝑰 is a modified Gram matrix, 𝑲 is the original Gram matrix, with its (𝑖, 𝑗)-th components is 𝐾 𝑖 𝑗 = 𝑘 (𝒙 (𝑖) , 𝒙 ( 𝑗 ) ; 𝜽), and 𝑰 is an 𝑛 × 𝑛 identity matrix. The set of hyperparameters in SVR includes 𝐶, 𝜽, and 𝜀, which need to be optimized to maximize the predictive power of SVR. This paper uses the squared-exponential function as the kernel for SVR.

C. Using a surrogate model in practice

The surrogate model can then be deployed to serve engineering purposes, such as to aid optimization, uncertainty analysis, or reliability analysis. For example, the calculation of statistical moments using Monte Carlo simulation in uncertainty analysis now only needs to call f (𝒙) in lieu of the actual function. The surrogate model, particularly GPR, is also helpful in Bayesian optimization that performs simultaneous exploration and exploitation of the design space [START_REF] Shahriari | Taking the human out of the loop: A review of Bayesian optimization[END_REF][START_REF] Zhan | Expected improvement for expensive optimization: a review[END_REF]. Finally, a surrogate model can uncover useful patterns or insights as a cheap approximation of the true relationship. Unfortunately, such information is often hidden by the highly complex expression of the model. Thus, an additional tool is needed to create a curtain to take a peek inside the model.

Most of the explanation tools in the interpretable machine learning literature are model-agnostic. Therefore, any surrogate model is explicable through the application of such tools. Although kernel-based methods are versatile and capable of approximating highly non-linear functions, they are not directly interpretable to humans. The next section explains SHAP, which acts as a post hoc explanation technique to dissect the surrogate model. Using the language of function decomposition, the prediction of a surrogate model can be seen as the sum of the contributions of individual variables and their interactions (or, using game theory jargon, "coalitions"):

III. Shapley Additive Values for Surrogate Modeling

f (𝒙) = ∑ 𝑢⊆ [1:𝑚] f𝑢 (𝒙 𝑢 ), (15)
where 𝑢 ⊆ [1 : 𝑚] includes both the empty set ∅ and all non-empty subsets, and f𝑢 (𝒙 𝑢 ) is the prediction of the model using the subset of variables as indexed in 𝑢.

B. Shapley values

Define a "game" with 𝑚 players and a value function val(.) that returns a real value to the individual players or their combinations thereof. All possible coalitions between players are defined in the subset of

𝜙 𝑗 = 1 𝑚 ∑ 𝑢⊆ { -𝑗 } 𝑚 -1 |𝑢| -1 (val(𝑢 ∪ { 𝑗 }) -val(𝑢)). ( 16
)
Consider a coalition 𝑢 that excludes 𝑗. The definition of (val(𝑢 ∪ { 𝑗 }) is the gain obtained when 𝑢 and 𝑗 are involved in the coalition. It is then easy to see that the term inside the bracket on the right-hand side of the equation is the marginal contribution of player 𝑗 to the specific coalition 𝑢. Notice that the marginal contribution is calculated for all possible coalitions 𝑢 ⊆ {-𝑗 }, for the marginal contributions to be summed out after being multiplied by the weight. Eq.( 16) can be interpreted as follows. The Shapley value for a player 𝑗 is defined as the sum of the marginal contribution of player 𝑗 to all possible coalitions divided by the number of coalitions excluding 𝑗 of this size. The sum is then divided again by the number of players to obtain the Shapley value of player 𝑗.

One important and desirable property of Shapley value is efficiency, which states that the sum of Shapley values for all players equals the value of the grand coalition (i.e., all variables are involved) val( [1 : 𝑚]). The Shapley value can then be interpreted as the individual contributions of the players to the grand coalition. Notice that we have not defined the "game" and the "player." Let us now put it into a context; that is, the "game" is the prediction of a surrogate model, and the "players" are the input variables. The value of the grand condition is then the prediction of the surrogate model, that is, val( [1 : 𝑚]) = f (𝒙). The following section explains SHAP, which uses the principle of Shapley value to explain a supervised machine learning model.

C. SHAP

There are several methods that use the principle of Shapley values (e.g., Shapley effects in GSA [START_REF] Song | Shapley effects for global sensitivity analysis: Theory and computation[END_REF][START_REF] Plischke | Computing Shapley effects for sensitivity analysis[END_REF]).Of interest in this paper is the SHAP that decomposes the prediction of an interpolation/prediction at any instance, in contrast to Shapley effects that work globally. Therefore, SHAP can explain why a machine learning model makes a certain prediction for any point in the input space. In addition, SHAP can also explain the model on a more global scale by using the information from multiple instances.

Let us formally define SHAP directly within the context of a predictive model. Consider a single vector of input 𝒙 and the corresponding prediction of the model, f (𝒙). The following decomposition is the fundamental building block of SHAP:

f (𝒙) = 𝜙 0 + 𝑚 ∑ 𝑗=1 𝜙 𝑗 (𝒙), (17)
where 𝜙 0 is the prediction of the model without involving any variables (i.e., empty set), and 𝜙 𝑗 is the contribution of the Using Eq. ( 16), the calculation of SHAP for an arbitrary point 𝒙 in the input space is written as

𝜙 𝑗 (𝒙) = 1 𝑚 ∑ 𝑢⊆ { -𝑗 } 𝑚 -1 |𝑢| -1 f𝑢∪{ 𝑗 } (𝒙 𝑢∪{ 𝑗 } ) -f𝑢 (𝒙 𝑢 ) , (18)
where for a coalition 𝑢, f𝑢∪{ 𝑗 } (𝒙 𝑢∪{ 𝑗 } ) is the prediction of the model involving all variables in the index 𝑢 ∪ { 𝑗 }, while f𝑢 (𝒙 𝑢 ) involves only the variables indexed in 𝑢. In this regard, the term inside the bracket on the right-hand side of Eq.( 18) is the marginal contribution of variable 𝑗 to the model's prediction on the coalition 𝑢 (which excludes 𝑗).

Although the formulation looks simple, the calculation of SHAP is time-consuming since it is necessary to construct 2 𝑚 models. Such a requirement comes from the necessity to calculate the value function for all possible permutations.

Therefore, the SHAP is usually estimated using approximation techniques, most notably KernelSHAP, which fits a weighted local linear model at the query point [START_REF] Lundberg | A unified approach to interpreting model predictions[END_REF].

D. Aggregated SHAP values

A metric for global sensitivity is obtained by averaging the absolute SHAP values, written as

Avg (𝜙) = 1 𝑛 𝑚𝑐𝑠 𝑛 𝑚𝑐𝑠 ∑ 𝑖=1 𝜙(𝒙 (𝑖) ) (19)
where 𝑛 𝑚𝑐𝑠 is the number of random samples for calculating Avg (𝜙). It is a common practice to compute the averaged SHAP values using only the training data set. For design optimization and uncertainty analysis purposes, it is preferable to use as many samples as possible to compute Avg (𝜙). To that end, we generate an external set X 𝑚𝑐𝑠 = {𝒙 (1) , 𝒙 (2) , . . . , 𝒙 (𝑛 𝑚𝑐𝑠 ) }, where 𝑛 𝑚𝑐𝑠 ≫ 𝑛 to estimate Avg.(𝜙). The distribution of X 𝑚𝑐𝑠 is the same as that of the training set (i.e., 𝛀).

In machine learning literature, SHAP is commonly used together with tree-based models, e.g., random forest or gradient boosting. However, such models are not suitable for simulation-based engineering problems. The reason why techniques such as GPR and SVR are preferred is that they provide smooth predictions. It is correct that SHAP is unsuitable if there is a strong dependency between the input variables [START_REF] Aas | Explaining individual predictions when features are dependent: More accurate approximations to Shapley values[END_REF]. Fortunately, most engineering design problems feature a set of independent variables.

E. Presenting SHAP values

There are several ways to present and visualize SHAP values to users. The information presented depends on how the SHAP values are visualized. In this regard, SHAP can be presented for a single prediction or at a more global level by showing the values for multiple predictions. However, in this paper, there is little interest in the SHAP values for a single prediction. Instead, we have more interest in displaying the information at a single predictor (i.e., input variable)

or a more global level. First, one can compute and show the averaged SHAP values to determine the most important variables. Although it is a common practice to calculate the averaged SHAP values using only the training samples, we recommend using an external sampling set consisting of a large amount of samples for such a purpose. The reason is that the limit and the upper bounds of the input variables, or the distributions, are typically known in the first place.

The SHAP summary plot depicts multiple useful information in a single plot. Notice that variables in the summary plot are usually sorted according to their importance as measured by the averaged SHAP values, in which the uppermost variable in the plot is the most important one. The correlation between input variables and the corresponding SHAP values is visible in the summary plot. However, it is better to visualize such information in a single dependent plot.

The SHAP summary plot is primarily useful to visualize the impact of all variables on the output simultaneously.

Nevertheless, observing the interaction and degree of nonlinearity from the SHAP summary plot is relatively difficult, which is where independent dependence plots are more useful.

The SHAP values for one particular variable can be shown with respect to the input value of the variable in the form of a scatter plot. Such a dependency plot is useful since it reveals multiple information regarding that particular variable in a single plot, including the level of nonlinearity and possible interactions with other variables. Interactions appear as scattered dots for a particular value of the input variable. The dots in SHAP dependence plot are usually colored by the input values of the other variable. The coloring will show a noticeable pattern if there is a strong interaction between the two variables.

In the next section, the functionality of SHAP visualization is demonstrated in multiple forms on various engineering problems. The SHAP is compared with the Morris elementary effect (EE) method [START_REF] Morris | Factorial sampling plans for preliminary computational experiments[END_REF][START_REF] Campolongo | An effective screening design for sensitivity analysis of large models[END_REF]. Morris' method yields the mean and standard deviation of the elementary effect, which gives information on the global sensitivity and nonlinearity to the user. In this paper, we use UQLab to calculate the Morris EE [START_REF] Marelli | UQLab: A framework for uncertainty quantification in Matlab[END_REF].

IV. Computational Results and Discussions

A. Wing weight function

The wing weight problem is an analytical function that models a light aircraft wing. The output of interest is the wing's weight which is affected by ten input variables listed in Table 1. The expression for the wing weight function reads as

𝑓 (𝒙) = 0.036𝑆 0.758 𝑤 𝑊 0.0035 𝑓 𝑤 𝐴 𝑐𝑜𝑠 2 (Δ) 0.6 𝑞 0.006 𝜆 0.04 100𝑡 𝑐 𝑐𝑜𝑠(Δ) -0.3 (𝑁 𝑧 𝑊 𝑑𝑔 ) 0.49 + 𝑆 𝑤 𝑊 𝑝 . ( 20
)
It is difficult to guess the true complexity of the problem by looking at Eq.( 20). However, there is an impression that Eq.( 20) is highly nonlinear, given the presence of power and trigonometric terms. An SVR model with 𝑛 = 500 is built to fit the function. The model yields 𝑅 2 = 1 and 𝑅𝑀𝑆𝐸 = 7.3 × 10 -3 , which accuracy is more than sufficient for knowledge discovery.

The result from EE analysis is shown in Fig. 1. Shown in these figures are the mean and standard deviation of the distributions, denoted as 𝜇 𝑀 𝑅 and 𝜎 𝑀 𝑅 , respectively. Also shown is the mean of the distribution of the absolute values 𝜇 * 𝑀 𝑅 , which is useful for analyzing the strength of the input variables. Notice that the EE information is extracted from the SVR model and not directly from the wing-weight function. As expected, the EE method does not fully reveal Regardless, it is difficult to distinguish which one is most dominant from EE: nonlinearity or interactions. Therefore, it can be seen that EE only shows partial information to the user. EE is useful on its own, but its capability to reveal information is relatively limited.

The averaged SHAP values calculated from X 𝑚𝑐𝑠 (10000 samples) are shown in Fig. 2. The averaged SHAP values indicate that 𝑁 𝑧 , 𝐴, 𝑡 𝑐 , 𝑆 𝑤 , and 𝑊 𝑑𝑔 significantly dominate the change in the wing-weight function. On the other hand, the impact of the other variables is much smaller. In terms of the input importance, the observation from SHAP agrees well with Morris' method. However, EE only informs whether the input non-linearly affects the output or interacts with other variables without any information regarding the level of nonlinearity and interaction.

Most importantly, SHAP can simultaneously display the non-linearity and interaction in single or multiple plots.

Fig. 3 depicts the SHAP dependence plots for the wing weight function. The inputs are shown in their normalized values (i.e., [0, 1]) to allow simultaneous visualization of multiple variables within a single plot. Fig. 3a shows the six most influential variables, while Fig. 3b is for the rest of the variables, which are significantly less influential than the others. Only 200 samples are shown to avoid too cluttered plots. It can be seen from Fig. 3 that the impact of all variables on the weight is mostly linear or slightly nonlinear at best. The only exception is the quarter-chord sweep (Δ) which nonlinearly affects the wing weight in a quadratic fashion, but its contribution is relatively insignificant.

The interactions between input variables are small and unimportant since the SHAP values are slightly scattered for only a few variables. For example, the scattered values in the SHAP for 𝑁 𝑧 indicates the slight nonlinear effect or interactions. Interestingly, SHAP uncovers that the wing-weight function is relatively simple, despite the complex expression in Eq.( 20). The expression in Eq.( 20) is also not easily decipherable by humans and might give a wrong impression of the complexity, while visualization from SHAP is easier to comprehend. SHAP also shows the correlation between input variables and the wing's weight. Variables 𝑁 𝑧 , 𝐴, 𝑆 𝑤 , and 𝑊 𝑑𝑔 positively correlate with the wing's weight, which makes sense. In addition, the SHAP plot also reveals that all variables but Δ monotonously affect the output of interest. The knowledge obtained from SHAP analysis matches the actual knowledge regarding the wing weight function. For example, keeping the wing area small is important to avoid an excessively heavy wing. To take another example, it also makes sense the paint weight 𝑊 𝑝 do not significantly affect the weight, compared to the flight design gross weight 𝑊 𝑑𝑔 .

B. Piston simulation function

The next problem is a simple analytical piston simulation function with seven input variables [START_REF] Ben-Ari | Modeling data from computer experiments: an empirical comparison of kriging with MARS and projection pursuit regression[END_REF]. The problem is widely used in UQ and GSA literature to benchmark various methods. The piston simulation function describes the circular motion of a piston within a cylinder, with the output of interest being cycle time 𝐶 (the time for completing one cycle) in seconds. There are seven input variables for this problem as listed in Table 2. The equation for the piston simulation function is written as

𝐶 (𝒙) = 2𝜋 √ 𝑀 𝑘 + 𝑆 2 𝑃 0 𝑉 0 𝑇 0 𝑇 𝑎 𝑉 2
, where

𝑉 = 𝑆 2𝑘 √ 𝐴 2 + 4𝑘 𝑃 0 𝑉 0 𝑇 0 𝑇 𝑎 -𝐴 𝐴 = 𝑃 0 𝑆 + 19.62𝑀 - 𝑘𝑉 0 𝑆 (21)
As one can see from Eq.( 21), the problem involves a chain of nonlinear functions. The first impression when seeing Eq.( 21) is that the function is highly nonlinear. For this problem, a GPR model with Gaussian kernel and 𝑛 = 500 samples is fitted. The accuracy of the built SVR is 𝑅 2 = 0.999 and 𝑁 𝑅𝑀𝑆𝐸 = 0.0066, which is already highly accurate for the model to be reliable in terms of the extracted knowledge.

The summary plots of the Morris' EE for the piston simulation function are shown in Fig. 4. The plots identified that the strongest variable is 𝑆, followed by 𝑉 0 , 𝑚, and 𝑘, with the contribution of the rest of the variables are not significant. with the other variables, as seen from the highly dispersed SHAP values. Therefore, the high standard deviation of 𝑘 is mainly due to the interaction with the other variables rather than nonlinearity. Similarly, the SHAP values of the piston surface area also show evidence of weak interaction with other variables.

C. Re-entry trajectory problem

The last problem is the stochastic re-entry trajectory analysis of an Apollo type with seven input parameters. The problem is an uncertainty quantification problem in which the input parameters are assumed to be normally distributed as shown in Table 3. We use the falling range, which is defined as the flight distance in the horizontal direction (see Fig.

for the schematic), as the output of interest to be approximated. The aerodynamic quantities are calculated using a modified Newtonian impact theory, with the capsule modeled with a mesh of triangle facets. More details on this problem can be found in Tokunaga et al. [START_REF] Tokunaga | Stochastic re-entry trajectory analysis with uncertain initial conditions for safety assessment[END_REF]. First, the Morris' EE summary plots are shown in Fig. 9. The significantly strong contribution of the attitude angle 𝛼 is obvious from the summary plot. Moreover, the EE summary plots also reveal that the overall impact of attitude angle is decreasing the falling range. Notice that the EE summary plots give the impression that increasing all the other variables lead to an increased falling range. However, as shown in the SHAP analysis soon, this turns out to be incorrect for the path angle 𝛾. 11), especially how the attitude angle affects the falling range. That is, the increase in attitude angle decreases the falling range, with a slight but visible nonlinear trend. The SHAP dependence plot for the path angle shows that increasing and decreasing the path angle from its nominal value leads to a decrease in the falling range (although the impact is less significant than that of the attitude angle); It is worth noting that such a variation is not visible from the EE summary plot. It can also be observed that the increase in altitude ℎ and velocity 𝑉 yields a higher falling range. As for the mass, increasing the mass leads to only an insignificant increase in the falling range. Interactions between variables exist, primarily those between the three most influential variables (namely, 𝛼, 𝛾, and ℎ). Lastly, the effect of pitch rate 𝑞 and moment of inertia 𝐼 on the falling range is negligible.

SHAP clearly shows its advantage over Morris' EE analysis on the re-entry trajectory problem. Although Morris' EE analysis is still useful on its own to provide a quick interpretation of the result (especially to show the relative strength of the input variables), it does not reveal important trends, such as the non-monotonous relationship between the path angle and the falling angle.

V. Conclusions and future works

This paper demonstrates the application of SHAP with surrogate modeling for aiding visualization and analyzing input-output relationships in an informative way. Despite its wide application in a general machine learning context, the use of SHAP is still not widespread in the engineering domain. In this paper, we demonstrate the applicability of SHAP for helping data-driven engineering analysis and design. SHAP allows users to observe and see several important trends intuitively, which is useful for exploratory analysis in the context of design, optimization, or uncertainty analysis.

The usefulness of SHAP as a tool to explain surrogate models is demonstrated on a ten-variable wing-weight function, seven-variable piston simulation function, and seven-variable re-entry trajectory analysis problem. The results

show that SHAP can uncover and visualize how the inputs change the output of interest, which greatly aids engineers in gaining important insight into the problem. In addition, SHAP also shows the magnitude of importance and how the input variables specifically interact with each other. We contrasted SHAP with Morris' elementary effect, in which the latter only gives information on the presence of nonlinearity and interaction without further detailed information. SHAP is also a versatile post-processing tool because it is a model-agnostic method; that is, it can be used with any surrogate model.

The KernelSHAP algorithm can still be slow, especially for a large sample size. For future works, analytical or fast calculation of SHAP is necessary to accelerate the analysis problem (especially in the context of the GPR and SVR model). Another interesting research direction is to compare SHAP with other interpretability techniques or established methods in global sensitivity analysis, e.g., Sobol indices. Finally, studying SHAP for individual points in multi-objective optimization is also one potential future research direction.

Define [ 1 :

 1 𝑚] := {1, 2, . . . , 𝑚} and the corresponding subset 𝑢 as 𝑢 ⊆ [1 : 𝑚]. Let us also define {-𝑢} = [1 : 𝑚] \𝑢 as the complement of 𝑢. Further, for an index 𝑢, denote Ω 𝑢 as the subset of the 𝛀 where Ω 𝑢 = 𝑖 ∈𝑢 Ω 𝑖 . Such a definition is important because we need to calculate the model's prediction when only the variables indexed in 𝑢 are involved.

[ 1 :

 1 𝑚]. Let us define 𝑢 ⊆ [1 : 𝑚] as the subset of input variables and the corresponding value function as val(𝑢). The Shapley value for a player 𝑗 is written as follows:

𝑗-th input

  to the prediction at 𝒙. It is common to define 𝜙 0 = E[ f (𝒙)], or as the mean of the responses of the training set/experimental design. From a pragmatical viewpoint, 𝜙 0 is the prediction made without any information regarding the input variables. The sum of the Shapley values (i.e., SHAP) equals the difference between f (𝒙) and 𝜙 0 . In this sense, 𝜙 𝑗 then represents the contribution of the input 𝑗 to the main prediction. A variable that contributes positively to the mean will have a positive SHAP value and vice versa.

( a )Fig. 1

 a1 Fig. 1 Summary plots of Morris' elementary effects for the wing weight function.

Fig. 2 Fig. 3 SHAP

 23 Fig. 2 Averaged SHAP values estimated from the SVR model for the wing weight function.

Fig. 5 Fig. 6 SHAP

 56 Fig. 5 Average SHAP values estimated from the GPR model for the piston simulation function.

Fig. 7 A

 7 Fig. 7 A schematic depiction of the Apollo-type capsule re-entry trajectory analysis problem [39].

Fig. 8 Fig. 9

 89 Fig. 8 Histograms of falling range from the actual simulation and the GPR model

Fig. 10 AveragedFig. 11 SHAP

 1011 Fig. 10 Averaged SHAP values estimated from the GPR model for the re-entry trajectory analysis problem.

Table 1 Variables used in the wing weight function.

 1

	Variables	Variable Name (unit)	[lower, upper bound]
	𝑆 𝑤	Wing area (ft 2 )	[150, 200]
	𝑊 𝑓 𝑤	Weight of fuel in the wing (lb)	[220, 300]
	𝐴	Aspect ratio	[6, 10]
	Δ	Quarter-chord sweep (degrees)	[-10, 10]
	𝑞	Dynamic pressure at cruise (lb/ft 2 )	[16, 45]
	𝜆	Taper ratio	[0.5, 1]
	𝑡 𝑐	Airfoil thickness to chord ratio	[0.08, 0.18]
	𝑁 𝑧	Ultimate load factor	[2.5, 6]
	𝑊 𝑑𝑔	Flight design gross weight (lb)	[1700, 2500]
	𝑊 𝑝	Paint weight (lb/ft 2 )	[0.025, 0.08]

Table 3 Stochastic input variables used in the re-entry trajectory analysis problem

 3

	Variables	Variable names (unit)	[Mean value, Standard deviation]
	ℎ	Altitude (m)	[80000,7000]
	𝑉	Velocity (m/s)	[7140,200]
	𝛼	Attitude angle/angle of attack (deg)	[180, 5]
	𝑚	Mass (kg)	[5762, 576]
	𝑞	Pitch rate (deg/)	[0, 20]
	𝐼	Moment of inertia (kg.m 2 )	[7150, 715]
	𝛾	Path angle (deg)	[15, 5]

Acknowledgement

Pramudita Satria Palar and Lavi Rizki Zuhal were funded partly through the Program Penelitian, Pengabdian kepada Masyarakat dan Inovasi ITB (P2MI) administered by Institut Teknologi Bandung. Part of the work was also carried out under the Collaborative Research Project 2022 of the Institute of Fluid Science, Tohoku University.

Table 2 Variables used in the piston simulation function

Variables

[lower, upper bound] Piston weight (𝑚) [START_REF] Shahriari | Taking the human out of the loop: A review of Bayesian optimization[END_REF]60] Increasing 𝑆 and 𝑘 tends to decrease the cycle time, while it is the opposite for the other variables. We can also see that 𝑘 and 𝑚 have similar values of 𝜇 * 𝑀 𝑅 , but the former has a higher standard deviation. This indicates that the effect of 𝑘 is more nonlinear than 𝑚, or 𝑘 interacts stronger with other variables. Again, EE does not reveal how the inputs specifically affect the behavior of the cycle time.

The averaged SHAP values for the piston simulation function are shown in Fig. 5. It can be seen that the most important variable is piston surface area, followed by initial gas volume, piston weight, and spring coefficient (the ranking is similar to that of the EE). The impact of the other three variables are significantly less than the first four variables. Fig. 6 depicts the SHAP dependence plot for all the seven variables, normalized to [0, 1] 7 , within a single plot.

Starting from the less influential variables, we observe that the effects of the ambient temperature (𝑇 𝑎 ) and filling gas temperature 𝑇 0 are significantly smaller than the other, as evidenced by their extremely small SHAP values. However, it is difficult to observe the non-significance of 𝑇 𝑎 and 𝑇 0 from the piston simulation equation as shown in Eq. [START_REF] Constantine | Active subspace methods in theory and practice: applications to kriging surfaces[END_REF]. It can be seen that the piston weight 𝑚 and initial gas volume 𝑉 0 positively correlate with the cycle time, with the increment in


Pramudita Kemas Zakaria

Satria Palar

Rizki Lavi

Joseph Zuhal

Morlier

Pramudita Satria Palar

Zuhal

Joseph Morlier

Physics-Informed Proper Orthogonal Decomposition for Data Reconstruction

Many engineering problems are governed by complex governing equations that are difficult and typically require high computational costs to solve. Machine learning and surrogate modelling aid such an endeavour by providing a cheap-to-evaluate prediction model that acts as a replacement of the original model. While most research focuses on predicting scalar values (e.g., lift and drag), predicting the solution field is also of interest in many practical engineering and scientific applications. This paper proposes a Physics-Informed Proper Orthogonal Decomposition (POD) technique that improves the solution field prediction by enforcing governing equations as a loss penalty. The proposed idea utilizes a reduced-order modeling technique based on POD to decompose solution snapshots into singular vectors and values. A Gaussian Process Regression is then utilized to predict the singular values from variable parameters. The predicted singular values from the data of the problem are then adjusted via optimization to minimize the physics-informed loss and achieve better prediction. In this paper, we illustrate the efficacy of the proposed method on simple two-dimensional partial differential equations. The result clearly shows that the proposed physics-informed POD outperforms the conventional POD in terms of approximation error.

I. Introduction

One of the exciting current research trends in engineering focuses on utilizing machine learning and data science technology to aid design optimization and exploration [START_REF] Li | Surrogate model uncertainty quantification for reliability-based design optimization[END_REF][START_REF] Zuhal | Polynomial-chaos-kriging with gradient information for surrogate modeling in aerodynamic design[END_REF][START_REF] Bouhlel | A Python surrogate modeling framework with derivatives[END_REF][START_REF] Dubreuil | Development of MDO Formulations based on Discplinary Surrogate Models by Gaussian Processes[END_REF], acquire hidden physical insights [START_REF] Karniadakis | Physics-informed machine learning[END_REF] or even solve the governing equation of the problem itself [START_REF] Raissi | Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[END_REF][START_REF] Mao | Physics-informed neural networks for high-speed flows[END_REF]. Currently, two machine learning paradigms are extensively used and studied in engineering: (1) data-driven approaches, which purely exploit available data, and (2) physics-informed approaches, which utilize a priori information about the physics of the problem. Data-driven approaches are generally used for predicting quantities of interests [START_REF] Laurenceau | Building efficient response surfaces of aerodynamic functions with kriging and cokriging[END_REF][START_REF] Chiplunkar | Gaussian process for aerodynamic pressures prediction in fast fluid structure interaction simulations[END_REF], multi-fidelity modeling [START_REF] Fischer | Bayesian low-fidelity correction approach to multi-fidelity aerospace design[END_REF][START_REF] Ghoreyshi | Integration of multi-fidelity methods for generating an aerodynamic model for flight simulation[END_REF], and flowfield reconstruction [START_REF] Yu | Flowfield reconstruction method using artificial neural network[END_REF] by solely relying on data (i.e., no prior physics information are involved). On the other hand, the physics-informed approach leverages physical laws to improve the performance of the data-driven approach by imposing certain physical laws into surrogate models [START_REF] Chiplunkar | Adding Flight Mechanics to Flight Loads Surrogate Model using Multi-Output Gaussian Processes[END_REF][START_REF] Chiplunkar | Sparse physics-based Gaussian process for multi-output regression using variational inference[END_REF], replacing specific modeling to help numerical solver [START_REF] Yang | Predictive large-eddy-simulation wall modeling via physics-informed neural networks[END_REF], or even acts as the solver of the governing equations by using machine learning approach techniques such as neural networks [START_REF] Raissi | Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[END_REF][START_REF] Kashinath | Physics-informed machine learning: case studies for weather and climate modelling[END_REF]. Moreover, unlike a traditional numerical approach, the physics-informed machine learning method can exploit the solution of a simpler equation to help predict the solution of a more complex governing equation [START_REF] Aliakbari | Predicting high-fidelity multiphysics data from low-fidelity fluid flow and transport solvers using physics-informed neural networks[END_REF].

Two main points motivate the development of the proposed method. First, although predicting scalar values is important, fast prediction of a physical field is also essential in many engineering applications. In this regard, reduced-order models aim to provide a quick-to-predict solution that significantly helps in the design, analysis, and optimization process [START_REF] Yu | Non-intrusive reduced-order modeling for fluid problems: A brief review[END_REF]. Furthermore, uncertainty quantification of a physical field is another crucial application that requires fast prediction of multiple outputs [START_REF] Tripathy | Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification[END_REF]. The second motivation is that many of the existing machine learning methods in this field are purely data-driven; thus, ignoring existing prior information or knowledge such as the governing equation of the physical problem [START_REF] Chang | Reduced-Order Modeling for Complex Flow Emulation by Common Kernel-Smoothed Proper Orthogonal Decomposition[END_REF][START_REF] Kramer | Nonlinear model order reduction via lifting transformations and proper orthogonal decomposition[END_REF][START_REF] Bertram | Fusing distributed aerodynamic data using Bayesian Gappy Proper Orthogonal Decomposition[END_REF]. However, as mentioned before, most existing prediction or reconstruction methods do not incorporate physical laws. The main aim of injecting physical laws into a machine learning method is to reduce the required data for accurate prediction since engineering data is typically expensive.

The proper orthogonal decomposition (POD) is a versatile and robust method that has been widely applied for field reconstruction [START_REF] Qamar | Steady supersonic flow-field predictions using proper orthogonal decomposition technique[END_REF][START_REF] Bui-Thanh | Proper orthogonal decomposition extensions for parametric applications in compressible aerodynamics[END_REF], modal analysis [START_REF] Taira | Modal analysis of fluid flows: An overview[END_REF][START_REF] Taira | Modal analysis of fluid flows: Applications and outlook[END_REF], design optimization [START_REF] Toal | Geometric filtration using proper orthogonal decomposition for aerodynamic design optimization[END_REF][START_REF] Zhu | Proper orthogonal decomposition assisted inverse design optimisation method for the compressor cascade airfoil[END_REF][START_REF] Oyama | Data mining of Pareto-optimal transonic airfoil shapes using proper orthogonal decomposition[END_REF], and uncertainty quantification [START_REF] Mohammadi | Efficient uncertainty quantification of stochastic heat transfer problems by combination of proper orthogonal decomposition and sparse polynomial chaos expansion[END_REF][START_REF] Mohammadi | Efficient uncertainty quantification of CFD problems by combination of proper orthogonal decomposition and compressed sensing[END_REF]. POD is a model order reduction technique that decomposes a physical field into the linear combination of orthogonal eigenfunctions multiplied by the coefficients. To construct POD modes, one needs to collect the snapshots obtained by varying the input parameters or from temporal evolution if the goal is to decompose a transient physical field. The orthogonal nature of POD decomposition allows easy interpretation of the produced model, which is especially insightful in modal analysis. The conventional POD is purely data-driven because no physics information is involved during the process. However, recent advancements in machine learning have shown that injecting prior knowledge or physics information can improve the accuracy of surrogate model [START_REF] Karniadakis | Physics-informed machine learning[END_REF]. The most notable algorithm is the physics-informed neural network that takes advantage of physical laws to constrain the machine learning prediction [START_REF] Raissi | Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[END_REF]. Another example of algorithms is the physics-informed Gaussian process (GP) [START_REF] Raissi | Hidden physics models: Machine of nonlinear partial differential equations[END_REF]33]. Besides the aforementioned well-known algorithm, there is also a potential for utilizing the physics-informed framework for other data-driven algorithms such as POD.

In this paper, we propose a Physics Informed POD (PI-POD) method that takes advantage of physical laws to reconstruct the solution of the engineering problem. Our method leverages a GP surrogate model [START_REF] Rasmussen | Gaussian processes in machine learning[END_REF] to aid in predicting POD coefficients. Similar strategies have been explored previously in the context of fluid-structure interaction [START_REF] Chiplunkar | Gaussian process for aerodynamic pressures prediction in fast fluid structure interaction simulations[END_REF] and multidisciplinary optimization [START_REF] Dubreuil | Development of MDO Formulations based on Discplinary Surrogate Models by Gaussian Processes[END_REF]. Notice that no physics is involved during the construction of our GP model since it primarily acts as a tool to map POD coefficients. However, there are variants of physics-GP in which the physics is incorporated into the GP formulation [START_REF] Chiplunkar | Adding Flight Mechanics to Flight Loads Surrogate Model using Multi-Output Gaussian Processes[END_REF][START_REF] Chiplunkar | Sparse physics-based Gaussian process for multi-output regression using variational inference[END_REF]33]. This paper explains the working mechanism of the proposed method and presents some preliminary results on a relatively simple problem.

The rest of this paper is structured as follows: Section II explains the details of our proposed method. Section III presents the demonstration of PI-POD in a two-dimensional partial differential equation (PDE). Section IV discusses the result of our proposed method and its comparison with conventional POD. Section V concludes the current results of PI-POD with a pointer of future works.

II. Method

In this section, we explain the main idea behind our proposed method. This section are divided into three parts: the data-driven POD part, the proposed PI-POD part which depends on the governing equation, and the last part explains the proposed method.

A. Proper Orthogonal Decomposition

Consider a set of solutions A of a governing PDE equation, with 𝑁 data for each solution and 𝑀 solutions, denoted as

A = [ 𝐴 1 , 𝐴 2 , ..., 𝐴 𝑀 ] ∈ R 𝑁 ×𝑀 (1)
In fluid dynamics literature, each solution in the solution matrix A is usually called a "snapshot", so we are going to refer to these solutions as "snapshots". In this regard, the 𝑀 solutions are obtained from varying the boundary conditions or other parameters (e.g., Reynolds number). The 𝑁 rows in solution matrix A refers to the number of points in the PDE solution.

We can perform the singular value decomposition (SVD) procedure on the solution matrix A to obtain the singular vectors U and V, and singular values 𝚺

A = U𝚺V 𝑇 = 𝑟 ∑ 𝑖=1 𝜎 𝑖 𝜉 𝑖 𝜓 𝑇 𝑖 , (2)
where

U = [𝜉 1 , ..., 𝜉 𝑟 ] ∈ R 𝑁 ×𝑟 V = [𝜓 1 , ..., 𝜓 𝑟 ] ∈ R 𝑀×𝑟 𝚺 = diag(𝜎 1 , ..., 𝜎 𝑟 ) ∈ R 𝑟×𝑟 𝑟 ≤ min(𝑁, 𝑀) (3)
Due to the unitary property of U, we can calculate a coefficient matrix 𝚫 such that A = U𝚫.

𝚫 = U 𝑇 A 𝚫 = [𝛿 1 , ..., 𝛿 𝑟 ] ∈ R 𝑀×𝑟 (4)
To reduce the computational cost of POD, the solution matrix A is usually approximated by truncating some columns in U. The approximated solution matrix  is computed as

 = Û Δ, (5)
where

Û = [𝜉 1 , ..., 𝜉 𝑘 ] ∈ R 𝑁 ×𝑘 (6)
Δ = Û𝑇 A (7)
The parameter 𝑘 is determined by defining an error tolerance 𝜖 𝑡𝑜𝑙

E(𝑘) = 𝑘 𝑖=1 𝜎 2 𝑖 𝑟 𝑖=1 𝜎 2 𝑖 ≥ 1 -𝜖 2 𝑡𝑜𝑙 (8)
where E(𝑘) denotes the energy percentage contained within the first 𝑘 modes.

We can then reconstruct a solution, given the POD coefficients, by

û * = ÛΔ * (9)
where û * is the predicted field and Δ * is the given POD coefficients for prediction. If we are given a solution 𝑢 * instead, we can calculate the corresponding POD coefficients by

Δ * = Û𝑇 𝑢 * . ( 10
)
An example of situations where we want to reconstruct a solution field is predicting a flow field given untested flow conditions (e.g., untested Mach number or Reynolds number). To that end, one needs to estimate Δ * so that the flow field can be reconstructed without running more computational fluid dynamics simulations. In this paper, we use the SVD implementation of MATLAB to perform the decomposition. As explained in the next section, our proposed method leverages a surrogate model based on GPR to estimate the POD coefficients for untested conditions. In this paper, we use MATLAB's SVD subroutine that removes extra rows and columns of zeros from the diagonal matrix of singular values and the corresponding rows and columns of the singular vector.

B. Gaussian Process Regression

GPR, or also known as Kriging, is a type of surrogate model that takes an input x ∈ R 𝐷 to predict the outputs 𝑦, which are treated as realizations of Gaussian Processes. The prediction structure of a GPR model is given by ŷ

(x) = 𝛽 𝑇 𝜙(x) + Z(x) (11)
The first term 𝛽 𝑇 𝜙(x) represents the mean term, or also known as the trend function, while the second term Z(x) is the stochastic part of the GPR model, respectively. Each single point in the input space is treated as a Gaussian random variable, in which all points in the input are space are correlated with each other as modeled by the covariance function.

The term 𝜙(x) denotes a set of regression functions and 𝛽 is the vector of the corresponding coefficients. In this paper, we utilize a constant trend (i.e., 𝜇) for the sake of simplicity. The corresponding GPR prediction then becomes

ŷ(x) = 𝜇 + Z(x) (12)
The stochastic term Z(x) with a constant trend can then be written as

Z(x) = 𝜓𝚿 -1 (y -1𝜇) (13)
where 𝚿 and 𝜓 are correlation matrix and vector respectively. The covariance function used in this paper is the Gaussian correlation function, reads as

Ψ(𝑖, 𝑗; 𝜃) = exp (𝑥 𝑖 -𝑥 𝑗 ) 2 2𝜃 , ( 14
)
where 𝜃 is the length-scale of the correlation function. Notice that a multi-dimensional correlation function is simply constructed by the product of one-dimensional correlation functions. We then have a vector of lengthscale 𝜽 = {𝜃 1 , 𝜃 2 , . . . , 𝜃 𝐷 } for a 𝐷-dimensional correlation for a 𝐷-dimensional approximation.

The vector of lengthscale 𝜽 is determined by maximizing the so-called likelihood function, as given by

ln(𝐿) = - 𝑛 2 ln 2𝜋 - 𝑛 2 ( σ2 ) - 1 2 ln 𝚿 - (y -1𝜇)𝚿 -1 (y -1𝜇) 2 σ2 , (15)
where

𝜇 = 1 𝑇 𝚿 -1 y 1 𝑇 𝚿1 (16)
and

σ2 = (y -1𝜇)𝚿 -1 (y -1𝜇) 𝑛 (17)
The optimization of Eq.15 is a 𝐷-dimensional optimization problem with respect to 𝜽. Once the optimum hyperparameters are found (including 𝜽, 𝜇, and 𝜎 2 ), we can then use GPR to make predictions at any point by using Eq.12.

GPR typically provides an accurate prediction for low sample size, a scenario that some supervised learning models cannot handle. Another reason why we picked GPR as the method of choice is that it naturally provides an uncertainty estimate for the mapping process in POD (explained in the next sections). Although not explored in this paper, the uncertainty estimate of GPR will be useful later for facilitating adaptive sampling to reduce the number of samples for the mapping.

C. Incorporating Physics Information

Suppose that partial-differential equation as the governing equation of the problem is given by

D (𝑢(x, 𝑡)) = 0, x ∈ 𝛀, 𝑡 ∈ [0, 𝑇] (18)
where 𝑢(x, 𝑡) represents the solution of the partial-differential equation 18, D represent the non-linear partial-differential operator, and 𝛀 is a subset of R 𝐷 in D-dimensional space. In this paper, we only consider a PDE defined in a spatial domain. The governing equation of the problem is be treated as loss function to adjust the singular values from the singular value decomposition. The adjustment variable, 𝛼 is computed from the minimization problem 19.

min 𝛼 L ( û) = D ( û) + 𝛾 𝑏𝑐 𝜖 𝑏𝑐 û = Ûℎ (𝛿 * + 𝛼) (19)
where the term 𝛾 𝑏𝑐 is a constant to impose boundary condition from the governing equation and 𝜖 𝑏𝑐 denotes the error at the boundary condition. Note that we need to perform this minimization problem each time we want to make a prediction using PI-POD, so the term 𝛿 * here denotes the predicted POD coefficient for this prediction. The term 𝛼 has size equal to the number of truncated singular values, so the dimensionality of the minimization problem in Eq. 19 equals the size of the truncated singular values. The value of variable 𝛾 𝑏𝑐 is defined to be adaptive, in the sense that when the value of 𝜖 𝑏𝑐 gets too high when minimizing the term D ( û), then the value of 𝛾 𝑏𝑐 is increased into significantly high value. In this paper, we set 𝛾 𝑏𝑐 equals 10 7 when 𝜖 𝑏𝑐 > 0.05, and 1 otherwise. Such a condition will force the model not to violate the boundary condition during the optimization process. There is no definitive reason why we choose 10 7 value for 𝛾 𝑏𝑐 . Setting 𝛾 𝑏𝑐 to other value would result in roughly the same prediction, as long as the value is high enough. An adaptive setting of 𝛾 𝑏𝑐 is one subject of future works.

D. Physics-Informed POD

Our objective is to reconstruct the field solution from POD and tune it according to the governing equations. We can create a solution matrix by concatenating all snapshots as columns in the solution matrix:

A = [𝑢 1 , ..., 𝑢 𝑀 ] ∈ R 𝑁 ×𝑀 , ( 20
)
where A is the solution matrix for our problem. The pseudo-algorithm of the PI-POD method is written in Algorithm 1 For the GPR-assisted mapping procedure, the next step is to build 𝑟 GPR models where 𝑟 is the number of bases retained in the POD model, and each GPR model predicts each POD coefficient as a function of all variable parameters in our problem. Up until this point, the proposed method is still data-driven. The main goal here is to predict 𝚫 for Algorithm 1 Constructing Physics-Informed POD (Offline Phase)

Require: Solution Matrix A

Perform SVD to Solution Matrix A Require: User-defined 𝜖 𝑡𝑜𝑙 value Compute 𝑘 by using Eq. 8.

Truncate singular sector U 𝑙 and U ℎ by its corresponding 𝑘 column Calculate truncated POD coefficients 𝚫 by using Eq. 7 by its corresponding singular vector U Create GPR mappings between 𝚫 𝑙 as inputs to 𝚫 ℎ as outputs the desired solution at an untested parameter. The main novelty of the proposed method is the physics-informed part when we reconstruct the predicted solution. The central idea here is to refine the predicted POD coefficients so that the reconstructed solution, ideally, satisfies the physical laws and boundary conditions. The POD coefficients are first predicted using GPR models from unobserved parameters. Finally, the proposed method applies a gradient-based optimization to minimize the loss function by optimizing 𝛼 (see Eq. [START_REF] Tripathy | Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification[END_REF], with predicted POD coefficients from the GPR model treated as the initial guess for the gradient-based minimization. The pseudo-algorithm for prediction using Physics-Informed POD is written in Algorithm 2.

Algorithm 2 Prediction using Physics-Informed POD (Online Phase)

Require: Parameters 𝛾 𝑏𝑐 , 𝛾 𝛼 Compute POD coefficients by GPR model from untested parameters Compute 𝛼 by solving minimization problem Eq. 19 Compute predicted solution ûℎ * with Eq. 9 by using truncated singular vector Û

The predicted solution û * is the final output of the Physics-Informed POD. Notice that the procedure outlined in Algorithm 1 is only performed once using the training data, while the prediction procedure in Algorithm 2 is performed every time we want to predict a different solution.

III. An Illustrative Example

A. Poisson Equation

For demonstration purposes, we use a simple linear two-dimensional partial differential equation. The governing equation used for this problem is the Poisson equation, reads as

∇ 2 𝑢 = 10, (21)
In this problem, the spatial domain is a 100x100 grid two-dimensional space within [0, 1] 2 , with Dirichlet boundary conditions on the four sides surrounding the domain. The boundary conditions imposed are the parameters of interests; That is, we aim to predict the physical field under variations in the boundary conditions. The samples for POD and PI-POD are then taken by sampling from the domain of the input parameters. There are two cases for this test problem. The first compares the predictive power of PI-POD and conventional POD with a small number of snapshots, and the second compares the robustness performance of PI-POD and conventional POD with more snapshots. Notice that the conventional POD here does not use the physics-informed correction. Instead, the conventional POD directly uses the mapped coefficients to reconstruct the high-fidelity solution.

The 𝜖 𝑡𝑜𝑙 value is chosen to be 0.01 for all cases in this problem. The root-mean-squared-error (RMSE) is chosen to be the error metric for this problem, written as

𝜖 𝑟 𝑚𝑠𝑒 = √ 𝑁 𝑖 ( û𝑖 -𝑢 𝑖 ) 2 𝑁 ( 22
)
where û represents the predicted field and 𝑁 is the number of nodes in the spatial domain. Notice that 𝜖 𝑟 𝑚𝑠𝑒 is calculated for a single combination of untested parameters. For multiple combinations (e.g., as in optimization or uncertainty analysis), we can depict multiple 𝜖 𝑟 𝑚𝑠𝑒 is the form of a boxplot to analyze the robustness of the algorithm.

Case 2: Medium size snapshots case

In the second case, we generated 100 snapshots with random variations in the boundary conditions. The variations in the boundary conditions are modelled by varying the vector of constants 𝝀, where 𝝀 = {𝜆 1 , 𝜆 2 , 𝜆 3 , 𝜆 4 }, as shown in the following:

𝑢(𝑥, 0) = 𝜆 1 sin(𝜋𝑥) 𝑢(𝑥, 1) = 𝜆 2 sin(𝜋𝑥) 𝑢(0, 𝑦) = 𝜆 3 sin(𝜋𝑦) 𝑢(1, 𝑦) = 𝜆 4 sin(𝜋𝑦) (23)
where 𝜆 𝑖 , 𝑖 = 1, 2, 3, 4, is a normally distributed random variable with the distribution of N (0, 2). In total, we generated 100 snapshots for this case, in which 70 and 30 snapshots were used as training and validation snapshots, respectively. The PI-POD and POD were constructed for three different sub-cases with different sizes of snapshots: 10, 20, and 30 snapshots for each sub-case (taken from 70 training snapshots with 50 random shuffles).

The metric that we use in the boxplot for the second case is the Averaged-RMSE, which is computed using the following equation

Averaged-RMSE = 𝑁 𝑣 𝑖=1 𝜖 𝑟 𝑚𝑠𝑒 𝑁 𝑣 , (24)
where 𝑁 𝑣 is the number of validation snapshots used.

IV. Results

A. Poisson Equation: Case 1 Results

For case 1, we analyze the reconstructed solution for each validation data separately so that we can perform a more thorough analysis.

First validation data

For the first validation snapshot, the resulting prediction and the error surfaces for Physics-Informed POD and conventional POD are plotted in Fig. 1. Besides the RMSE in the whole domain, the RMSE at the boundary points is also calculated. For this result, the accuracy for both Physics-Informed POD and the conventional POD is summarized in Table 1. It is clear that the Physics-Informed POD outperforms the conventional POD in terms of overall accuracy, as indicated by the lower total RMSE and RMSE in the entire domain and at the boundary conditions. It can also be seen from Fig. 1 that the reconstructed solution from the PI-POD is closer to the true solution than the conventional POD.

The error level of the PI-POD is notably lower than the conventional POD. It can also be seen that the error from the PI-POD is the highest at the center of the computational domain, with the errors are lower at locations close to the boundaries. This indicates the success of PI-POD in better predicting the solution by an extra adjustment to satisfy the governing equations. On the other hand, the errors from the POD are more spread across the computational domain, with the highest errors observed at the boundaries.

Second validation data

For the second validation snapshot, the resulting prediction for PI-POD and conventional POD are plotted in Fig. 2. The accuracy of both PI-POD and conventional POD for the second validation data is summarized in Table 2. As shown in Table 2, the PI-POD model still outperforms POD in terms of overall accuracy, indicated by the higher RMSE value of the latter. Although there is still a mismatch between the true solution and PI-POD, the solution from PI-POD closely resembles the true solution better than POD. Furthermore, notice that the RMSE at the boundaries is higher for the conventional POD than the PI-POD. Another important result to notice is the difference in the error surface between the PI-POD and the POD model (see Fig. 2). The error surface of the PI-POD model spreads more evenly, with noticeably small errors around the boundaries. On the other hand, the error surface for the POD model is significantly high at locations close to the boundaries. Interestingly, the errors from POD are the highest at the boundary with zero values (i.e., 𝑥 = 0). Although the error surface of PI-POD looks symmetric compared to the first validation data, there is no particular reason why it should be symmetric. In fact, despite not being visible from the error surface, there is a slight asymmetry in the error surface of PI-POD.

Third validation data

Finally, the resulting prediction for PI-POD and conventional POD for the third validation data are plotted in figure 3. For this result, the accuracy for PI-POD and conventional POD is summarized in Table 3. The boundary condition imposed on this case has a significantly higher amplitude compared to the boundary conditions of the training data, which is why we expected that predicting this third validation data would be more difficult. Indeed, from be seen that the POD model produces a highly inaccurate prediction for this case. The reason why POD performs poor on the third validation data is because it needs to extrapolate outside the training data. On the other hand, PI-POD still managed to yield a highly accurate prediction for the third validation data. Such high accuracy is due to the incorporation of physics information that gives the ability to better extrapolate outside the training data by obeying the corresponding governing equation.

B. Poisson Equation: Case 2 Results

The second case is a more realistic application of POD because it acts more like an interpolation rather than an extrapolation, as in the third validation data of Case 1. The boxplots of RMSE for both types of POD are plotted in Fig. 4. It is interesting that the proposed method consistently outperforms the conventional POD for all variations in the number of snapshots. The better accuracy of the PI-POD for the smallest number of snapshots (i.e., ten snapshots) is a clear evidence of how readjusting the coefficients with physics information can improve the accuracy of POD reconstruction. Furthermore, it is interesting to see that the PI-POD yields lower errors even for a high number of snapshots. The results for Case 2 then show the potential of PI-POD to reconstruct high-fidelity solutions, with potential applications in uncertainty quantification and design optimization.

V. Conclusion and Future Works

This paper proposes a method that reconstructs a PDE solution based on Physics-Informed Proper Orthogonal Decomposition (PI-POD). The proposed method improves the POD-based reconstruction method by incorporating physics information to increase accuracy and force the prediction to satisfy the physics and boundary conditions. Further, the proposed method also takes advantage supervised learning method (i.e., GPR) to map between POD coefficients and variable parameters. We demonstrate the efficacy of the proposed method on a simple two-dimensional PDE (i.e., Poisson equation) in a squared-shaped domain. Our proposed method consistently outperforms conventional POD in terms of accuracy and robustness. The PI-POD method especially yields better accuracy than POD at the boundaries since the former also incorporates boundary losses in the prediction process.

For future works, we plan to study the efficacy of PI-POD on a set of physical problems with more complex PDEs. In particular, we aim to apply PI-POD on a set of structural and fluid mechanics problems. Furthermore, the use of other methods than GPR should also be studied. It is also important to investigate the impact of PI-POD parameters (e.g., 𝛼 𝐵𝐶 ), and develop an adaptive method to automatically tune the parameters.

Averaged-RMSE

PI-POD vs POD comparison

1. Case 1 :

 1 Few snapshots caseWe only use six snapshots as training data to create our model for the first case of this problem. The snapshots are created by using six variations of boundary conditions. The goal is then to reconstruct the solution field for boundary conditions outside the training snapshots. The boundary conditions for the training snapshots are written as follows:B.C. 1: 𝑢(𝑥, 0) = 0 𝑢(𝑥, 1) = sin(𝜋𝑥) 𝑢(0, 𝑦) =sin(𝜋𝑥) 𝑢(1, 𝑦) = 0 B.C. 2: 𝑢(𝑥, 0) =sin(𝜋𝑥) 𝑢(𝑥, 1) = 0 𝑢(0, 𝑦) = 0 𝑢(1, 𝑦) = sin(𝜋𝑥) B.C. 3: 𝑢(𝑥, 0) = 2 sin(𝜋𝑥) 𝑢(𝑥, 1) = -0.5 sin(𝜋𝑥) 𝑢(0, 𝑦) = 0 𝑢(1, 𝑦) = 0 B.C. 4: 𝑢(𝑥, 0) = 0 𝑢(𝑥, 1) = 0 𝑢(0, 𝑦) = -2 sin(𝜋𝑥) 𝑢(1, 𝑦) = 0.5 sin(𝜋𝑥) B.C. 5: 𝑢(𝑥, 0) = -2 sin(𝜋𝑥) 𝑢(𝑥, 1) = 2 sin(𝜋𝑥) 𝑢(0, 𝑦) = 0.5 sin(𝜋𝑥) 𝑢(1, 𝑦) = -0.5 sin(𝜋𝑥) B.C. 6 : 𝑢(𝑥, 0) = 0.5 sin(𝜋𝑥) 𝑢(𝑥, 1) = -2 sin(𝜋𝑥) 𝑢(0, 𝑦) = 0.5 sin(𝜋𝑥) 𝑢(1, 𝑦) = -0.5 sin(𝜋𝑥) The accuracy of the constructed POD models is evaluated using the following three boundary conditions as our testing data: B.C. 1: 𝑢(𝑥, 0) = 0 𝑢(𝑥, 1) = sin(𝜋𝑥) 𝑢(0, 𝑦) = -2 sin(𝜋𝑥) 𝑢(1, 𝑦) = 0 B.C. 2: 𝑢(𝑥, 0) = 0 𝑢(𝑥, 1) = 1.3 sin(𝜋𝑥) 𝑢(0, 𝑦) = -0.8 sin(𝜋𝑥) 𝑢(1, 𝑦) = 0.6 sin(𝜋𝑥) B.C. 3: 𝑢(𝑥, 0) = -6 sin(𝜋𝑥) 𝑢(𝑥, 1) = 3 sin(𝜋𝑥) 𝑢(0, 𝑦) = 7 sin(𝜋𝑥) 𝑢(1, 𝑦) = -5 sin(𝜋𝑥)

Fig. 1

 1 Fig. 1 Results for Case 1, first validation data, and the corresponding error surfaces.

Fig. 2

 2 Fig. 2 Results for Case 1, second validation data, and the corresponding error surfaces.

Fig. 3

 3 Fig. 3 Results for Case 1, third validation data, and the corresponding error surfaces.

Fig. 4

 4 Fig. 4 The RMSE boxplot of PI-POD and POD for Case 2.

Table 1 RMSE of POD and PI-POD for Case 1, first validation data.

 1

	Model	𝜖 𝑅𝑀𝑆𝐸 𝜖 𝑅𝑀𝑆𝐸 at boundaries
	PI-POD	3.35	1.12
	POD	13.13	17.61

Table 2 RMSE of POD and PI-POD for Case 1, second validation data.

 2

	Model	𝜖 𝑅𝑀𝑆𝐸 𝜖 𝑅𝑀𝑆𝐸 at boundaries
	PI-POD	18.59	1.12
	POD	20.09	31.97

Table 3

 3

	, it can

Table 3 RMSE of POD and PI-POD for Case 1, third validation data.

 3

Acknowledgement

This work is supported by The Ministry of Finance of Republic of Indonesia, through Indonesia Endowment Fund for Education Institution (Lembaga Pengelola Dana Pendidikan Indonesia), under International Conference Funding Programme for doctoral students. The authors also acknowledge the financial support from Institut Teknologi Bandung.


Introduction

This teaching and learning toolkit is designed for teachers in circus schools who wish to use video annotation as part of their pedagogy. The video annotation tool used is the free and open-source web application MemoRekall (www.memorekall.com). The toolkit is the result of the work carried out within the framework of the COSMIC project (CO-creating and Sharing digital Methodologies In Circus Education), funded by the ERASMUS+ program. This project was led by the FFEC (Fédération Française des Écoles de Cirque) and FEDEC (Fédération Européenne des Écoles de Cirque Professionnelles/European Federation of Professional Circus Schools) from 2020 to 2022.

Its objective is to introduce European professional circus schools to the use of digital technology and develop innovative teaching methods that complement traditional ones. If you want to know more about the project's approach, a white paper is available on the FEDEC website (www.fedec.eu).

The first part of the toolkit concerns teaching and learning scenarios. The design of these scenarios was the subject of various laboratories in professional circus schools. We have selected nine main scenarios developed by teachers from these schools. They are explained here in the form of templates and illustrated by capsules created with the students. For each scenario, the target audience, teaching goal, video content and annotations, different steps, and integration in the pedagogy are specified. Feel free to adapt the content of these scenarios to your practice, to your own needs, and to invent others.

The second part of the toolkit gives MemoRekall users tutorials for the advanced functions of MemoRekall (the basic functions are presented in the form of video tutorials on the site www.memorekall.com). They can be handed out to students so that they can work in the application by themselves.

EXAMPLE: https://project.memorekall.com/en/capsule/preview/inac---cyr-wheel-falls---introduction

Learn Something New

Target audience:

Students.

In the classroom:

This is a capsule you can create as a teacher, filming the steps and simultaneously guiding the student through them. Once created, you can ask other students to study it outside the class. If you identify more variations or errors, record them, upload them on YouTube, and link them to the capsule.

How to:

Choose a very precise movement you want to explain; separate out the different steps; film it; edit the video (you can add a title at the beginning + credits at the end); upload the video on YouTube; create the capsule; add text and visual annotations to give more explanation; take pictures of the positions; upload photos and add comments; identify common errors; film + edit + upload on Youtube; create the links; add the credits; share.

Teaching goal:

To help a student to learn something new, paying attention to the different steps, the most common errors, and proposing variations.

Video:

The video was created especially for the capsule. It presents the four different steps of the principles of falling from or dropping a Cyr Wheel.

EXAMPLE: https://project.memorekall.com/en/capsule/preview/fedec-video

Understanding a technical trick

Target audience:

Students and teachers.

In the classroom:

The format can vary depending on whether it is a student-led or teacher-led project.

Annotations:

Short explanations on screen captured using visual annotation to break down the exercise into different steps; explanations and integration of links/photos/ notes into the capsule.

How to:

Identify the trick that you wish to study and carry out an analysis of the movement. The key steps, with explanation, should be developed first on paper (key steps, focus on specific moments/details, external links); subsequently create the video content in the training space; edit it in the required sequence using an external video editor; add short annotations directly to the video in the editor or with the MemoRekall visual annotation tool; upload the video on YouTube; create the capsule; add more specific details with other links/photo stills/videos.

Teaching goal:

To break down a complex acrobatic movement into different steps (reverse engineering). Finding the moments where the focus is on the details of individual steps, and others where it is on introductory exercises and specific physical preparation.

Video:

The video was created especially for the capsule. Firstly it shows the technical trick (this first part can be borrowed from an existing video) and then the different steps of the movement.

EXAMPLE: https://project.memorekall.com/en/capsule/preview/inac---injury-management-and-prevention-1

Exercise Routine

Target audience:

Students.

In the classroom:

This is a capsule you can create as a teacher/coach/ trainer. You can film the sequence executed by yourself or by a student. You can talk the student through the execution or prepare everything in advance so that the video is silent. Once created, you can share the capsule with other students so they can perform the exercises outside of class by following the video when it's convenient for them. This style of capsule is relatively static once produced. If you discover that some details are not clear you can return to the capsule to add clarifying material -videos, text, images, and links.

Annotations:

Highlighted text annotations provide names for each exercise; for each section, additional videos are available that show the same exercise being executed from a different angle; still images can show critically important details (angles/alignment); text documents can be added to explain in writing some of the finer detail of the exercise or provide anatomical information.

How to:

Choose the subject of the video and prepare the appropriate sequence of movements; film the main sequence from a static angle in a single take; film the individual exercises from one or more points of view; edit the main video if needed (you can add a title at the beginning + credits at the end); upload the video on YouTube; create the capsule; add text annotations to the different parts of the video to give a name to the various exercises; add the secondary videos; add the support material (text and still images); share.

Teaching goal:

To share a specific sequence of exercises with a student to make them able to reproduce it on their own by following the video without the presence of a teacher.

Video:

The video is created especially for the capsule. It presents a sequence of exercises to achieve physical preparation and prevent injury.

In the classroom:

With this capsule, the teacher/director can apply the "reading circus" method to their class/creation OR create a how-to guide to let students work by themselves with the method the teacher wants them to use.

TEACHING AND LEARNING TOOLKIT

EXAMPLE: http://project.memorekall.com/en/capsule/preview/reading-circus-methodology-example-1

Reading Circus

Target audience:

Students.

Annotations:

The annotations show more detailed information about the "reading circus" method as well as more in-depth information about the personal creative process of the student.

How to:

Either: use this capsule to understand a part of Franziska Trapp's "reading circus" method and apply it to your own creative process OR: go through the steps provided in the capsule with your creation and document the changes and thoughts behind it. You might need to start from a shorter video and replace the video later on if you added more steps OR: document your own method to make a how-to guide. For this, it is essential to make a clear overview of the steps and decide what needs to be visible in the capsule itself first and decide what is better to attach as an annotation or other document after.

Teaching goal:

To use the "reading circus" methodology of Franziska Trapp to study the outcome of a creative process (https://www.YouTube.com/ watch?v=rvSlGJSDg4M). It can be used to learn the method as well as how to document a step-by-step working plan to feed your creativity.

Video:

The video can be created especially for the capsule or an existing YouTube/Vimeo video recording can be used.

EXAMPLE: https://project.memorekall.com/en/capsule/preview/inac-cosmic-lab-4---callums-creation

Document a Creative Process

Target audience:

Students, teachers, artists, directors, and audience.

In the classroom:

This is a capsule that the performer, the director, or an assistant can create. It can be used to show the students what to expect from a creation process as future performers or directors, or why some decisions have been made; as a source of inspiration; to share the creative experience with a new performer that enters an already existing group; to rehearse the same creation with a different performer.

Annotations:

Highlighted text annotations label the different stages of the creation process. More (non-highlighted) text annotations document the process and describe the work done daily. For each day, additional material can be included: photos of the notes taken by the performer, videos of the exercises/methods used for research, audio recordings of the various discussions that happen during creation, or photos to document different moments of the day.

How to:

Choose a placeholder video to begin the capsule; start the creation process; document as much as you can during creation -photos, videos, audio recordings, texts, books, or other sources quoted; record each draft of the piece. As the creation proceeds, you can attach the material to the capsule as you go without worrying about timestamps or placement. Once the creation period is over, edit the video: put the final version of the piece/act at the beginning, and follow it with each draft that you want to show. Finally, switch the placeholder video in the capsule with your edited video and go through the attached material to adjust the timestamps so that each annotation/file/link appears where you want it on the timeline.

Teaching goal:

An accompaniment to the creation process. It can be made purely as a "documentary" or used to feed the creative process.

Video:

The video is created especially for the capsule. It shows a montage of the different stages of the creation process. It begins with the finished act, then shows various "drafts" of the same act, each recorded at the end of each day of creation and presented in chronological order. Since the finished act was unavailable at the start of the process, the capsule was initially made with a placeholder video (a 10 minutes long black screen), which was later switched for the final edited video.

EXAMPLE: https://project.memorekall.com/en/capsule/preview/ivo-nicolau-work-in-progress-20

Feedback at a Distance

Target audience:

Students, teachers, and artists.

In the classroom: This is a capsule for students preparing their final project or collective performance. The teacher or teaching team may supervise it. The external eye can be outside the school and chosen by the teacher or student. Create several capsules during the creative process to get a timeline or journal of your project.

Annotations:

Three stages of notes. 1: notes by the student and teacher to present the project, the school, and the curriculum (all documents to help understand the project); 2: notes by the student with questions; 3: notes by an external eye answering the questions and giving feedback to the student. All the people involved can be tagged with their names to identify the contributors.

How to:

Choose what kind of feedback you require: the whole performance, a part of it, the evolution of a process, etc; film according to this choice; edit the main video if needed; upload the video on YouTube; create the capsule; add notes from the first and second stages; share the capsule with the people who will give the feedback; wait and see their comments.

Teaching goal:

To help a student or artist receive feedback at a distance from an external eye or a director during the creative process.

Video:

The video is created especially for the capsule. The example is the creative process of a student (final year project) but a performance rehearsal could be used. It can start with a presentation by the teacher or the company and then one or more steps in the creative process.

EXAMPLE: https://project.memorekall.com/en/capsule/preview/oskar-mauricio-vakuum-aerial-mouthpiece-straps EXAMPLE: https://project.memorekall.com/en/capsule/preview/project-analyse-video-cercio

Analyse a circus number/show

Target audience:

Individual students or student groups (teacher-led).

In the classroom:

The first part of this project can be a home study, with the second part of the project aimed at presenting the capsule and group discussion around it. The example is the introductory lesson, which could be developed much further (in various steps if needed) over an arc of months/years to do a full study/analysis of a complete circus show, including dramaturgy, show description, lighting, stage design, scenography, etc.

Annotations:

Using the tasks as set out by the teacher, the students add the following annotations to the capsules, using any format that may be required: history (at the beginning of the capsule, a brief introduction to the history of the discipline); biography of the artist in the video (if found); photographs of the artist in different shows; discipline used (other examples of styles); link to website (if any); link to companies (if any).

How to:

The teacher asks the students to search for a video of their discipline online on YouTube/Vimeo. They create a capsule. Then the teacher gives them tasks they need to complete over weeks (depending on the teaching aims and plan set out by the teacher). These tasks result in annotations (texts as pdf or doc files, web links, images, drawings, audio files with comments, etc.)

Teaching goal:

To analyse a circus show. This project can be developed over a period of weeks, depending on the nature and complexity of the educational aim. The project in the example was developed over three weeks to give the students time to research, develop their thoughts, create the content as homework, and then present the video.

Video:

Video of an existing performance, full length if possible, not a trailer. EXAMPLE: https://project.memorekall.com/en/capsule/preview/synaethesia

Present your performance

Target audience:

Students, teachers.

In the classroom: This is a capsule presenting the student's final project. The teacher can set the students a series of tasks that will allow them to reflect on their creative process and present this in the capsule. It can also be used to teach them how to video record a performance and what documents are required by professionals for touring.

Annotations:

There are essentially three types of annotation. 1: technical notes, descriptions of movement, and why specific moves have been chosen; 2: dramaturgical notes on narrative, character, and content; 3: documents for production and touring like the technical rider, set design plans, press package and tour dates.

How to:

Film your performance, if possible, with professional resources and edit it (it could be a trailer or the complete performance); upload the file on YouTube/ Vimeo; create the capsule; add annotations according to your needs (for example, you can add visual annotations for lighting effects if its important on tour); share and update it for the tour (for example with a new technical rider or new dates).

Teaching goal:

To document a performance for touring or to reflect on a creative process to inform the next development.

Video:

The video is created especially for the capsule, if possible, with professional resources.

Tutorials

For the basics, see the video tutorials on www.memorekall.com

USER SPACE

Capsule groups

Associate your capsules with different groups. Create groups and manage them from your user space. One capsule can be associated with various groups. And when groups are defined, add capsules to groups: screen capture Filter capsules by groups:

Change the video of a capsule

A new video can be set for an existing capsule. All annotations are kept in place.

CAPSULES

New sidebar organisation

Mosaïc view

The annotations of a video can be displayed in a dedicated mosaic view. In this mosaic view, you can group and filter the annotations according to criteria such as file type and user-defined tags.

Control playback rate

You can control the playback rate of the video in two ways: → First, define the playback rate in any annotation. Then, MemoRekall will play the part of the video corresponding to this annotation according to the playback rate chosen. This way, it's possible to slow or accelerate only one section of the video without editing it.

→ Second, override the current rate using a dedicated control in the sidebar menu. This playback rate will be defined for the whole video length.

Tutorials VIDEO EDITING (SHOTCUT)

Sometimes, it can be helpful to edit the video first before importing it to MemoRekall. Here are some free tools : Step 1:

Create a new project

To create a new project, set a project folder path > name it > select one of the video modes (SD, HD or UHD, for instance "HD 1080p 30fps" for the basic standard) > Start.

Step 2: Set the video editing layout

There are four types of editing layouts inside Shotcut: Timeline Project, Clip-only Project, Playlist Project and Player. Each of them shows you different options in the interface.

For video editing, choose Timeline Project.

Step 3: Import the media files Then drag the videos in the Playlist to Timeline area for editing.

Better enable the «Toggle snapping» option to be able to zoom in or out the timeline.

  annotations to describe more precisely what you see on the video; photos showing the wheel positions; links to other videos showing common errors or variations.

→

  Video editing: Shotcut https://shotcut.org/ (see below) → Sound editing and treatment: Audacity https://www.audacityteam.org/ → Online video editing : Youtube studio (directly on youtube when you upload your video). Tutorial: https://www.youtube.com/watch?v=UHEBB1-vPf8 → Merge several video clips together online without any software: https://airmore.com/fr/merge-video-online The main steps to edit a video in Shotcut, a free open-source video editing software available for Mac and PC, are below.

  Click «Playlist» in toolbar > drag and drop your video files there.

Submit your application for an audition

Target audience:

Candidates for professional programmes and teaching teams in schools.

In the classroom:

The capsules are viewed by the pedagogical team to select candidates.

Annotations:

The annotations include all the documents that candidates need for their application. For example, application forms, curriculum vitae and videos (self-presentation, exercises, choreography, etc.). It presents an example of the video and all documents needed for the application.

How to:

Create the video following the set format with all the figures required; upload the video on YouTube; create the capsule; add all the documents needed -in the order requested; send the capsule link to the school for application.

Teaching goal:

To provide an online application template for candidates and schools.

Video:

The video is created especially for the capsule. The capsule shows an example by the school. Each candidate will have to replace it with their own video following the template.

Tutorials

Define tags for annotations

Define and add custom tags to any annotation. Then, use those tags to filter annotations in the mosaic view. For example, you can tag people, types of annotations, techniques, performance analysis, feelings, etc.

Visual annotation

Visual annotation on a video frame

Draw on a frame of the video and add it as an annotation. The typical workflow would be : → Pause the video → Take a screenshot with your favourite tool (the "Print Screen" key, for instance) or with the clipboard button.

→ Add the screenshot as image annotation, either with the "add file annotation" or with the "add clipboard annotation," depending on where your tool puts your screenshot (on macOS, it is saved as a file, on Windows, it is saved to clipboard).

→ Draw and or add texts on the screenshot annotation as explained above.

Tutorials

Step 4: How to add text Step 5: How to crop video

Click 'Filters' in the toolbar > Video > Crop (circle, rectangle or source).

Step 6: How to split video

Move down to the Timeline area and slide the playhead to where you want to split. Click 'Split At Playhead' (right beside the 'Toggle snapping' button, or simply press S)

Step 7: How to speed up video To speed up video, change speed to any value higher than 1x (up to 50x).

To make slow motion effect, set speed value lower than 1x, like 0.5x, 0.1x, etc.

Step 8: How to add transitions Drag two or more video clips into the same video track. There shouldn't be any gap between the clips. Get two videos overlapped and then the transition zone appears. Choose a transition type in 'Properties', like Dissolve, Cut, etc.

Step 9: Export videos For interviews, or oral presentations : → 10 seconds "empty" before speaking + 5 seconds still/mute after the speech (it will help with transitions in the postproduction).

→

If possible, use a dedicated microphone so that your sound quality is maximised.

→ Try to limit gesticulations as much as possible, these will not render well on the camera.

→

You can download some free app to help you. We suggest: Android: https://play.google.com/store/apps/ details?id=norton.five.teleprompter iOS: https://apps.apple.com/us/app/teleprompter/ id941620509

Credits: you can begin the video with a title and end it with credits (director, performers, supports, etc.). It's very important if your video is being seen on YouTube or Vimeo, outside MemoRekall (for example for a trailer). Otherwise you can add credits as a note in MemoRekall (at the very beginning or at the end of the capsule).