4.2. ARAŞTIRMANIN MODEL VE HİPOTEZLERİ
4.2.2. Araştırmanın Hipotezleri
6.1 Data preprocessing and transformation
Machine learning algorithms, like the Gaussian processes, are used to automatically extract information from machine-readable datasets. The success of the ML algorithms usually depends on the quality of the data that is fed to the algorithm, where data of high quality will lead to results of high quality. Data preprocessing is an umbrella term that covers many different methods and techniques that all aims to increase the quality of the data before it is fed to the ML algorithm [43].
The measurement unit of the features in a dataset can affect the forthcoming data analysis. For example; a feature measured in kilograms or grams, or measured in meters or centimeteres, may lead to different results. Data normalization or data transformationare techniques that can be applied to a dataset to avoid dependence on the choice of measurement units [31]. The words transformation or normalization will in this thesis be used interchangeably in the same context.
Data normalization is performed on the dataset to scale the maximum and minimum values of the features to lower values such as [-1,1] or [0.0, 1.1]. This will, in gen-eral, give the feature a larger range [31]. The two most common methods for data transformation will be presented in what follows.
For the two transformations, let A denote a feature, which can be represented by nu-55
meric values and letx =[x1, ...,xn]be the vector ofnobservations of A. Furthermore, letx0 denote the vector ofnnormalized observations of A.
TheMin-max normalization performs a linear transformation of the original ob-served values inx. Let minA denote the minimum value of A, and let maxA denote the maximum value of A. The min-max normalization maps a valuexof A tox0, where allxi0 are in the "new" range [min
0
A,max0A]. The mapping is achieved by computing x0 = x −minA
Thezero-mean normalizationare a normalization that are based on the (sample) mean value of A, denoted ¯Aand the (sample) standard deviation of A, denotedsA. The zero-mean normalization is defined as,
x0 = x−A¯
sA , (6.1.2)
where the sample mean of the observations of feature A can be computed by A¯= 1n
n
X
i=1
xi fori=1, ...,n. (6.1.3)
Furthermore, the sample standard deviation,sA, is the positive square root of the sample variance, that in this setting is given by
sA2 = n1
n
X
i=1
(xi−A¯)2, fori =1, ...,n. (6.1.4)
The both normalization techniques are achieved from, [43] and [31], which can be consulted for further preprocessing techniques.
6.2 Cross validation
Cross validation, CV, will be used to compare different data configurations or experi-ments with each other. This is a statistical method for comparing and evaluating the outcome of different machine learning models against each other. Cross validation partitions the dataset into parts, i.e. groups, where one part will be used to train the GP or ML regression model, and the other to validate i.e. test the model and its performance [66]. The three most common methods for CV are called theHold-out Validation,Leave-one-out Cross-validationandK-fold cross validation[65, 66, 81].
The so-called Hold-out Validation it is both simple and fast [13]. In the hold-out validation method, the whole dataset is divided into two disjoint sets, a training and
a testing set. The test will be completely independent of the training set, and will only be considered after the model has been trained. This method will only iterate through all points in the dataset once, which imply that it will be less time-consuming than the other two CV methods. The downside of this method is that it is highly dependent on the choice of the training/test splits [66]. Furthermore, a small test set can give rise to a performance estimate that may have large variance [65].
In thek-fold cross validation, abbreviated k-fold CV, the dataset consisting ofn obser-vations is partitioned intok equally sized, disjoint sets.k sequential iterations are performed, one for each of the k-folds. In each of thek iterations, the test set will consist of thekth k-fold, while the training set is the union of the otherk−1 sets [66]. The procedure will be repeatedktimes, such that allksubsets will be used in both a training and test set [65]. Figure 6.1 shows an illustration of k-fold CV, where the data has been portioned into k = 8 k-folds. The grey box indicates which part of the data that is considered for testing, while the green boxes are combined into the training set.
The overall performance of the model is computed by, for example, the k-fold Bias, that is presented in the following section.
Figure 6.1:Illustration of k-fold CV, with k = 8 folds. The whole available data set consist of all the data in the eight boxes. The grey box indicate the part of the data that is considered for testing in one round, while the green boxes indicates the training data.
TheLeave-one-out Cross-validationmethod, also denoted LOOCV, is a special case of the k-fold CV withk=n. This implies thatniterations are performed, one for each observation in the dataset. For each of theniterations; one single observation at the time is considered to be the test set while the restn−1 observations are considered to be the training set [66]. Whenn is a large number, this model selection method can come with a large computational cost [65]. Similar to k-fold CV, the overall performance of the model can be computed by for example the k-fold Bias.
LOOCV is a popular method when the size of the whole dataset is small [66], but performing k-fold CV is less time consuming than the LOOCV, as onlyk iterations are performed instead ofN. It is assumed thatk n. This thesis will only consider k-fold cross validationsince it would be to time consuming to perform LOOCV on the dataset that this thesis considers.
6.3 Performance validation
This section will define different validation methodologies that in Ch. 7-Ch. 9 can be used to evaluate the outcome from different outcomes from the k-fold CV. The following three statistical parameters,
• the root mean squared error, henceforth RMSE
• the Bias
• the standard deviation error, henceforth STDE
have been used in several different settings to evaluate the model performance, see for example [15, 47, 91], and will therefore also be used in this thesis.
For this section, let f∗,k denote the true output in the test set of the kth k-fold, thus f∗,k = [f(x1∗,k), ...,f(xj∗,k)]. Furthermore, let ˆf∗,k denote the estimated/pre-dicted output of the GP or ML regression model, from the kth k-fold, i.e ˆf∗,k = [fˆ(x1∗,k), ...,fˆ(xj∗,k)]. The indexjdenote the size of a k-fold in both cases.
In addition, let f∗ denote the true output from all the 600 k-folds, i.e. f∗ = [f(x1∗), ...,f(xn∗)], wheren = 18,000 in this thesis. Furthermore, let ˆf∗ denote the estimated/predicted output of the GP or ML regression model fromall the the 600 k-folds, i.e. ˆf∗ =[fˆ(x1∗), ...,fˆ(xn∗)].
The overall RMSE can then be computed by RMSE =
A small overall RMSE, i.e. close to zero, indicates that the estimated output is close to the true output, while a larger overall RMSE indicates that estimated output deviates from the true output. The square root in Eq. 6.3.1 guarantees that the overall RMSE will have the same unit as ˆf(xi∗)and f(xi∗)[91].
An interpretation of the overall Bias or the k-fold Bias, abbreviated Biask will be
used to evaluate the data tendency. A positive overall Bias or Biask indicates that the predicted outputs tend to be an overestimate of the true output, while a negative overall Bias or Biaskindicates that the predicted outputs tend to be an underestimate of the true output [15]. The overall Bias is defined as,
Bias= n1 Xn i=1
fˆ(xi∗)− f(xi∗)
, (6.3.2)
while Biask is defined as,
Biask = N1 XN i=1
fˆ(xi∗,k)− f(xi∗,k)
, (6.3.3)
whereN indicates the size, i.e. the number of observations/data points of a k-fold.
Plotting a histogram over Biaskfori =1, ...,kcan give an indication of the distribution of the Bias for the k-fold CV of an experiment. Both the Bias and the Biask will have the same unit as ˆf(xi∗)and f(xi∗).
The overall STDE can be computed through the overall RMSE and the overall Bias in the following way,
STDE=q
RMSE2
− Bias2. (6.3.4)
The square root in the expression of the overall STDE ensures that it will have the same units as the RMSE and the Bias.
The overall RMSE, Bias and STDE will be used to compare the overall performance of different experiments and investigations. The overall STDE will be prioritised when different experiments are be compared to each other [15], since it both considers the error through the RMSE and the tendency through the Bias.
6.4 BIC
The Bayesian information criterion, abbreviated BIC, will be used when different GP models are compared with each other, see Sec. 8.3. The choice of using BIC is inspired by the work presented by Duvenaud [26] who chose to use this criterion because of its simplicity for model comparison, since it can be used to approximate the integral of the marginal likelihood over all free parameters in the GP model [71].
BICρ =log
whereρdenote number of kernel parameters in the GP model,ndenote the number of observations/data points in the dataset. BIC should penalize the marginal likelihood of a GP model in proportion to how many parameters the model has. The model with the highest BIC will be considered to be the optimal [26].
6.5 Comparing the quantity of catch
Two additional measurements will be considered in the forthcoming experiments an analysis in order to set the Biask introduced in Sec. 6.3 in context to the expected average quantity of catch in the k-folds. The actual average quantity of catch per k-fold, denoted Quantk-foldcan be computed by the following,
Quantk-fold= 1k Xk
j=1
XN i=1
f(xi∗,k, forj=1, ...,kandi =1, ...,N, (6.5.1)
wherej denote the index of thekthk-fold,kis the number of k-folds and N is the number of observations/data points in a k-fold. Similarly can the average predicted quantity of catch per k-fold,QuantFk-foldbe computed by,
QuantFk-fold= k1
k
X
j=1 N
X
i=1
fˆ(xi∗,k) forj=1, ...,kandi =1, ...,N, (6.5.2)
recalling that ˆf(xi∗,k) denote the predicted quantity of catch for observation i in k-foldkwhile f(xi∗,k)denote the actual quantity of catch for observationi in k-fold k. Both Quantk-foldandQuantFk-foldwill in this thesis be given in kg.