Preface
2. Model Training
Model training forms the backbone of the entire modelling process just like how studying constitutes the majority of the study experience in a university course. Model training essentially optimises the performance of the model over the training set by searching for optimal values of its parameters.
2.1 The Goal of Model Training
Given a model type and set hyperparameters, model training aims at finding values for the model parameters so that the model performs reasonably well over the training set. Under the context of supervised learning, we would like the model to predict on average satisfactorily close to the actual observations. The distances between the predictions and actual observations are quantified by loss functions. For example, mean squared errors capture squared Euclidean distance between predictions and actual observations.
The goal of model training is to minimise such loss over the training set so that the model has a decent performance at least over the training set. This is achieved by updating its parameters iteratively so that loss decreases successively.