There are several variations of the cross-validation algorithm. In the k-fold cross-validation, we have k-fold, and we divide the training set to multiple folds and run k-fold cross-validation. In some cases, usually when we are running machine learning diagnosis tests to see if our problem suffers from high variance or high bias, we split the dataset to 3 different splits of train, validation, and test, and measure the trained model on train set on all of these 3 splits by adding more data points each time.
Another usage for having a separate validation set is when we have a complex model which takes a long time to train or when we deal with big data. In any of these two cases such as when we are training deep neural networks, k-fold cross validation is so expensive. In each epoch, we just validate the model trained by the validation set and based on the results on the validation set, we continue to update the hyper-parameters. We also compare the results with the test set that we have never used during the training to make sure our training can generalize.
Courses
Feedback