Cross Validation
1 Motivation
- We want to know how our model will generalize
- We want to make sure that our results aren't simply the result of getting lucky with a particular train/val/test split
2 Procedure
- Split the data into \(k\) groups.
- For each group, hold it out as a validation set and train on the remaining \(k-1\) groups. Evaluate on the held out group and write down the evaluation results.
- Summarize the \(k\) evaluation results, e.g. mean and standard deviation