Cross validation

One of the classical model "re-sampling" validation techniques.
 

In cross-validation, several models based on the same architecture are built on several special subsets of the available data. The generalization performance of each of these models is estimated on the the remaining data. The results are then combined in a global estimation of the generalization performance of the model architecture (rather than of an individual model).

• "Leave-one-out" is a special case of cross-validation. Each of the training subsets contains all of the available individuals but one. If the data base contains N individuals, then N models have to be built, which makes Leave-One-Out a rather cumbersome approach. Fortunately,  it is often possible to simulate Leave-One-Out through various approximations, and therefore by-pass the actual construction of the models. Statisticians appreciate Leave-One-Out because of it is an unbiased estimator of the generalization capacity of the model.
• "Bootstrap" was initially a technique for estimating how trustworthy is, say, the measured average of a quantity. But it developped into a sophisticated re-sampling technique for estimating the generalization capacity of a model architecture. In practice, it looks quite like Cross-Validation, the main difference being in how to select  the various training sets.
  

____________________________________________________________

Related readings