Validating a model is trying to estimate its generalization capacity.
Now recall that the true quality of a model cannot be infered from its performance on data that was used to build it in the first place (except for exceptional cases, like Linear Regression in ideal conditions, see Tutorial on Linear Regression). As a matter of fact, it is quite common to come across models whose performance on "building data" is good (but illusory), and that behave poorly once put in the field in real world conditions.
Validating a model is a delicate and time consuming endeavor. Classical approaches to validation include :
- Some mathematical theories, that we can't describe here, proceed with a linear approximation (in parameter space) of the model, from which an estimation of performance of the model can be derived. These new techniques are beginning to be implemented in the context of supervised Neural Networks.
Related readings :