TRAINING COURSE "SUPERVISED NEURAL NETWORKS"

Supervised Neural networks are simply the best regression models at this time, thanks to their capacity to approximate just about any regression function. Of course, this capacity translates into exceptional performances in probabilistic classification. These qualities are now widely acknowledged, and just about every major Data Mining software now incorporates supervised Neural Networks.

Yet, this very power is a double-edged sword, and many Neural Networks-based projects end up in closets because of a lack of understanding of their basic inner workings. As a consequence, these truly wondeful tools are widely underused, and their reputation is not up their true capabilities.

Depending on previous background, this 1, 2 or 3 days training (see outline below) will bring you up to the necessary level for fully taking advantage of the power of the most popular supervised Neural networks. The training is illustrated with many demonstrations done with commercial software.
 

Course outline

General principles

The concept of "universal approximation"

Universal approximation and regression

Regression and probabilistic classification

The Multilayer Perceptron (MLP)

The "standard" neuron

Activation function

Role of the weights : slope and orientation

Threshold (or bias) : a special weight

Architecture and main properties of the MLP

Role of the hidden layer

How many neurons in the hidden layer ?

Classification : output layer and a posteriori probabilities

MLP and parcimonious approximation

Training of a MLP

Chosing an error function in classification

Initialisation before training

Descent optimization and local minima

First order training algorithms ("BackProp")

Second order training algorithms (Quasi-Newton, Levenberg-Marquardt, Conjugate Gradients)

Overparametrization and overtraining, validation set

RBF networks

The notion of "influence zone"

What are "Radial Basis Functions" ?

Architecture and main properties of RBF networks

Training of RBF networks

Gradient descent methods

Heuristics

MLP or RBF : which one should you chose ?

Training

Generalization capacity

How to use (supervised) Neural networks

When should you use (or not use) Neural networks ?

Comparison of Neural networks with other regression techniques (Linear Regression, Multiple Linear Regression), and classification techniques (Discriminant Analysis, Logistic Regression, Decision Trees)

How do you estimate the generalization capacity of a Neural Network ?

Validation, multiple validation

Re-sampling methods : cross-validation, "leave-one-out", bootstrap

Leverage and confidence interval. Simulated  Leave-one-out

How can you improve the performance of a Neural Network ?

Chosing the right variables, dimensionality reduction

How to optimize the hidden layer

"Early stopping", regularization, pruning

Cooperation between several networks in regression and classification