A normalização para cada neurônio das redes neurais
So, considering the method BatchNormalization. Each batch is normalized in the training for a given batch.
Each neuron has a Gaussian distribution. And this distribution is achieved for that batch, Karpathy stated.
The preactivations are normalized.
Really great learning.