Tutorial 1: Sparsity and Sparse Coding¶

• Effects of monocular deprivation in kittens

• Emergence of simple-cell receptive field properties by learning a sparse code for natural images

• Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition

Tutorial 2: Normalization¶

• Layer Normalization

• Probabilistic Models of the Brain: Perception and Neural Function. Natural Image Statistics and Divisive Normalization

• Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes

• Flexible gating of contextual influences in natural vision

• Normalization as a canonical neural computation

• Attention-related changes in correlated neuronal activity arise from normalization mechanisms

• Spatially tuned normalization explains attention modulation variance within neurons

• Attention-related changes in correlated neuronal activity arise from normalization mechanisms

Tutorial 3: Attention¶

• Attention and the detection of signals

• Parallel Distributed Processing

• Attention is all you need

• Inductive biases and variable creation in self-attention mechanisms

• Transformer as a Graph Neural Network

• Attention? Attention!