Aapo Hyvärinen: Publications

Publications by topic

Unsupervised deep learning Causal discovery Estimation theory Brain imaging data analysis Natural image statistics The FastICA algorithm Independent component analysis Further unsupervised learning Experimental vision research

Publications front page

Home page

Natural image statistics and the visual cortex

[Why are the receptive fields in the visual cortex as they are? The modern answer to this question emphasizes adaptation to the statistical structure of ecologically valid stimuli (natural images). Our work models complex cell receptive fields, topography, and even V2]

General

A. Hyvärinen, J. Hurri, and P. O. Hoyer. Natural Image Statistics. Springer-Verlag, 2009.
Book home page
[A monograph-textbook on statistical models of natural images and their application in computational neuroscience.]

A. Hyvärinen Statistical models of natural images and cortical visual representation. Topics in Cognitive Science 2:251-264, 2010.
pdf
[A short review on models of natural image statistics and the visual cortex.]

Beyond primary visual cortex

H. Hosoya and A. Hyvärinen. A mixture of sparse coding models explaining properties of face neurons related to holistic and parts-based processing. PLoS Computational Biology, 2017.
Open Access article
[A model of face processing combining inference in a mixture model with sparse coding.]

H. Hosoya and A. Hyvärinen. A Hierarchical Statistical Model of Natural Images Explains Tuning Properties in V2. J. of Neuroscience, 35:10412-10428, 2015.
pdf
[Estimates a new variant of ICA (sparse coding) on simulated complex cell outputs to predict V2 properties. Develops a classification of cell types and finds new type related to corner detection. The results have excellent consistency with neurophysiological measurements.]

M. U. Gutmann and A. Hyvärinen. A Three-layer Model of Natural Image Statistics. J. of Physiology (Paris), 107:369-398, 2013.
pdf (Note: 21MB)     matlab code
[Develops a full three-layer model of natural image statistics, analysing both patches of large images, and whole image downsampled to be tiny.]

H. Sasaki, M. U. Gutmann, H. Shouno and A. Hyvärinen Correlated Topographic Analysis: Estimating an Ordering of Correlated Components. Machine Learning, 92:285-317, 2013.
pdf     Matlab code
[A new principle for topographic formation, and a new model of depdendencies of linear components, featuring linear correlation instead of energy correlations.]

H. Sasaki, M. U. Gutmann, H. Shouno and A. Hyvärinen Simultaneous Estimation of Non-Gaussian Components and their Correlation Structure. Neural Computation, 29:2887-2924, 2017.
pdf
[Shows how to estimate the connections in the model of the preceding paper, leading to a general-purpose model of components with dependencies, including linear correlations. Based on AISTATS2014 paper.]

A. Hyvärinen, M. Gutmann and P.O. Hoyer. Statistical model of natural stimuli predicts edge-like pooling of spatial frequency channels in V2. BMC Neuroscience, 6:12, 2005.
Open access journal article
[Analyzes the independent components of complex cell outputs, and shows that they are higher-order contour coding units that pool multiple frequency bands. An example of prediction of new kinds of cells based on natural image statistics.]

P.O. Hoyer and A. Hyvärinen. A Multi-Layer Sparse Coding Network Learns Contour Coding from Natural Images. Vision Research, 42(12):1593-1605, 2002.
Postscript  gzipped PostScript   pdf
[Earlier work on the same subject as the preceding paper.]

J.T. Lindgren, J. Hurri and A. Hyvärinen.Spatial dependencies between local luminance and contrast in natural images. Journal of Vision 8(12)1-13, 2008
Open access article
[Investigates dependencies between local contrast and luminance.]

J.T. Lindgren and A. Hyvärinen.On the Learning of Nonlinear Visual Features from Natural Images by Optimizing Response Energies. Proc. Int. Joint Conf. on Neural Networks (IJCNN2008), Hong Kong, 2008.
pdf
[Considers learning of conjunctive features (e.g. corner detectors) from natural images, continuing the work in the following paper.]

J.T. Lindgren and A. Hyvärinen. Emergence of conjunctive visual features by quadratic independent component analysis. Advances in Neural Information Processing Systems (NIPS2006),
pdf
[Investigates what happens when ICA is done in a quadratic feature space with natural images. Emergence of features with quite unexpected properties.]

J.T. Lindgren and A. Hyvärinen. Learning high-level independent components of images through a spectral representation. Proc. Int. Conf. on Pattern Recognition (ICPR2004), pp. 72-75, Cambridge, UK, 2004.
gzipped PostScript   pdf  
[Shows how to learn independent components that characterize whole images (scenes).]

Sparse coding / ICA models of V1

H. Hosoya and A. Hyvärinen. Learning Visual Spatial Pooling by Strong PCA Dimension Reduction. Neural Computation, 28:1249--1264, 2016.
pdf
[Proposes a very simple method for learning pooling in e.g. complex cells based on PCA and denoising.]

M. U. Gutmann, V. Laparra, A. Hyvärinen and J. Malo. Spatio-Chromatic Adaptation via Higher-Order Canonical Correlation Analysis of Natural Images . PLoS ONE, 9(2):e86481, 2014.
paper   code  
[Proposes a general model on how to find energy-correlated components in two datasets, and applies it to colour coding.]

V. Laparra, M. U. Gutmann, J. Malo and A. Hyvärinen. Complex-valued independent component analysis of natural images . Proc. Int. Conf. on Artificial Neural Networks (ICANN2011), Helsinki, Finland, 2011.
pdf  
[Shows how simple and complex cell receptive fields are easily learned from natural images by just doing complex-valued ICA in the Fourier domain.]

U. Köster, J. Lindgren, M. Gutmann and A. Hyvärinen. Learning Natural Image Structure with a Horizontal Product Model. Proc. Int. Conf. on Independent Component Analysis and Blind Source Separation (ICA2009), Paraty, Brazil, 2009.
pdf  
[Provides a new kind of generative model in which the variances of pixels are modelled by another ICA-like model.]

U. Köster, J. Lindgren and A. Hyvärinen. Estimating Markov Random Field Potentials for Natural Images. Proc. Int. Conf. on Independent Component Analysis and Blind Source Separation (ICA2009), Paraty, Brazil, 2009.
pdf  
[Estimates the features used in a Markov random field, which turn out to be Gabor-like. Provides a model for whole images instead of just patches.]

U. Köster and A. Hyvärinen. A Two-Layer Model of Natural Stimuli Estimated with Score Matching, Neural Computation, 22:2308-2333, 2010.
pdf
[Shows how to learn weights both layers of a two-layer model which generalizes ICA, and the models below.]

A. Hyvärinen and P. O. Hoyer. A Two-Layer Sparse Coding Model Learns Simple and Complex Cell Receptive Fields and Topography from Natural Images. Vision Research, 41(18):2413-2423, 2001.
Postscript  gzipped PostScript  pdf
[Uses a simplified version of topographic ICA (see next) to model the topography (spatial organization) of the cells in primary visual cortex, in addition to simple cell and complex cell receptive fields.]

A. Hyvärinen, P.O. Hoyer and M. Inki. Topographic Independent Component Analysis. Neural Computation, 13(7):1527-1558, 2001.
Postscript  gzipped PostScript   pdf
[Introduces an extension of ICA. The dependencies of the estimated "independent" components are visualized as a topographic order. A new principle for topographic organization, based on higher-order statistics. Applied on image data, both topography and complex cell properties emerge (see preceding paper). This paper is more concentrated on the mathematical formulation, and the preceding one on the biological application.]

A. Hyvärinen and P.O. Hoyer. Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces. Neural Computation, 12(7):1705-1720, 2000.
Postscript  gzipped PostScript  pdf
[A simpler extension of ordinary ICA which was a precursor of topographic ICA. Here the goal of independence of scalar independent components is replaced by the independence of the norms of projections on certain subspaces. This is applied on image data, and complex cell properties are shown to emerge.]

A. Hyvärinen and P. O. Hoyer. Emergence of Topography and Complex Cell Properties from Natural Images using Extensions of ICA. Advances in Neural Information Processing Systems 12 (NIPS*99), pp. 827-833, 2000.
Postscript  gzipped PostScript  pdf.
[A short summary of the three preceding papers.]

P.O. Hoyer and A. Hyvärinen. Independent Component Analysis Applied to Feature Extraction from Colour and Stereo Images. Network: Computation in Neural Systems, 11(3):191-210, 2000.
Postscript  gzipped PostScript  pdf
[Shows that the independent components of colour and stereo images are quite similar to the corresponding V1 receptive fields.]

A. Hyvärinen. An alternative approach to infomax and independent component analysis . Neurocomputing, 44-46(C):1089-1097, 2002.
Postscript  gzipped PostScript  pdf
[Considers the problem of nonrobustness of the ordinary infomax approach to ICA, and proposes a solution to this problem, based on a more biological noise model.]

A. Hyvärinen and U. Köster. Complex Cell Pooling and the Statistics of Natural Images . Network: Computation in Neural Systems, 18:81-100, 2007.
pdf
[Shows that independent subspace analysis (see above) is a really better model for natural images, in the sense of a statistical criterion, than independent component analysis.]

MATLAB code for estimating ICA, ISA, and TICA bases from image data is also available (by P. O. Hoyer).

Temporal coherence models of V1

J. Hurri and A. Hyvärinen. Simple-Cell-Like Receptive Fields Maximize Temporal Coherence in Natural Video. Neural Computation, 15(3):663-691, 2003.
Postscript  gzipped PostScript   pdf
[Analyzes the temporal correlations of natural image sequences, and shows that temporal coherence (correlation) leads to emergence of the same kind of receptive fields that have previously been found by ICA (sparse coding).]

J. Hurri and A. Hyvärinen.Temporal and spatiotemporal coherence in simple-cell responses: A generative model of natural image sequences, Network: Computation in Neural Systems, 14(3):527-551, 2003 (special issue on sensory coding and natural stimuli).
pdf
[Shows how both layers in a two-layer model can be estimated using temporal coherence criteria. One of the layers learns simple-cell receptive fields, and the other learns something similar to cortical topography and complex cell pooling.]

J. Hurri and A. Hyvärinen.Temporal Coherence, Natural Image Sequences, and the Visual Cortex. Advances in Neural Information Processing Systems 15 (NIPS*02), MIT Press, pp. 141-148, 2003.
Postscript  gzipped PostScript   pdf
[A short summary of the two preceding papers.]

A. Hyvärinen, J. Hurri, and J. Väyrynen. Bubbles: a unifying framework for low-level statistical properties of natural image sequences. Journal of the Optical Society of America A, 20(7):1237-1252, 2003 (Special Issue on Bayesian and statistical approaches to vision).
Postscript  gzipped PostScript  pdf
[An unifying framework that combines sparseness / ICA, temporal coherence, and topography / complex cell pooling in a single model.]