Phase Transitions in Sparsely Connected Boltzmann Machines

Serge Santos: Phase Transitions in Sparsely Connected Boltzmann Machines. Diploma work, Report C-1994-15, Department of Computer Science, University of Helsinki, April 1994. 62 pages. <http://www.cs.helsinki.fi/TR/C-1994/15>

Full paper: gzip'ed Postscript file
Metadata: XML file

Abstract

The primary motivation of this work has been the study of phase transitions in the simulated annealing schedules of Boltzmann machines.

At first, we introduce and focus on the harmonium Boltzmann machine, which can be used to implement a Bayesian reasoning scheme. These sparsely connected Boltzmann machines are composed of two connected clusters of independent nodes, allowing a massive parallel updating scheme.

The study of critical phenomena in simulated annealing presupposes a deeper understanding of Monte Carlo methods, especially in connection with statistical physics.

We discuss various aspects of the influence of the correlation between the elements of Monte Carlo samples on the determination of the relevant physical quantities.

The close analogy between Boltzmann machines and spin glass models is brought to the fore in our investigations, by computational simulations, of the thermodynamical properties of the harmonium Boltzmann machine. The study of the specific characteristics of this model contributes to a closer comprehension of the emergence of critical phenomena, corroborated by the results of our simulations.

For the larger framework of sparsely connected Boltzmann machines, we extensively present the theoretical methods of statistical physics applied to strongly diluted spin glasses. Finally, the predicted equilibrium properties are compared with the empirical results.

Index Terms

Categories and Subject Descriptors:
D.3.4, F.1.1, G.1.6, Y.2.1

General Terms:

Additional Key Words and Phrases: