Course description

581286-6 Three concepts: information (4 cu)

General Information

This course is the first course in the "Three concepts" series, and provides an introduction to information and coding theory for computer science students. In fact much of the course can be viewed as applications of Shannon's central result known as the source coding theorem. The theoretical results will be illustrated by various descriptions of practical data compression systems from Huffman coding to Rissanen's arithmetic coding. In order to demonstrate the wide applicability of information-theoretic concepts, the role of Bayesian inference in data compression is discussed, and we end the course by describing application of information-theoretic principles to (statistical) modeling, i.e., the Minimum Description Length Principle (MDL).

Who should attend?

Anyone interested in such topics as data compression, learning, neural networks, Bayesian inference, data mining or artificial intelligence. This course is intended to give you new ideas, approaches or tools for your problems. The course is aimed at senior undergraduates and graduate students and is required for students pursuing in M.Sc. or Ph.D. in the field of Adaptive and Intelligent Systems.

Note that the maximum number of participants is 20. You can registrate by filling in the registration form.

What prerequisites are required?

The course is an introductory course, and only elementary knowledge on information theory is required. Different parts of the course, however, have different requirements with respect to the mathematical machinery needed to apply the concepts in question. Typically only elementary analysis and probability theory are needed. The course projects involve programming, thus programming skills are necessary.

What is needed to get the credit (4 cu) for the course?

The course does not have regular weekly assignments, but beware - this does not mean that there is no work involved! In addition to participating to the classes, the students are expected to

finish 2 projects during the course (50% of the grade)
prepare a poster presentation for the joint poster session during the course (30% of the grade)
write a "term paper" at the end of the course (20% of the grade)

Much of the project work is performed in groups (size of these groups depends on the number of participants). Participation to the classes is not enforced, but strongly encouraged, as many of the preliminaries for the projects will be discussed during the classes.

What if I find out that the course is not for me and want to drop the course?

During the first two classes of the course anybody can drop out by just sending email to the instructor. After this introductory period the project work starts, poster topics will be assigned etc. and the students are expected to be committed to the course. Also at this point the vacant slots will be filled from the pool of students on the waiting list.




3 Concepts: Information	2002