Chaos and Complexity Courses for Fall 2004, UW-Madison

Statistics 840

Statistical Model Building and Learning
(Stat 709 NOT required, ok for nonmajors, see prereq below)
T Th 4:00-5:15   Fall 2004
Room 5295 MED SC CTR (1300 University Ave)
Grace Wahba, Instructor

This course is about various aspects of statistical model model building, supervised machine learning and multivariate function estimation given scattered, noisy, direct, and indirect data, primarily using reproducing kernel Hilbert space (rkhs) methods, regularization, and splines.

1. Background, introduction to the theory of reproducing kernel Hilbert spaces (rkhs). Varieties of splines on various domains. Representer theory. Connections between smoothing splines, Bayes estimates, optimization problems in rkhs and regularization.

2. Degrees of freedom for signal and the bias-variance tradeoff. Generalized cross validation, generalized approximate cross validation, unbiased risk, maximum likelihood and other model tuning methods.

3. Model selection and model building methods suitable for spline and related models. Bayesian and bootstrap confidence intervals. Penalized likelihood models for risk factor modeling. Two and multicategory support vector machines, and other large margin classifiers, 'hard' and 'soft' classification.

4. Numerical methods for medium sized to very large data sets. Randomized trace estimation for the degrees of freedom for signal. Early termination of iterative methods as a form of regularization. Basis function thinning methods.

5. Applications in biostatistics and bioinformatics (risk factor modeling, classification), statistical learning theory (supervised machine learning, support vector machines), meteorology (ill-posed inverse problems, remote sensing, tuning, variable selection and classification), physics (signal detection), and other areas, will be discussed, according to the interests of the class.

Prerequisites: - Statistics Majors, mathematical maturity to the level of a year of graduate work, and either multivariate analysis, or, some exposure to Hilbert spaces, or cons. instr. Those unfamiliar with Hilbert spaces will be asked to read the first 33 pages of Akheizer and Glazman, Theory of Linear Operators in Hilbert Spaces, vol. I at the beginning of the course. Graduate students in CS, AOS, ECE, Biostatistics, Physics, and other physical sciences, Engineering, Math, Economics, and Business may find some of the techniques studied here useful and are welcome to sit in, or, take the course for credit if they have exposure to linear algebra, sufficient math background to read the Akhiezer and Glazman assignment, and are familiar with the basic properties of the multivariate normal distribution, as found, e. g. in Anderson, Multivariate Analysis, or Wilks, Mathematical Statistics. Otherwise, the development will be self-contained. If in doubt, please contact the instructor by e-mail ( or come to the first class. This will be a seminar-type course. There will be no sit-down exams. Students taking the course for credit will be expected to do one or two computer projects studying the behavior of some of the methods discussed on simulated or experimental data, and one or two projects in an area of application of their choice with a possible project being the presentation of a lecture in class on a recent paper or recent resarch. Text:  Wahba, Spline Models for Observational Data (1990). Material from selected recent papers, books and conferences will be discussed, tba.

Math 801 - Topics in Applied Mathematics  (Prof. Amir Assadi)                          2:30-3:45

Descriptive Title: Biological Computation and Mathematics with Applications to Learning in Intelligent Systems
Prerequisite: Graduate standing or consent of instructor for Undergraduate students.
        (1)    Mathematical Biology, 2nd Edition or later, by J.D. Murray. Springer ISNB 0-387-95228-4
        (2)    (Recommended) Dana Ballard, Introduction to Natural Computation (MIT Press, ISBN 0-262-02420-9. . Soft cover also available but with a different ISBN);
        (3)    (Recommended) Evolution of Networks, by S.N. Dorogovtsev and J.F.F. Mendes. Oxford University Press. ISBN 0-19-851590-1

DESCRIPTION: This course will treat topics in Biological Computation and Mathematics. Its objective is to introduce the students to selected research topics in cross disciplinary mathematics, computational biology, bioinformatics, and modeling complex biological systems. The lectures will discuss the mathematical foundations and computational methods for topics from four biological space/time scales: (a) biomolecular information processing (with very small time scales for events to take place), (b) modeling spiking neurons (micrometer length with milli-second time scale), (c) biological learning at system level (millimeters long or higher lengths and seconds/minutes time scale), (d) evolution of biological information processing at population level (large space and large time scales). I will cover the following from the above-mentioned topics: (a) DNA computation, its mathematical and computational challenges and promises, as well as selected methods for analysis of data in bioinformatics, such as analysis of the gene-chip and micro-array data (used in the Human Genome Project, for example); (b) modeling neurons and excitable cells, with a brief introduction to dynamical systems related to such models; (c) Neural networks and biological intelligence, memory and learning (for example, the sensory systems in the human brain, with discussion of some concrete applications to one or more of vision, audition, and pain); (d) evolutionary computation, such as genetic algorithms and programming, and their applications in optimization and solution of real-world problems. Undergraduate mathematics as typically covered by science/ engineering students will be assumed. I will review advanced topics as needed for exploring the selected biological topics. I will also review the related molecular biology, the basic cell biology of brain cells and basic facts from evolutionary biology. There will be a tutorial for students who need to get started with hands-on computation with MATLAB as needed for term projects. The course grade is based on a term project. Sample projects could be found by following the appropriate links in my web page: