EECS 659

Adaptive Signal Processing - Fall 2011  
Tu-Th 9:00-10:30, 1012 EECS

Instructor: Alfred Hero  
Textbook: There is no required textbook. See below for references.
How to reach me:
Office: 4417 EECS
Tel.: 763-0564

email: hero at eecs.umich.edu
Office Hours: T 12:00PM-1:30PM and Th 10:30AM-12:00PM and by appointment
  • Ctools web site: .html (Contains electronic course materials)

  • Final exam will be posted here on Wed 14 Dec. Return by 5PM on Wed. Dec 21, 2011.

    Course description

    Adaptive signal processing refers to a collection of theory and design techniques which seeks to develop detection, estimation, or filtering algorithms which operate reliably in uncertain or changing environments. In many cases additional constraints are placed on adaptive signal processing algorithms such as: data recursivity, low complexity, low power, real-time operation, and parallelizability. Much of the early work on adaptive signal processing was in the context of linear prediction using least-squares theory. This leads to the Widrow LMS algorithm, recursive least squares, and other variants. More recently, methods of stochastic filtering and machine learning have made an impact. The course will cover the theory and practice of linear and non-linear adaptive estimation, classification, and filtering. Applications will be drawn from signal and image processing areas including: communications, network security, biomedical signals, antenna array processing, image processing, sensor networks, and remote sensing.

    Course Requirements

    The only exam will be a takehome final. 50% of your course grade will be based on this takehome final with the other 50% based on an individual course project. More details on the final and the project are given below.

    Some important information

    There are a number of class meetings when I will out of town and have scheduled makeups. Here are the dates on which class is canceled and the makeup times and dates. Unless otherwise indicated, makeups will be held in our normal classroom (EECS1012).

    CLASS CANCELED
    		 MAKEUP (EECS????)
    T Sept. 27 and Th Sept. 29
    		 M Sept. 12 and M Sept. 19, 3:30-5:00PM
    Th Oct 13 and Th Oct 20
    		 M Oct 3 and M Oct 10, 3:30-5:00PM
    T Oct 25 and Th Oct 27
    		 M Oct. 31, 3:30-5:00PM
    T Nov 7 and T Nov 15
    		 M Nov 28, 3:30-6:30PM
    T Dec 13
    		 M Dec 5, 3:30-5:00PM

    Lecture topics and background reading

    Weeks 1-6: Offline and online adaptive estimation: linear filtering with quadratic loss:
  • LMS and RLS: Chs 1-5 of "Adaptive Signal Processing," Kang and Solo, 1995.
    Weeks 6-8: Adaptive estimation: non-linear Bayesian filtering
  • Bayesian filtering in discrete time: the Chapman-Kolmogorov equations and Laplace's approximation (.ps)
  • Bayesian filtering theory in continuous time: Fokker-Planck equations and discrete-continuous implementation. See Poor textbook, Ch. VII, Section D, "An introduction to signal detection and estimation," (.html) and Jazwinsky texbook Ch. 5, Section 3, "Stochastic processes and filtering theory," Academic Press 1970.
    Weeks 8-9: Monte-Carlo approaches to adaptive estimation and filtering
  • Chs 2 and 3 of "Monte Carlo Strategies in Scientific Computing," Jun Liu, 2001.
  • Particle filtering: A. Doucet, S. Godsill, C. Andrieu, "On sequential Monte Carlo sampling methods for Bayesian filtering," Statistics and Computing, vol. 10, no. 3, pp. 197-208, 2000. (.html)
    Weeks 9-11: Sparse adaptive signal processing
  • Sparse regression and variable selection: [HTF] ch. 3.
  • Sparse LMS and sparse RLS algorithms: Y. Chen et al "Regularized Least-Mean-Square Algorithms," 2010. arXiv:1101.5734 (.html) . Y. Chen et al, "Recursive $\ell_{1,\infty}$ lasso," 2011. arXiv:1101.5734v1 (.html) .
  • Adaptation for classification: LDA, QDA and logistic regression: [HTF] ch. 4 and [A] ch. 4,5 and 7.
    Weeks 11-12: Adaptive sensing and learning
  • multi-stage and myopic learning with sensor scheduling: Chs. 2, 3 of [HCCK]

    Important dates

  • Final exam: TAKE HOME will be posted here on M Dec. 12 and will be due M Dec 19 before 5PM (Turn into Shelly Feldkamp, EECS4230)
  • Projects:  
    1-2 page proposal due: Tu Oct. 4 (turn in to Shelly Feldkamp EECS4230)  
    10 page project write-up due: Th Dec. 8 before 5PM (Turn in to Shelly Feldkamp in EECS4230) 
    Oral presentations: Sat., Dec. 10, starting at 8:30AM. Location 1012EECS
    Useful links:
  • UM Academic Calendars
  • Michigan Internet Auction (Used Textbook Auction organized by UM students). 

    Books on reserve at Media Union

  • [A] Categorical data analysis , A. Agresti, Wiley, 2002.
  • [C-BL] Prediction, Learning and Games, N. Cesa-Bianchi and G. Lugosi, Cambridge, 2006.
  • [HCCK] Foundations and applications of sensor management, A. Hero. D. Castenon, D. Cochran, K. Kastella, Springer, 2007.
  • [HTF] The elements of statistical learning: data maining, inference, and prediction, T. Hastie, R. Tibshirani, J. Friedman, Springer-Verlag, 2001.
  • [J] Stochastic processes and filtering theory Jazwinsky, Academic Press, 1970.
  • [SK] Adaptive signal processing algorithms: stability and performance, V. Solo and X. Kong, Prentice-Hall, 1995.
  • [L] Monte Carlo Strategies in Scientific Computing, Jun Liu, 2001.
  • [P] An introduction to signal detection and estimation, V. Poor, Springer, 1998.
  • [R] Pattern recognition and neural networks, Ripley, Cambridge Univ. 1996.
  • [Sa] Fundamentals of adaptive filtering, A. Sayed, Wiley, 2003.
  • [WS] Adaptive Signal Processing, B. Widrow and S. Stearns, Prentice-Hall, 1985.

    General Policies and Guidelines

    All Undergraduate and Graduate students are expected to abide by the College of Engineering Honor Code as stated in the Student Handbook and the Honor Code Pamphlet. This applies to inclass exams, to takehome exams, and to all assigned homeworks. Violation of the Honor Code and the following policies is grounds for me to initiate an action that would be filed with the Dean's office and would come before the College of Engineering's Honor Council. If you have any questions about this policy, please do not hesitate to contact me.

    Exam Policy

    The exam will be take-home and you will be allowed to use any resources you like to solve the problems on the exam. However, all of the work must be your own -- no discussion or collaboration will be allowed with any other individuals.

    Project Policy

  • The project will be graded on the written proposal (.10), the written report (.60), and the quality of your presentation (.30).
  • Your project proposal should be no more than 2 pages long (exluding figures and references), double spaced with no less than 11 point pitch. If you are going to base your project on a paper or two please include them with the project proposal and the final proposal report. You should come and see me in advance of the proposal deadline to discuss your project. Your written final project report should be typed double spaced, with no less than 11 point pitch, and it should be no more than 10 pages in length (excluding figures and bibliography). Each report should include an abstract, an introductory section, a main section or sections, and a conclusion section. Also you need to include a bibliographical citation index at the end of the report. All figures should be clearly labeled and you should include self contained explanatory figure captions which the reader should be able to understand without having to dig through the main body of the report. The proposal and written report will be evaluated based on the difficulty of the topic or work undertaken; the organization and clarity of exposition (spell check before submitting!); and the technical contribution (extension of results of a paper, implementation of a new idea, analysis of an adaptive algorithm, etc).
  • The oral presentation is a major component of the project. Each project will be alotted a 20 minute time slot for presentation including 5 minutes for questions. All presentations should be carefully prepared before the talk. As a rough guideline, a 15 minute talk can accomodate no more than about 5 major ideas or conclusions and 20 slides or so. Slides should not be cluttered with equations and should be readable. Talks should have the following structure: 1) a brief introduction explaining the nature of the problem and the approaches taken; 2) a concise development of the technical content of your final report which includes only the essential concepts and equations needed to understand the problem and your approach; 3) your conclusions. Do not include lengthy derivations in your talk and make sure that all symbols and concepts are defined before they are shown to the audience. Remember the background of the audience: they know the material in EECS659 but they may not be familiar with your problem area. In as far as possible use intuitive explanations and illustrative examples, and explain difficult concepts by using graphical explanations instead of mathematical equations. Practice on a dry run is highly recommended to ensure that the talk can be pulled off in the alloted amount of time. The oral presentation will be evaluated according to: organization and clarity of exposition; the quality of the transparencies; and the speaker's enthusiasm.

    Contact Information:

    Prof. Alfred O. Hero III
    Systems Program
    Dept. of Electrical Engineering and Computer Science
    The University of Michigan
    1301 Beal Avenue
    Ann Arbor, MI 48109-2122
    Tel. (734) 763-0564
    FAX: (734) 763-8041
    email: