Class Meets
When: Tuesdays and Fridays, 3:25pm-5:05pm
Where: Shillman Hall 305. See map.
Course Description
Introduces supervised machine learning, which is the study and design of algorithms that enable computers/machines to learn from experience or data, given examples of data with a known outcome of interest. Offers a broad view of models and algorithms for supervised decision making. Discusses the methodological foundations behind the models and the algorithms, as well as issues of practical implementation and use, and techniques for assessing the performance. Includes a term project involving programming and/or work with real-world data sets. Requires proficiency in a programming language such as Python, R, or MATLAB.
Class Materials
Textbooks:
The Elements of Statistical Learning — by T. Hastie, et al., Springer, 2009.
Machine Learning: A Probabilistic Perspective — by K. P. Murphy, The MIT Press, 2012
Recommended books:
Pattern Recognition and Machine Learning — by C. M. Bishop, Springer, 2006.
Machine Learning — by T. M. Mitchell, McGraw-Hill, 1997
Supplementary materials: to be provided in class.
Topics
- Mathematical foundations of machine learning (random variables and probabilities, probability distributions, high-dimensional spaces, principles of optimization)
- Basics of parameter estimation (point estimates, Bayesian formulation)
- Classification algorithms: linear and non-linear algorithms (perceptron, logistic regression, naive Bayes, decision trees, neural networks, support vector machines)
- Regression algorithms (least squares linear regression, neural networks, deep networks)
- Kernel methods (taught within classification and regression; dual representations, RBF networks)
- Ensemble methods (bagging, boosting, gradient boosting, random forests)
- Bayesian networks (learning & inference, structure learning)
- Practical aspects in supervised learning (data preprocessing, overfitting, accuracy estimation, parameter and model selection)
- Special topics, if time permits (introduction to PAC learning, sample selection bias, learning from graph and time series data)
Grading
- Midterm exam: 20%
- Final exam: 20%
- Homework assignments: 30%
- Mini project: 25%
- Class participation: 5%
Late Policy and Academic Honesty
All assignments and exams are individual, except when collaboration is explicitly allowed. All the sources used for problem solution must be acknowledged, e.g. web sites, books, research papers, personal communication with people, etc. Academic honesty is taken seriously; for detailed information see Office of Student Conduct and Conflict Resolution.
Last updated: September 7, 2023