DS 5220: Supervised Machine Learning and Learning Theory

Announcements

• How to prepare for the course? See here for some guidance.
• First class: Friday, September 8, see 2023-2024 academic calendar 
• Thanksgiving break: November 23-24, no class on November 24
• Midterm exam: Week 8, Tuesday, in class.
• Final exam: Friday, December 8, in class
• Mini project report due: Monday, December 11
• This class will be held on ground.

———————————————————————

Last updated: December 5, 2023

Weekly Schedule

———————————————————————

Week 14, December 4 ☀️

Final exam: Friday in class!

Topics

• Kernel machines for graphs
• Review of topics for final exam

Reading materials

• Vishwanathan et al. Graph kernels. J Mach Learn Res, 2010. See here.

Handouts and code

• Kernel machines slides 

———————————————————————

Week 13, November 27 

Topics

• Committee machines
• Support vector machines (SVMs)

Reading materials

• Textbook #1 (Bishop): Combining models (Chapter 14)
• Textbook #1 (Bishop): Sparse kernel machines (Chapter 7) 
• Section 7.1  

Handouts and code

• Committee machines slides (last updated: 12/05/2023) ☀️
• Committee machines code 
• Support vector machines slides 

Homework Assignments

• Mini project instructions available here 

———————————————————————

Week 12, November 20 

No class on Friday, Thanksgiving break.

Topics

• Classification and regression trees

Reading materials

• Tan et al., Introduction to Data Mining (Chapter 4)
• Mitchell: Machine learning (Chapter 3)

Handouts and code

• Classification and regression trees slides 

———————————————————————

Week 11, November 13 

Topics

• Empirical evaluation
• Classification and regression trees

Reading materials

• Bootstrapping tutorial: Efron & Tibshirani 1986 

Handouts and code

• Classification and regression trees slides 

Homework Assignments

• Assignment #4 available here 

———————————————————————

Week 10, November 6

Topics

• Convolutional and deep neural networks
• Empirical evaluation

Reading materials

• A nice online tutorial on CNNs available here 

Handouts and code

• Deep neural networks slides 
• Empirical evaluation slides (last updated on 11/17/2023) ☀️

———————————————————————

Week 9, October 30 

Topics

• Neural networks

Reading materials

• Textbook #1 (Bishop): Neural networks (Chapter 5)
• Sections 5.1-5.3 (skip 5.3.4), 5.5 (skip 5.5.4 and on)
• The RPROP algorithm can be found here 

Handouts and code

• Neural network slides (last updated on 11/05/2023) ☀️

———————————————————————

Week 8, October 23

Midterm exam, Tuesday in class

Topics

• Data preprocessing

Reading materials

• Tan et al. Introduction to Data Mining (Chapter 2: Data)

———————————————————————

Week 7, October 16

Topics

• Generalized linear models
• Data preprocessing

Reading materials

• Lecture notes (Radivojac & White): generalized linear models 

Handouts and code

• Generalized linear models slides 
• Data preprocessing slides 

Homework assignment

• Assignment #3 to be available here.

———————————————————————

Week 6, October 9

Topics

• Perceptron 
• Logistic regression

Reading materials

• Lecture notes (Radivojac & White): linear classification 
• Textbook #1 (Bishop): Linear models for classification (Chapter 4) 
• Sections 4.1 (4.1.1, 4.1.2, 4.1.3, 4.1.7), 4.3 (4.3.2, 4.3.3)

Handouts and code

• Perceptron slides 
• Pocket algorithm code 
• Logistic regression slides 
• Logistic regression code
• Mini projct slides

Homework assignment

• Assignment #2 available here 

———————————————————————

Week 5, October 2 

Topics

• Linear regression
• Linear regression for nonlinear problems
• Regularization

Reading materials

• Textbook #1 (Bishop): Linear models for regression (Chapter 3)
• Sections 3.1, 3.2, 3.3 (light reading)
• Lecture notes (Radivojac & White): linear regression 

Handouts and code

• Linear regression slides
• Linear regression for nonlinear problems slides 
• Linear regression code 

———————————————————————

Week 4, September 25 

Topics

• Prediction
• Naive Bayes models 
• Principles of optimization 

Reading materials

• Lecture notes (Radivojac & White): prediction 
• Lecture notes (Radivojac & White): optimization 

Handouts and code

• Prediction slides
• Optimization slides 

———————————————————————

Week 3, September 18 

Topics

• Basics of parameter estimation

Reading materials

• Lecture notes (Radivojac & White): parameter estimation 

Handouts and code

• Parameter estimation slides 

Homework assignment

• Assignment #1 available here 

———————————————————————

Week 2, September 11 

Topics

• Probability theory
• Random variables

Handouts and code

• Random variables slides 

———————————————————————

Week 1, September 4

Topics

• Class overview and logistics
• Probability theory

Reading materials

• Textbook #1 (Bishop): Introduction (Chapter 1)
• Lecture notes (Radivojac & White): probability 

Handouts and code

• Class overview slides 
• Probability theory slides 

———————————————————————

———————————————————————