Announcements
- • Prerequisites: CS 5800 or CS 7800 with a minimum grade of C-. I will not enforce these pre-requisites this year. However, note that you are taking the course at your own risk. I will assume the knowledge of algorithms as well as intellectual capacity and work ethic of a student who passed such a course. I cannot add you to the course directly. Please show this note to your graduate advisor to enroll you in the course if the system blocks you from doing it due to prerequisites.
- • How to prepare for the course? See here for some guidance.
- • Computing resources at Northeastern: request access to Discovery Cluster
- • First class: Friday, September 9, see 2022-2023 academic calendar
- • Thanksgiving break: November 24-25, no class on November 25
- • Midterm exam: Week 8, Tuesday, in class.
- • Final exam: Friday, December 9, in class
- • Mini project report due: Monday, December 12
- • This class will be held on ground.
———————————————————————
Last updated: December 5, 2022
Weekly Schedule
———————————————————————
Week 14, December 5 ☀️
Final exam: Friday in class!
Topics
- • Kernel machines for graphs
- • Review of topics for final exam
Reading materials
- • Vishwanathan et al. Graph kernels, J Mach Learn Res, 2010. See here.
Handouts and code
- • Kernel machines slides
———————————————————————
Week 13, November 28
Topics
- • Principal component analysis (PCA)
- • Support vector machines (SVMs)
Reading materials
- • Textbook #1 (Bishop): Continuous latent variables (Chapter 12)
- • Sections 12.1, 12.3, 12.4
- • Textbook #1 (Bishop): Sparse kernel machines (Chapter 7)
- • Section 7.1
Handouts and code
Homework assignments
- • Mini project instructions available here
———————————————————————
Week 12, November 21
No class on Friday, Thanksgiving break.
Topics
- • Committee machines
Reading materials
- • Textbook #1 (Bishop): Combining models (Chapter 14)
Handouts and code
———————————————————————
Week 11, November 14
Topics
- • Empirical evaluation
Reading materials
- • Bootstrapping tutorial: Efron & Tibshirani 1986
- • Lecture notes (Radivojac & White): empirical evaluation
Handouts and code
- • Empirical evaluation slides
Homework assignments
- • Assignment #4 available here
———————————————————————
Week 10, November 7
No class on Friday, Veterans Day.
Topics
- • Convolutional and deep neural networks
Reading materials
- • A nice online tutorial on convolutional networks available here
Handouts and code
- • Convolutional and deep neural networks slides
———————————————————————
Week 9, October 31
Topics
- • Generalized linear models
- • Neural networks
Reading materials
- • Lecture notes (Radivojac & White): generalized linear models
- • Textbook #1 (Bishop): Neural networks (Chapter 5)
- • Sections 5.1-5.3 (skip 5.3.4), 5.5 (skip 5.5.4 and on)
- • The RPROP algorithm can be found here
Handouts and code
———————————————————————
Week 8, October 24
Midterm exam, Tuesday in class.
Topics
- • K-means algorithm
Reading materials
- • K-means acceleration papers: Elkan 2003, Arthur 2006, Hamerly 2010, Hamerly 2014
Handouts and code
- • K-means slides
———————————————————————
Week 7, October 17
Topics
- • Data preprocessing
- • Generalized linear models
Reading materials
- • Tan et al. Introduction to Data Mining (Chapter 2: Data)
Handouts and code
- • Data preprocessing slides
———————————————————————
Week 6, October 10
Topics
- • Logistic regression
Reading materials
- • Textbook #1 (Bishop): Linear models for classification (Chapter 4)
- • Sections 4.1 (4.1.1, 4.1.2, 4.1.3, 4.1.7), 4.3 (4.3.2, 4.3.3)
Handouts and code
Homework assignments
- • Assignment #3 available here
———————————————————————
Week 5, October 3
Topics
- • Linear regression for nonlinear problems
- • Regularization
- • Perceptron
Reading materials
- • Lecture notes (Radivojac & White): linear classification
Handouts and code
- • Linear regression for nonlinear problems slides
- • Linear regression code
- • Perceptron slides (last updated: 10/07/2022)
- • Perceptron code
Homework assignments
- • Assignment #2 available here (last updated on 10/12/2022)
———————————————————————
Week 4, September 26
Topics
- • Prediction
- • Naive Bayes models
- • Linear regression
Reading materials
- • Textbook #1 (Bishop): Linear models for regression (Chapter 3)
- • Sections 3.1, 3.2, 3.3 (light reading)
- • Lecture notes (Radivojac & White): prediction
- • Lecture notes (Radivojac & White): linear regression
Handouts and code
———————————————————————
Week 3, September 19
Topics
- • Expectation-maximization algorithm
- • Basic principles of optimization
Reading materials
- • Textbook #1 (Bishop): Mixture Models and EM (Chapter 9)
- • Sections 9.1, 9.2, 9.3, 9.4 (light reading)
- • Lecture notes (Radivojac & White): optimization
Handouts and code
———————————————————————
Week 2, September 12
Topics
- • Basics of parameter estimation
Reading materials
- • Textbook #1 (Bishop): Introduction (Chapter 1)
- • Lecture notes (Radivojac & White): parameter estimation
Handouts and code
Homework assignments
- • Assignment #1 available here
———————————————————————
Week 1, September 5
Topics
- • Class overview and logistics
- • Short review of probability theory
Reading materials
- • Textbook #1 (Bishop): Introduction (Chapter 1)
- • Lecture notes (Radivojac & White): probability (last updated 09/11/2022)
Handouts and code
———————————————————————
———————————————————————