Recent advances in generative models using deep neural networks have enabled scalable modeling of complex, high-dimensional data including images, videos, text and audio and have been transformative for science and industry. In this course, we will study the foundations and learning algorithms for deep generative models, including diffusion models, variational autoencoders, generative adversarial networks, autoregressive models, normalizing flow models, energy-based models, and score-based models. Part of the course will be devoted to teaching different approaches to generative modeling and the other part will be devoted to reading and discussing most recent papers about algorithms and applications of deep generative models in image and video understanding, graph mining and natural language modeling.

Basic knowledge about machine learning and probability as well as programming in Python.

1. Deep Generative Models

• Introduction and Background

• Autoregressive Models

• VAEs

• Normalizing Flows

• GANs

• Energy Based Models

• Score Based Models

• Diffusion Models for Continuous, Discrete and Graph Data

• Evaluating Generative Models

2. Paper Presentations on Deep Generative Models: This part of the course involves presentations of recent papers by students on algorithms and applications of deep generative models for image, video, audio and text generation and understanding.

• Image Generation, Editing, Inpainting and Super-resolution

• Video Generation, Editing and Learning

• Large Language Models (LLM) and Large Vision-Language Models (LVLMs)

• Few-shot, Zero-shot and Weakly-Supervised Learning using Deep Generative Models

• Deep Generative Models for Out-of-Distribution and Data Anomaly Detection

• Other Applications of Generative Models

Homeworks are due at the beginning of the class on the specified dates. No late homeworks or projects will be accepted.

• Paper Presentations (30%)

• Research Paper Synopses (25%)

• Project (35%)

• Class Participation (10%)

Project can be done individually or in teams of two people. The project typically involve leveraging existing generative models to solve an interesting application problem or extending a deep generative model learning/algorithm to address a new/existing application.

All students in the course are subject to the Northeastern University's Academic Integrity Policy. Any submitted report/homework/project by a student in this course for academic credit should be the student's own work. Collaborations are only allowed if explicitly permitted. Per CCIS policy, violations of the rules, including cheating, fabrication and plagiarism, will be reported to the Office of Student Conduct and Conflict Resolution (OSCCR). This may result in deferred suspension, suspension, or expulsion from the university.