CS-GY 9223: Selected Topics in CS - Visualization for Machine Learning

Instructor: Claudio Silva (csilva@nyu.edu)
Teaching Assistant: Parikshit Solunke (pss442@nyu.edu)
Meeting Time: Mondays 5:00 PM - 7:30 PM
Classroom: Jacobs Hall, 6 Metrotech Room 473, Brooklyn Campus
Make-up Class: Tuesday, October 14 (for Fall Break)

Announcements

⚠️ Please Note: Course schedule, assignments, and materials are tentative and subject to updates during the semester. Students will be notified of any changes via Discord and course announcements.

Course materials will be posted as the semester progresses

Upcoming Classes

Week 1 (Sept 2) - Labor Day

• No Class - Labor Day Holiday

Week 2 (Sept 8)

• Topics: Course Introduction, Syllabus, Introduction to Visualization, Hands-on Vega-Lite
• Materials:
  • Course Introduction & Syllabus
  • Information Visualization Introduction
  • Vega-Lite Lab
• Assignment: TBD

Week 3 (Sept 15)

• Topics: Perception for Design, Color Theory for Visualization
• Materials:
  • Perception for Design
  • Color Theory for Visualization
  • Lab: Perception and Color in Practice

Week 4 (Sept 22)

• Topics: Model Assessment and Evaluation
• Materials:
  • Model Assessment and Evaluation
  • Lab: Materials to be posted
• Recommended Readings:
  • Squares: Supporting Interactive Performance Analysis for Multiclass Classifiers (Ren et al., 2016)
  • Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels (Görtler et al., 2022)
• Content:
  • Confusion Matrices and ROC Curves
  • Visual Analytics Systems for Model Performance
  • Calibration Theory and Practice

Week 5 (Sept 29)

• Topics: Visualization for White-box Machine Learning Models
• Materials:
  • White-box Model Interpretation
  • Lab: Interpretable ML Methods
• Recommended Readings:
  • A Partition-Based Framework for Building and Validating Regression Models (Mühlbacher & Piringer, 2013) - Best Paper Award, IEEE VAST 2013
  • Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models (Hohman et al., 2019)
  • BaobabView: Interactive Construction and Analysis of Decision Trees (van den Elzen & van Wijk, 2011)
• Content:
  • Linear Regression and Visual Analytics Systems
  • Generalized Additive Models (GAMs) and Explainable Boosting Machines
  • Tree-based Models and Visualization Techniques
  • Decision Rules and Global Surrogate Models

Week 6 (Oct 6)

• Topics: Black-box Model Interpretation, Project Discussion
• Materials:
  • Black-box Model Interpretation
  • Lab: Black-box Explainability Methods
  • Project Discussion
• Recommended Readings:
  • “Why Should I Trust You?” Explaining the Predictions of Any Classifier (Ribeiro et al., 2016, KDD)
  • SHAP Book: A Unified Approach to Interpreting Model Predictions (Molnar, 2024)
• Content:
  • Partial Dependence Plots (PDP)
  • Local Interpretable Model-agnostic Explanations (LIME)
  • SHAP (SHapley Additive exPlanations)
  • Project Ideas and Guidelines

Week 7 (Oct 14 - Tuesday Make-up Class)

• Topics: Clustering and Dimensionality Reduction, Default Project Details
• Materials:
  • Clustering and Dimensionality Reduction
  • Default Project: Visual Analytics for AI-Generated Urban Infrastructure Maps
  • Lab: Dimensionality Reduction
• Recommended Readings:
  • Wolfram Clustering Tutorial - Required
  • A Tutorial on Principal Component Analysis (Shlens, 2014)
  • Scikit-learn PCA User Guide
  • Mapping the walk: A scalable computer vision approach for generating sidewalk network datasets from aerial imagery (Hosseini et al., 2023)
• Content:
  • Introduction to Unsupervised Learning
  • K-means Clustering and DBSCAN
  • The Manifold Hypothesis and Intrinsic Dimensionality
  • Principal Component Analysis (PCA)
  • Eigenvectors, Eigenvalues, and Covariance Matrices
  • Singular Value Decomposition (SVD)
  • Local Linear Embedding (LLE)
  • Default Project Overview and Ideas

Week 8 (Oct 20)

• Topics: Dimensionality Reduction (continued)
• Materials:
  • Dimensionality Reduction (continued)
• Recommended Readings:
  • How to Use t-SNE Effectively (Wattenberg, Viégas, Johnson, 2016) - Required
  • Visualizing Data using t-SNE (van der Maaten & Hinton, 2008)
  • UMAP: Uniform Manifold Approximation and Projection (McInnes, Healy, Melville, 2018)
  • Understanding UMAP (Coenen & Pearce)
• Content:
  • t-SNE: Theory and Pitfalls
  • UMAP: Uniform Manifold Approximation and Projection
  • Topomap: Topologically-Constrained Dimensionality Reduction
  • Interactive Dimensionality Reduction Techniques

Assignments

Weekly Assignments (50% of grade)

• Assignments will be posted as the semester progresses
• Programming exercises will be given throughout the first half of the semester

Research Project (45% of grade)

• Team formation - Week 3
• Project Proposal (4-page writeup) - Week 5 - 10%
• Project Updates (1-page writeup) - Week 8 - 10%
• Final Project (8-page writeup + presentation) - Weeks 14-15 - 25%

Class Participation (5% of grade)

Quick Links

• Discord: Join Course Discord
• Brightspace: [Course materials and submissions]
• Office Hours: TBD

Course Description

This course explores the intersection of visualization and machine learning, focusing on how visualization techniques can help understand, debug, and improve machine learning models. Students will learn to create visual analytics systems for model assessment, feature analysis, and result interpretation. Topics include visualization for model performance, feature importance, clustering, dimensionality reduction, deep learning architectures, and interpretable AI.

Prerequisites

• Solid programming skills (Python and JavaScript)
• Basic knowledge of machine learning concepts
• Familiarity with web technologies (HTML, CSS) helpful but not required