Visualization for NLP and LLMs

CS-GY 9223 - Fall 2025

Claudio Silva

NYU Tandon School of Engineering

2025-11-03

NLP and Large Language Models

Today’s Agenda

1. Natural Language Processing (NLP) basics
   • Tasks and challenges
   • Analysis and representation
   • General resources
   • Sparse and dense text representation
   • Neural Network recap.
2. Visualization for NLP
   • General Text Visualization
   • Model agnostic explanation
   • Recurrent Neural Network (RNN) Visualization
   • Transformers (LLM) Visualization

Natural Language Processing has undergone a revolution with the advent of large language models. Today’s lecture bridges traditional NLP visualization techniques with modern LLM analysis methods. We’ll start with fundamental text processing concepts, then explore how visualization helps us understand both classical models like RNNs and modern transformer-based architectures. The transition from traditional NLP to LLMs represents not just a scale change but a fundamental shift in how we approach language understanding. Visualization plays a crucial role in making these black-box models more interpretable.

NLP basics: Tasks and challenges

The NLP pipeline begins with raw text and transforms it through multiple stages into a form suitable for machine learning. Tokenization splits text into units (words, subwords, or characters). Normalization handles case, punctuation, and special characters. Vectorization converts tokens to numbers - from simple one-hot encoding to sophisticated contextual embeddings. Each stage involves design decisions that significantly impact downstream performance. For example, aggressive normalization might remove important signals (like capitalization indicating proper nouns), while preserving too much variation increases vocabulary size and sparsity.

NLP basics: Tasks and challenges

Word embeddings revolutionized NLP by providing dense, continuous representations where similar words have similar vectors. The key insight is the distributional hypothesis: words appearing in similar contexts have similar meanings. Word2Vec uses shallow neural networks to predict context words (CBOW) or target words (Skip-gram). GloVe combines global matrix factorization with local context windows. FastText extends Word2Vec by using character n-grams, helping with out-of-vocabulary words. These embeddings capture surprising regularities - the famous “king - man + woman = queen” demonstrates that vector arithmetic can encode analogical reasoning.

NLP basics: Tasks and challenges

Table 1: NLP can be divided into tasks related to acquisition, analysis, representation, etc.

acquisition	structure	meaning	representation
sound wave	phonetics/phonology	semantics	bag-of-words
text corpus	morphology	pragmatics	n-gram
	syntax	discourse	word2vec
…	…	…	…

sound wave (signal captured by micropones, mixing consoles, audio interfaces) text corpus ()

phonetics/phonology (knowledge about linguistic sounds) morphology (knowledge of the meaningful components of words)
syntax (knowledge of the structural relationships between word)

semantics (knowledge of meaning) pragmatics (knowledge of the relationship of meaning to the goals and intentions of the speaker) discourse (knowledge about linguistic units larger than a single utterance)

NLP basics: Tasks and challenges

NLP tasks span from low-level syntactic analysis to high-level semantic understanding and generation. Token-level tasks like POS tagging and NER identify properties of individual words. Sequence-level tasks like machine translation and summarization transform entire texts. Classification tasks like sentiment analysis and topic modeling categorize documents. Generation tasks create new text. This taxonomy matters for visualization because different tasks benefit from different visual representations. Token-level tasks use attention heatmaps and dependency graphs. Document-level tasks use embedding projections and clustering visualizations. Understanding task requirements helps select appropriate models and interpretability methods.

Word & Morphosyntactic Level

• Named entity recognition
• Parts-of-speech tagging
• Dependency parsing
• Grammatical error correction
• Word sense disambiguation
• Coreference resolution

Document & Semantic Level

• Text summarization
• Question answering
• Machine translation
• Sentiment analysis
• Topic modeling
• Dialogue systems

NLP basics: Analysis and representation

1. Lexical and morphological analysis
   • Finite-state morphological parsers
2. Syntactic recognition and representation
   • Shallow parser or chunker
   • Context-free Grammar
3. Morphosyntactic analysis
   • Part-of-Speech (POS) tagging
4. Representing Meaning
   • First-order logic
   • Semantic Network
   • Conceptual Dependency Diagram
   • Frame-based approach

Jurafsky, 2025

NLP basics: Analysis and representation

1. Lexical and morphological analysis
   • Finite-state morphological parsers
2. Syntactic recognition and representation
   • Shallow parser or chunker
   • Context-free Grammar
3. Morphosyntactic analysis
   • Part-of-Speech (POS) tagging
4. Representing Meaning
   • First-order logic
   • Semantic Network
   • Conceptual Dependency Diagram
   • Frame-based approach

Jurafsky, 2025

NLP basics: Analysis and representation

1. Lexical and morphological analysis
   • Finite-state morphological parsers
2. Syntactic recognition and representation
   • Shallow parser or chunker
   • Context-free Grammar
3. Morphosyntactic analysis
   • Part-of-Speech (POS) tagging
4. Representing Meaning
   • First-order logic
   • Semantic Network
   • Conceptual Dependency Diagram
   • Frame-based approach

Jurafsky, 2025

NLP basics: Analysis and representation

1. Lexical and morphological analysis
   • Finite-state morphological parsers
2. Syntactic recognition and representation
   • Shallow parser or chunker
   • Context-free Grammar
3. Morphosyntactic analysis
   • Part-of-Speech (POS) tagging
4. Representing Meaning
   • First-order logic
   • Semantic Network
   • Conceptual Dependency Diagram
   • Frame-based approach

Jurafsky, 2025

NLP basics: General resources

1. Lexicon: list of stems and affixes (prefix or suffix), together with basic information about them.
2. Thesaurus: list of words and their synonyms from a specific domain
3. Treebank: list of words labeled with syntatic (POS-tagging) trees
4. Prop(osition) bank: sentences annotated with semantic roles related to verbs
5. FrameNet: sentences annotated with semantic roles related to frames of words.
6. Ontology: hierachy of concepts related to a domain

NLP basics: Sparse and dense text representation

1. Sparse embeddings ¹
   1. One-hot encoding
   2. Bag-of-Words (BoW)
   3. Term Frequency-Inverse Document Frequency (TF-IDF)
2. Dense embeddings

Galli, 2023

1. We can represent words, n-grams, sentences, etc.

NLP basics: Sparse and dense text representation

1. Sparse embeddings ¹
   1. One-hot encoding
   2. Bag-of-Words (BoW)
   3. Term Frequency-Inverse Document Frequency (TF-IDF)
2. Dense embeddings

Sharma, 2024

1. We can represent words, n-grams, sentences, etc.

NLP basics: Sparse and dense text representation

1. Sparse embeddings ¹
   1. One-hot encoding
   2. Bag-of-Words (BoW)
   3. Term Frequency-Inverse Document Frequency (TF-IDF)
2. Dense embeddings

Mai, 2019

1. We can represent words, n-grams, sentences, etc.

NLP basics: Sparse and dense text representation

1. Sparse embeddings ¹
   1. One-hot encoding
   2. Bag-of-Words (BoW)
   3. Term Frequency-Inverse Document Frequency (TF-IDF)
2. Dense embeddings

Alammar, 2019

1. We can represent words, n-grams, sentences, etc.

NLP basics: Neural Network recap.

MLP - Jaokar, 2024

NLP basics: Neural Network recap.

MLP - Jaokar, 2024

RNN - Colah, 2015

NLP basics: Neural Network recap.

Transformer - Vaswani, 2017

NLP basics: Neural Network recap.

Transformer - Vaswani, 2017

NLP basics: Neural Network recap.

Transformer - Park, 2019

Today’s Agenda

1. Natural Language Process (NLP) basics
   • Tasks and challenges
   • Analysis and representation
   • General resources
   • Sparse and dense text representation
   • Neural Network recap.
2. Visualization for NLP
   • General Text Visualization
   • Model agnostic explanation
   • Recurrent Neural Network (RNN) Visualization
   • Transformers (LLM) Visualization

Visualization for NLP: General Text Visualization

1. Exploring and Visualizing Variation in Language Resources
2. Termite: Visualization Techniques for Assessing Textual Topic Models
3. Scattertext: a Browser-Based Tool for Visualizing how Corpora Differ

Visualization for NLP: General Text Visualization

1. Exploring and Visualizing Variation in Language Resources
2. Termite: Visualization Techniques for Assessing Textual Topic Models
3. Scattertext: a Browser-Based Tool for Visualizing how Corpora Differ

Visualize linguistic variation between document categories in a language-independent way. The main visualization is a scatterplot of terms, using their frequencies as position and a probabilistic score that defines how strong the connection is between a word and a category. The authors tested it with a speech dataset from the 2012 Democratic and Republican conventions. In this case, Scattertext can highlight linguistic style and key themes of each dataset.

Visualization for NLP: Model agnostic explanation

1. Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models
2. ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models
3. iSEA: An Interactive Pipeline for Semantic Error Analysis of NLP Models

Visualization for NLP: Model agnostic explanation

1. Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models
2. ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models
3. iSEA: An Interactive Pipeline for Semantic Error Analysis of NLP Models

Automatically discovers semantically-grounded subpopulations with high error rates in the context of a human-in-the-loop interactive system. The user can learn more, validate error sources, and test hypotheses about errors.

1. Model Performance Overview:
2. Rule Discovery and 3) Rule Editor A rule is created by pattern mining that finds features combinations (rules) that frequencly occur in ‘model error cases’. That mining is applied over semantic and syntatic features, such as dependency labels, POS-tags, semantic roles, and lexical markers.
3. Overall statistics
4. Model prediction with SHAP tokens score, raw texts, ground-truth/predicted label
   
5. Concept construction enables a user to group Rules together.

Visualization for NLP: RNN Visualization

1. Understanding Hidden Memories of Recurrent Neural Networks
2. RNNbow: Visualizing Learning Via Backpropagation Gradients in RNNs
3. LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks

Visualization for NLP: RNN Visualization

1. Understanding Hidden Memories of Recurrent Neural Networks
2. RNNbow: Visualizing Learning Via Backpropagation Gradients in RNNs
3. LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks

1. Select view

• Select a text section and define an activation threshold to highlight hidden states (blue lines)
• Extra information presented in the meta-track, such as word POS-tagging and top-k model predictions

2. Match View

• Shows similar hidden state patterns in the dataset
• Shows a heatmap with matched sentences (rows), hidden states (cols), and the activation value (color)
• Shows a second heatmap encoding POS-tag of sentence words

Tests with datasets of 1) news from the wall street journal 2) parenthesis language 3) dna sequences, and 4) musical chord progressions

Visualization for NLP: LLM Visualization

1. BertViz: A tool for visualizing multihead self-attention in the BERT model
2. Sanvis: Visual analytics for understanding self-attention networks
3. Attention flows: Analyzing and comparing attention mechanisms in language models
4. Dodrio: Exploring transformer models with interactive visualization
5. TopoBERT: Exploring the topology of fine-tuned word representations
6. Attentionviz: A global view of transformer attention
7. On the Biology of a Large Language Model
8. POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models

Visualization for NLP: LLM Visualization

2. Sanvis: Visual analytics for understanding self-attention networks

1. Attention piling (heatmap): key-words (x-axis), query-words (y-axis), and cell values are the attention score over the layer heads. Besides, instead of show information for all the layer heads, it aggregates them to show only one heatmap (piles).
2. Small multiple: Heads showed with some sort criterion (Head number, Entropy, Position)
3. Sanky diagram: how strong is the attention between two tokens
4. Head leans: extended information about each layer head.

comprehend the alignment between encoder and decoder and the relationship between tokens

Visualization for NLP: LLM Visualization

4. Dodrio: Exploring transformer models with interactive visualization

1. Dependency view: shows sintatic dependences between tokens like a dependency tree/graph or a saliency view
2. Semantic attention graph: a graph that presents informations of a specific layer and head. Nodes are tokens and egdges are attention between them. Different views: radial, grid, and force
3. Attention head overview: grid of layers (rows) per heads (columns). Color palette maps a semantic score (read) and syntatic score (blue). Besides, the circle size means an importance score.

attention weights syntatic dependences semantic information

Visualization for NLP: LLM Visualization

6. Attentionviz: A global view of transformer attention

1. Creates an aligned projected space of Query and Key embeddings
2. In the main view, it shows a grid of projections per layer (rows) nad heads (columns)
3. In a detailed view of text models, it shows a BertViz-like view that connects query-key tokens with their attention scores.

Visualization for NLP: LLM Visualization

7. On the Biology of a Large Language Model

Does Claude plan its rhymes?

Mental math

Visualization for NLP: LLM Visualization

7. On the Biology of a Large Language Model

Are Claude’s explanations always faithful?

Multi-step reasoning

Visualization for NLP: LLM Visualization

7. On the Biology of a Large Language Model

Hallucinations

applying attribution graphs to study a particular language model

Visualization for NLP: LLM Visualization

8. POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models

1. Prompt panel: craft and edit prompts
2. Reasoning panel:
3. Evaluation panel: global and local evaluation with comparison of prompt editing history

a visual analytics system to facilitate efficient prompt engineering for steering the multimodal reasoning performance of LLM

Summary: NLP Foundations

Key Takeaways:

1. Even if human language is convoluted, we can still build advanced NLP systems that achieve good results.
2. These systems are difficult to explain and interpret.
3. Dense representations have much more information than sparse ones, such as the meaning, position, and relations between tokens.
4. In a transformer, Q and K spaces are the mechanisms for selection (interpretable via patterns of relevance), while V space is the content being selected (abstract, combined, and thus harder to ground linguistically on its own).