Chanho Park
  • tile = one visual token of my photo (28 × 28 px)

COCO val2017 #2153 · 24×18 tokens

I. The question

My research asks whether multimodal models can actually use the information available to them—and how we can verify that they do. Spatial information may be present in an image yet encoded in a form that a model cannot effectively exploit.

  • color · lift = attention from the answer (top-20 retrieval heads)
  • line = the 9 most-attended tokens
  • box = the people/objects the question is about (hand-labeled)
  • outline = tokens represented like the one you point at (layer 16)

Fig. 1. Same photo, two questions: one in the camera’s frame, one from another point of view. Answers and top-20 retrieval-head attention from Qwen2.5-VL-7B (MLX 4-bit, run locally); boxes and ground truth are hand-labeled.

II. Use

In Perspective-Aware Reasoning and Token Warping, I address this by transforming the representations that models reason over rather than the pixels themselves.

  • patch = an original token, pixels unchanged
  • how far it sits out = estimated nearness
  • gap = never seen from the original camera

Fig. 2. Token warping, live: each token is placed by its estimated depth (Depth Anything V2) and turned to face the camera — nothing is re-rendered. Drag for another nearby viewpoint (±25°).

III. Verify

But improved accuracy alone does not show that a model relied on the right evidence. In Retrieval Heads Meet Vision, I identify a small set of attention heads—Visual Retrieval Heads—that carry visual evidence, and causally test their role in grounding by masking them.

  • color · lift = attention from the answer (top-20 retrieval heads)
  • line = the 9 most-attended tokens
  • box = the object asked about (hand-labeled)

Fig. 3. Attention from the question to image tokens. Loading attention extracted from Qwen2.5-VL-7B…

IV. Over time

I am now extending this line of inquiry over time: asking what long-horizon models should retain in memory, when stale internal states should be revised, and how these mechanisms can support multimodal agents that perceive, reason, and interact in real-world environments.

  • front plate = the live video; behind = frames kept in memory, one every 1.5 s
  • color · lift = attention from the question, asked at the last frame
  • line = attention reaching back in time
  • faded = little attention (not needed for the answer)
6.0 s

Q · In Frame 1, where is the beige ball? asked at t = 6.0 s

Fig. 4. A bumblebee rolling a ball (CC BY 4.0) plays as tokens; every 1.5 s a frame drops into memory — the five frames the model is given. Loading attention extracted from Qwen2.5-VL-7B…

§ News

§ Publications all papers →

NeurIPS ’26

Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Information

Chanho Park, Daehyeon Choi, Jihyun Lee, Minhyuk Sung

CVPR ’26

Token Warping Helps MLLMs Look from Nearby Viewpoints

Phillip Y. Lee*, Chanho Park*, Mingue Park, Seungwoo Yoo, Juil Koo, Minhyuk Sung (* equal contribution)

WACV ’26

BoxSplitGen: A Generative Model for 3D Part Bounding Boxes in Varying Granularity

Juil Koo*, Wei-Tung Lin*, Chanho Park, Chanhyeok Park, Minhyuk Sung (* equal contribution)

ICCV ’25

Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation

Phillip Y. Lee, Jihyeon Je, Chanho Park, Mikaela Angelina Uy, Leonidas Guibas, Minhyuk Sung

CVPR ’24

Posterior Distillation Sampling

Juil Koo, Chanho Park, Minhyuk Sung

Elsewhere: email, Scholar, GitHub. My Erdős number is 3.11Chanho Park → Leonidas Guibas → Frances Yao → Paul Erdős.