My work sits in video understanding, and the question I
keep coming back to is what a representation of a long video should
actually carry. A representation is only ever good for something:
the cut that answers “what step is happening now?” is not the
one that answers “why did this go wrong?” That tension,
between a fixed representation and a shifting question, is what most of
my recent work circles.
One line of it asks what learned representations already encode.
Latent-INR
showed that implicit neural representations of video can carry
discriminative semantics rather than only reconstructing pixels;
LEIA found latents for
3D articulation that hold up across viewpoints;
CSD pulled style out as
a measurable, separable property of generated images. I have also worked on learning from noisy human and
VLM
preferences with TREND.
The other line is temporal.
VidParse
(ECCV 2026) treats online action understanding in egocentric video as
graph-constrained inference: it finds semantic boundaries from
manipulation-anchored features off frozen foundation models, then decodes
with a beam search restricted to valid step transitions. It takes no
gradient steps and is up to 10× more accurate at multi-step parsing
than strong trained online baselines.
Where this is going. A parse fixed in advance is a
strong assumption, and procedural task graphs are a rigid way to hold
knowledge — they have to be induced or authored, and they do not transfer.
I am starting to look at representations whose structure is not settled
ahead of the question being asked. Vision–language models are the
obvious way in, since they already carry a great deal of this knowledge
implicitly. This is early; no results yet.
video understanding
video representation learning
egocentric & procedural video
video–language models