READ: What Makes a Pretrained VLA Teachable by Demonstration?
Show a pretrained VLA five demonstrations of a new task and it performs the task with no gradient steps, matching per-task fine-tuning on held-out tasks.
PhD student making multimodal agents more capable and adaptable
Advised by Kenneth Marino · m.pham-dinh (at) utah.edu
“The authors claim this is the first work to address this problem. This is false (Schmidhuber et al, 1999).” — Reviewer 2 idea from my college buddy Jacob
I am a PhD student at the University of Utah, advised by Professor Kenneth Marino. These days I am mostly interested in enhancing the general capabilities and adaptation of multimodal agents: agents that see, read, and act, and that can pick up new skills and adjust to unfamiliar environments without being retrained from scratch.
Before Utah, I completed my undergraduate degree at Colby College, where I conducted AI and assistive robotics research at the Davis Institute for Artificial Intelligence and the INSITE Lab. I’m always open to collaborations on agents, both on screens and in the physical world.
Show a pretrained VLA five demonstrations of a new task and it performs the task with no gradient steps, matching per-task fine-tuning on held-out tasks.
Compact browser agents succeed on unseen websites by practicing reusable interaction skills, then composing them with a planner to solve full tasks.
Web agents adapt to unfamiliar websites and redesigns by scouting the interface and remixing existing adapters, with no retraining on the target.
A science agent that plans by querying a structured, temporal knowledge graph in a multi-turn inner monologue, improving on 8 of 9 ScienceWorld subjects.