Zijiao Yang Xiangxi Shi Eric Slyman Stefan Lee
WACV 2025
We study adversarial attacks on Vision-and-Language Navigation (VLN) agents through localized modifications in the agent’s operating environment. We develop a white-box environmental attack that optimizes the appearance of a 3D attack object to induce targeted behaviors in pretrained VLN agents. The attack can cause agents to ignore instructions, terminate early, or follow an attacker-defined trajectory, and generalizes to novel instructions and paths not used during optimization.
Paper | Project Page | Supplementary | arXiv | arXiv PDF | DOI
BibTeX
@InProceedings{Yang_2025_WACV,
author = {Yang, Zijiao and Shi, Xiangxi and Slyman, Eric and Lee, Stefan},
title = {Hijacking Vision-and-Language Navigation Agents with Adversarial Environmental Attacks},
booktitle = {Proceedings of the Winter Conference on Applications of Computer Vision (WACV)},
month = {February},
year = {2025},
pages = {6094-6103}
}