A mobile manipulator, an aerial detection of a can, and the updated navigation map.

Long-horizon mobile manipulation

ADM-Planner

LLM-guided planning with attention-enhanced dynamic memory that stays current without carrying the full history.

Jiaping Xiao^†, Pingyuan Ji^†, and Mir Feroskhan

^†Co-first authors · Nanyang Technological University, Singapore

Watch the video View repository

100%DN-14 success

95.8%less context

100%PyBullet success

01 Overview

Planning from the state that matters now.

Long-horizon plans fail when their world context goes stale. Keeping every observation is not the answer either: it produces a growing record of redundant and conflicting state.

ADM-Planner separates persistent workspace knowledge from object-centric dynamic state. Asynchronous observations and action outcomes update one current record per entity, while a bounded retriever exposes only the entries that can affect the next decision. The LLM replans when an accepted update invalidates the remaining plan.

Physical recycling mission showing aerial detection, memory update, and a revised mobile robot trajectory. — System in action An aerial detection updates object-centric memory and triggers a revised ground-robot plan.

02 Method

Update. Retrieve. Replan.

A compact decision state connects asynchronous perception to validated, task-level robot skills.

Update current state

Perception and verified skill outcomes revise pose, confidence, source, time, and task status in one object-centric record.

Retrieve selectively

Confidence, recency, source reliability, and distance rank active entities. At most K records enter the prompt.

Replan only when needed

The current plan continues unless changed fields affect an unexecuted action, avoiding unnecessary LLM queries.

ObservationDynamic memoryTop-K stateLLM planRobot skills

03 Results

Current context, without context growth.

Evaluation progresses from controlled task simulation to live LLM planning, rigid-body execution, and physical missions.

1,500task-simulator episodes

14.4%fewer live-LLM input tokens

60paired PyBullet trials

5/5containers per physical mission

Task-level memory evaluation

Selective retrieval preserves success as the scene gets harder.

In the 14-container noisy dynamic setting, ADM reaches 100% full-task success, compared with 62% for static memory and 97% for unfiltered dynamic memory. It cuts the context-size proxy from 53,049 to 2,212 units.

Plots comparing task success and context size for five memory methods across three scenarios. — Success remains high while bounded retrieval prevents context growth.

Rigid-body execution

Dynamic state remains executable under physics.

Across 20 paired seeds per method, ADM and unfiltered dynamic memory complete every trial. Static memory succeeds in only half. ADM reduces stale-target visits and uses 74.0% less context than unfiltered dynamic memory.

Six plots comparing success, completion, time, path length, stale-target visits, and context size in PyBullet. — PyBullet validation with contacts, gravity, articulated manipulation, and collision constraints.

04 Video

From aerial reveal to ground execution.

The narrated video covers the method and physical missions in indoor, outdoor, narrow-passage, and nighttime settings.

05 Citation

Build on ADM-Planner.

Code and additional project materials will be released in this repository. Please use the citation below when referencing the work.

Browse the repository

@article{xiao2027adm,
  title   = {ADM-Planner: LLM-Guided Long-Horizon Planning
             for Mobile Manipulators with Attention-Enhanced
             Dynamic Memory},
  author  = {Xiao, Jiaping and Ji, Pingyuan and Feroskhan, Mir},
  journal = {https://arxiv.org/abs/2609.29212},
  year    = {2026}
}