Asynchronous robot policy deployment

On Matching Execution Length to Inference Latency in Asynchronous Robot Policy Deployment

Authors TBD

Video BibTeX

Simulation in RoboCasa with X-WAM. Replayed from logged actions, aligned to the recorded wall clock. Failures remain visible. The labelled dissolve at the end returns to the opening frame.

TL;DR

    The deployment surprise

    Faster model, worse robot

    The same execution length can give a very different result when inference gets faster.

    Same button task and initial state. The fast model fails; waiting or matching the execution length restores success. This is an illustrative episode. Rates below come from the full button-task panel.

    Try the execution window

    Why: latency moves the executed window

    Which part of an action plan does the robot execute? Adjust inference delay and execution length to move the executed window. A window past the end of the plan means the robot must wait.

    A new plan is based on an earlier observation. Latency decides where execution starts; execution length decides where it ends. This diagram shows the window mechanism.

    A training-free planner

    Method

    Measure a little. Choose how long to execute.

    Measure the synchronous curve S(n)

    cheap: no asynchronous runs

    One asynchronous calibration run

    cost of acting on stale observations

    Score every (d, E) and choose

    Q(d, E) = S(d + E)
    − κ · d · (d + E)
    execution length E(d)latency budget d*

    Try the planner

    Predicted success for each execution length
    Predicted Q(d, E) (%)

    Execution length E (steps)

    point scoreplanner's choicemeasured in the blind test

    Choices made before the test

    Does it pick well?

    Blind-test success

    Execution length E (steps)

    Inference delay d · steps / milliseconds

    preselected Ebest measured E

    preselected E vs. best measured E

    latency budget

    predicted = measured

    Across five policy families

    Success after acceleration
    keep source Eplanner's E

    From execution length to capacity

    Deployment: one GPU, many robots; slower devices, fewer

    Longer execution means fewer calls per robot. The same GPU can serve more robots.

    no waitone robotlargest no-wait run
    Show numbers
    Current measured capacity. Each comparison uses the same service latency and execution length.
    Service / runRobots that never waitRobots with success keptΔ vs one robot
    at the largest no-wait run

    Spend less on inference

    Compute vs tuned temporal ensembling

    On the X-WAM test block, matching execution length keeps success while reducing inference cost.

    success difference vs tuned temporal ensembling
    less inference GPU time per episode

    Limitations

      BibTeX

      Author details will be added with the paper.