Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
My question
- auto-regressive and parallel, how do they interact with each other in the transformer
- placeholder?
- consistency between actions and states?
Model Design
直接沿用的Cosmos-Predict2-2B模型，本质上只是将多模态的数据伪造成图片然后通过causal mask选择具体的训练性质。
流程
| Phase / Model | Base Model | Planning Model |
| ——————– | ———————————————————————————————————————————————————- | ————————————————————————————————————————————————————————————————- |
| Objective | 作为一个低级控制器，直接根据当前状态输出可执行的动作。 | 作为一个模拟器和评估器，评估候选动作的好坏，用于规划。 |
| Dataset | Demonstration data only: suppose m success and n failure (m » n), 50% m for policy training, 25% m+ 50%n for world-model and value function training. | Rollouts dataset: Real trajectories generated by the base model, with various initial conditions. 90% for fine-tuning world model + value function evenly separated, 10% for policy training. |
| Training Recipe | From scratch | Finetune on base model. |
| Checkpoint | Checkpoint A that outputs actions | Checkpoint B that predicts future states and also the |
| Inference Recipe | Parallel decoding -> $p(a,s’,V(s’) \vert s)$ | Autoregressive -> $p(a, s’, V(s’)\vert s)$ -> $p(V(s’),s’\vert a,s)$ -> $p(V(s’)\vert a,s,s’)$ |
Planning mode deployment:
- 用
Checkpoint A采样生成 3x5 个候选动作序列 。 - 用
Checkpoint B(Autoregressive 模式) 预测这 3x5 个动作未来的状态和 Value，选出得分最高的动作执行 (Best-of-N sampling) 。 - 由于计算密集，是同步推理