https://crazyjassbread.github.io/tags/vlm/ Recent content in VLM on CrazyBread' Blog Hugo -- 0.147.3 en-us Tue, 26 May 2026 13:22:30 +0800 https://crazyjassbread.github.io/posts/worldmodel/vlm/ Tue, 26 May 2026 13:22:30 +0800 https://crazyjassbread.github.io/posts/worldmodel/vlm/ <h2 id="目录">目录</h2> <p><a href="#Odysseus-Scaling-VLMs-to-100-Turn-Decision-Making-in-Games-via-Reinforcement-Learning">Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning</a></p> <h2 id="odysseus-scaling-vlms-to-100-turn-decision-making-in-games-via-reinforcement-learning">Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning</h2> <p><a href="https://arxiv.org/abs/2605.00347">论文链接</a></p> <p>文章最有意思的地方在于 VLM 可以同时输出多个动作决策，这就可以用来解决 Zelda 环境中需要一次执行两个动作才能解决特定任务的情形了（而且因为单帧输入后状态几乎没有改变，所以他们也加入了动作循环，来让模型在连续的帧中执行同样的动作）</p> <p>这篇研究的背景知识是 <strong>POMDP</strong>（部分可观测马尔可夫决策过程） </p> $$\begin{aligned} POMDP &= \langle S, A, T, R, \Omega, O, \gamma \rangle \\ s_{t+1} &\sim T(s_{t+1} | s_t, a_t) \\ a_t &= \pi(o_t) \\ r_t &= R(s_t, a_t) \\ o_t &\sim O(o_t | s_t) \\ \text{Goal} &= \max_\pi \mathbb{E_{\pi}} \left[ \sum_{t=0}^{\infty} \gamma^t r_t \right] \end{aligned}$$<p>对于 VLM 模型的 finetuning，这篇论文发现用 Critic free 的 RL 方法在 Super Mario 游戏中训练效果不稳定（没给出有信服力的解释）</p>