AI圈报
论文研究普通

Mean-field Token Steering

信息来源:arXiv·

内容摘要

We consider stochastic token steering through a deep stack of transformer layers as a mean-field control problem for an interacting particle system on the unit sphere, where interactions arise from the attention mechanism. In the mean-field limit, the token population evolves via a controlled integro-partial differential equation where the feedback control acts as a corrective vector field guiding the population toward an aggregate configuration of tokens in the representation space. We deduce the minimum effort control to accomplish this task over a fixed time horizon interpreted as normalized layer depth. We show that the resulting controlled dynamics for the token population can be understood as a mean-field Schrödinger bridge adapted to the sphere geometry. Building on this, we propose a novel generalization of the dynamic Sinkhorn recursion to compute the consistent token evolution. Numerical experiments confirm that the proposed approach achieves exact steering of the token population.
内容分类AI 论文与研究
内容层级普通情报
发布时间(北京时间)
本站收录时间(北京时间)
信息来源arXiv
站内情报编号intel-2902955390d388830fd84d7e