The Preference Gap: Why Mobile Agents Fail at True Personalization

Mobile GUI agents have reached an inflection point. Benchmarks like AndroidWorld and MobileWorld demonstrate that contemporary models can navigate complex multi-app workflows, execute precise touch gestures, and complete well-defined tasks with increasing reliability. The technical challenge of converting natural language instructions into sequences of UI actions, once the primary barrier to practical deployment, is steadily yielding to advances in multimodal foundation models and structured reasoning techniques.

Yet this progress masks a deeper deficiency. As products like Doubao Mobile Assistant and OpenClaw pivot from passive tools to proactive personal assistants, the evaluation criteria must shift accordingly. The relevant question is no longer whether an agent can execute "order a sugar-free Coca-Cola on TaoDian," but whether it can handle "order me lunch" by inferring dietary constraints, budget limitations, and platform preferences from historical behavior, all while knowing when to ask for clarification versus when to remain silent.

The paper "KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation" exposes this gap with striking clarity. The authors, representing Zhejiang University, Apple, and Tencent, introduce a benchmark that reveals how frontier models, including Claude Sonnet 4.6, plummet below 50% accuracy when tasks require preference inference from behavioral logs rather than explicit context. The bottleneck is not GUI navigation, which agents handle with competence, but the messier human challenges of preference acquisition, consent negotiation, and intervention calibration.

Rethinking Personalization as an Interactive Process

Existing approaches to personalized mobile agents have largely treated user modeling as a static prediction problem. Datasets like FingerTip 20K mine behavioral logs to predict next actions, while frameworks such as PersonalAlign and Me-Agent focus on recovering intent from fixed historical trajectories. These methods assume that user preferences are fully observable in past data, and that the agent's role is merely to pattern-match against these records.

KnowU-Bench dismantles this assumption through a simple but profound architectural choice: user profiles are hidden from the agent. Instead of providing a structured preference file, the benchmark exposes only raw behavioral logs, forcing the agent to perform genuine inference rather than context lookup. When faced with a vague instruction like "order me a case of canned Cola," the agent cannot simply query a database of stated preferences. It must interpret past orders, notice patterns in brand selections, and crucially, recognize when ambiguity remains.

This design necessitates a capability that prior benchmarks ignore: interactive preference elicitation. KnowU-Bench instantiates an LLM-driven user simulator grounded in structured profiles, enabling multi-turn clarification dialogues. If the agent cannot determine whether the user prefers sugar-free or regular beverages from history alone, it must ask. This transforms personalization from a batch prediction task into an online, partially observable decision process. The simulator provides realistic feedback, allowing agents to probe for constraints, confirm hypotheses, and handle the fluid nature of human preference.

The technical implementation spans 86 personalized tasks alongside 42 general GUI tasks, all executing within a containerized, rooted Android emulator. This reproducible environment ensures that evaluation