Vibotaku’s AI in Games Weekly: 2026-08-03

2026-07-27 - 2026-08-03

Highlights

GUI agents are becoming full workflow agents. Qwen-UI-Agent mixes mobile control, desktop use, browser tasks, CLI execution, and proactive services in one training and evaluation stack.
World-action models are moving from video prediction toward planning. World Action Planner uses imagined rollouts as a search surface for compositional tasks and changed layouts.
Egocentric data generation is getting more useful for long-horizon agents when it keeps scene anchors, camera geometry, and end-effector motion aligned.
3D world generation work is paying attention to navigability. Genie Sim PanoWorld turns a single panorama into a freely roaming 3D Gaussian scene through a trajectory-controllable video stage.
Security is catching up with world models. The new survey frames poisoning, spoofing, prompt injection, trajectory attacks, and memory compromise as lifecycle risks for embodied agents.

Paper	Recommendation Index	Highlight
Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents	SSS	A production-shaped GUI agent stack with real-device mobile training, hybrid GUI plus CLI actions, online RL beyond 100 turns, and strong public community traction.
World Action Planner: Generalizable Decision-Making with Action-Conditioned World Models	SSS	Uses VLM planning plus action-conditioned world model rollouts to refine actions before execution on compositional and zero-shot manipulation tasks.
EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE	SS	Generates action-aligned egocentric rollouts and reports real-robot OOD gains when synthetic trajectories are added to scarce demonstrations.
Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulation	SS	Builds freely navigable 3D Gaussian indoor scenes from one 360 degree panorama through explicit trajectory-controlled panoramic video.
Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation	A	Treats world models as a security boundary across data, grounding, imagination, evaluation, execution, memory, and tools.

Recommendation Index: SSS (Must Read)
Links: arXiv / Hugging Face / GitHub / Project page
Vibotaku’s Note: Qwen-UI-Agent is worth reading as agent infrastructure, even for game teams. It treats GUI control as a long-horizon runtime problem across mobile, desktop, web, DeepSearch, and shell commands, with batched actions in a single model turn. The report claims online RL on trajectories over 100 turns using more than 10,000 concurrent environments, plus a data flywheel where agents build tasks, diagnose failures, and choose the next iteration. The numbers are strong: 82.1% on MobileWorld, 92.2% on MobileWorld-Real, 97.5% on AndroidDaily, 79.5% on OSWorld-Verified, 73.6% on WebArena, and 81.5% on ScreenSpot-Pro. The GitHub repo had 1.9k stars, 181 forks, 57 commits, and an Aug 2 commit when checked. For game-agent teams, the useful pattern is the harness: combine UI actions, tools, stateful workflows, verifiers, and online rollouts instead of treating the agent as a one-shot screen clicker.

Recommendation Index: SSS (Must Read)
Links: arXiv / Hugging Face / GitHub / Project page
Vibotaku’s Note: World Action Planner has the shape I want to see in game agents: a planner proposes an action, imagines the result, then searches or optimizes before touching the environment. The system combines a VLM with a pose-image conditioned world model and tests compositional generalization, changed layouts, and zero-shot Robosuite tasks using hard-coded grasp and release primitives. The project page shows the planner using imagined rollouts to handle transition actions, object relocation, collision risk, and fine cube alignment. The repo is early, with 6 stars, 3 commits, and a July 29 README update when checked, but the code path is public. For games, this maps cleanly to NPCs that need to try plans in imagination before spending expensive environment steps.

Recommendation Index: SS (Strong Read)
Links: arXiv / Project page
Vibotaku’s Note: EgoGenesis focuses on the part of synthetic experience that usually breaks first: action alignment over time. It uses Online Anchored Projective Memory to preserve the first-frame 3D scene anchor while refreshing recent state, then uses Action-3D RoPE to put end-effector motion into camera-aware 3D coordinates. The project page reports 210K source-balanced training clips, 79.3% lower depth error and 78.3% lower camera error at frame 80 against a RoPE baseline, and OOD success gains when adding 400 generated trajectories to 400 real ones: 77% to 84% on single-arm tasks and 53% to 70% on dual-arm tasks. For games and embodied agents, the lesson is simple: synthetic rollouts need stable object identity, metric action control, and persistent scene memory before they become useful training data.

Recommendation Index: SS (Strong Read)
Links: arXiv
Vibotaku’s Note: Genie Sim PanoWorld is a scene-generation paper with a very practical hook: start from a single 360 degree panorama, generate trajectory-controlled panoramic video, then lift that into a real-time free-viewpoint 3D Gaussian scene. The paper uses a NavMesh-planned SE(3) roaming trajectory, dense geometry-warped conditioning, long and short trajectory mixed training, and a shortcut-model consistency objective to produce video in four CFG-free denoising steps. For virtual worlds, this is interesting because the output is not just a pretty view. It is a navigable asset with an explicit path through the scene, which is closer to what simulation, QA, and NPC training pipelines need.

Recommendation Index: A (Should Read)
Links: arXiv
Vibotaku’s Note: This survey is useful because it names the attack surface around world models instead of treating them as a black-box planner component. The paper walks through threats across data construction, representation learning, state grounding, imagination, trajectory evaluation, execution, adaptation, memory, and tools. That matters for game agents too. A compromised world model can make an agent trust the wrong affordance, imagine a safe path through a dangerous space, accept poisoned feedback, or carry a bad memory into future plans. The practical takeaway is to evaluate world-model failures by lifecycle stage: provenance checks for data, robust grounding for state, uncertainty for imagined futures, trajectory gating before action, and audit logs for feedback and memory updates.

Qwen-UI-Agent arXiv: https://arxiv.org/abs/2607.28227
Qwen-UI-Agent Hugging Face page: https://huggingface.co/papers/2607.28227
Qwen-UI-Agent project page: https://tongyi-mai.github.io/Qwen-UI-Agent/
Qwen-UI-Agent GitHub: https://github.com/Tongyi-MAI/MAI-UI
World Action Planner arXiv: https://arxiv.org/abs/2607.27599
World Action Planner Hugging Face page: https://huggingface.co/papers/2607.27599
World Action Planner project page: http://worldactionplanner.github.io/
World Action Planner GitHub: https://github.com/XiangchengZhang/world-action-planner
EgoGenesis arXiv: https://arxiv.org/abs/2607.28243
EgoGenesis project page: https://egogenesis.github.io/
Genie Sim PanoWorld arXiv: https://arxiv.org/abs/2607.26646
Security of World-Model-Based Embodied AI arXiv: https://arxiv.org/abs/2607.28226