GAMs explained: 3D Grounded Action Models boost accuracy 61%

September 26, 2026

arXiv International

Overview

A new arXiv paper introduces Grounded Action Models (GAMs), a novel paradigm for robot foundation models built on 3D grounding, achieving up to 61% average success rates. GAMs can be conditioned using language, point, or box prompts, transforming them into a shared object-centric representation for selected objects. This model achieved a 55.3% average success rate on RoboTwin 2.0 and a state-of-the-art 61% average success rate across 16 perturbation settings in LIBERO-PRO, significantly enhancing robot object understanding and task execution precision.

In Depth

Key Findings

A paper published on arXiv introduces “Grounded Action Models (GAMs),” a novel paradigm for robot foundation models constructed upon 3D grounding. GAMs achieved an average success rate of 55.3% on the RoboTwin 2.0 benchmark and a state-of-the-art average success rate of 61% across 16 perturbation settings on the LIBERO-PRO benchmark, surpassing existing cutting-edge technologies. This represents a groundbreaking advance in the robot’s ability to deeply understand object-environment interactions and plan and execute precise actions.

Technical / Clinical Details

At the core of Grounded Action Models (GAMs) is their capacity to generate an “object-centric representation” of the target object for manipulation from diverse input formats, including natural language prompts, 3D points, or 2D bounding box prompts. This representation integrates the geometric and semantic properties of the selected object, providing a robust foundation for the robot to infer appropriate actions. For instance, given a language instruction like “pick up the blue block,” GAMs identify the blue block from visual input, convert its 3D position and shape into an object-centric representation, and subsequently generate suitable grasping poses and motion paths. This model demonstrates versatility across multiple robot manipulation tasks, exhibiting robustness and precision particularly in complex object manipulation scenarios.

Background & Context

Robot foundation models are gaining significant attention as a next-generation approach towards achieving general-purpose robot intelligence capable of handling diverse tasks and environments. However, previous models often struggled with adequately “grounding” their understanding in the 3D geometry and semantic relationships of the physical world. GAMs address this challenge by prioritizing the concept of 3D grounding. This enables robots not just to perform pattern recognition but also to more accurately predict physical relationships between objects and the physical outcomes of their own actions.

Strategic Significance & Outlook

The advent of GAMs holds the potential to significantly enhance robots’ ability to autonomously perform complex real-world tasks. Operations previously considered exclusive to humans, such as precise assembly in manufacturing, handling diverse items in the service industry, and intricate household chores, could become achievable by robots. Particularly, the capability for robots to understand and execute tasks solely from language instructions simplifies robot programming, opening up a future where more non-specialists can utilize robots. This technology is expected to dramatically boost the intelligence of general-purpose humanoid and industrial robots, serving as a foundational catalyst for transforming human life and industrial paradigms.

Source: https://arxiv.org/abs/2609.23863

Get our weekly technology intelligence — free

Receive an infographic that lets you judge at a glance whether each field’s analysis report is worth reading.

Subscribe Free — Weekly Tech Intelligence

By subscribing, you’ll receive Troy-Technical’s weekly technology intelligence newsletter.