Stanford and Caltech Connect GPT-6 Astra Directly to Humanoid Robot
Researchers bypass traditional vision-language-action policy layers, letting OpenAI's frontier model orchestrate physical manipulation through modular skill tools.
Key takeaways
- Stanford and Caltech introduced HomeBody, an architecture that uses GPT-6 Astra as a high-level task planner for a Unitree G1 humanoid robot.
- The system bypasses task-specific Vision-Language-Action (VLA) control layers, instead utilizing structured tool calls into a five-skill library.
- The robot builds a digital twin in Nvidia Isaac Sim using exploration data, enabling long-horizon loco-manipulation from spatial memory.
- Practical limitations include reasoning latency, finger-servo overheating, and local compute requirements on an RTX 4090 GPU.
Researchers from Stanford University and the California Institute of Technology have unveiled HomeBody, a system that connects OpenAI's GPT-6 Astra directly to a humanoid robot to complete complex multi-step household chores. The project demonstrates that a general-purpose vision-language model can autonomously navigate an unfamiliar kitchen, tidy up, and retrieve items from drawers without relying on task-trained intermediate control layers, according to The Decoder.
Developed at Stanford's Movement Lab by Gio Huh, Cayden Gu, Takara E. Truong, C. Karen Liu, and Guy Tevet, HomeBody replaces the conventional vision-language-action (VLA) policy pipeline. Instead, GPT-6 Astra runs remotely as a high-level planner, issuing structured tool calls to trigger modular robot skills while controllers and modular skills execute physical motions, as detailed on Stanford's project page.
How GPT-6 Astra Controls the Unitree G1
Traditional approaches to embodied AI typically rely on a trained control or vision-language-action (VLA) layer. As reported by AI Weekly, HomeBody bypasses this layer by presenting the robot's capabilities to GPT-6 Astra as an extensible skill library with five core actions:
- Navigate: Follows a planned route to a 2D goal and facing point in map coordinates.
- Pick: Identifies an image coordinate normalized from 0 to 1000, predicts analytical grasps using stereo depth, and lifts an object with a chosen arm.
- Place: Moves a held item to a specified 3D release target relative to the torso frame.
- Open drawer: Aligns visually with a drawer handle, hooks it, and walks backward.
- Pick from drawer: Reaches into an open drawer and lifts the selected object clear of the rim.
To translate Astra's high-level tool selections into physical execution, the local system employs Fast-FoundationStereo for real-time depth estimation from a D435i camera, paired with SAM 2.1 and SAMURAI memory tracking for visual servoing during approaches. Whole-body balance during loco-manipulation is maintained by a pretrained Adaptive Motion Optimization (AMO) controller. Arm commands operate at 250 Hz, while AMO updates lower-body gait and posture at 50 Hz, according to Stanford's project documentation.
Real2Sim Spatial Memory and Autonomous Cleanup
Before executing tasks, the Unitree G1 explores the room autonomously under Astra's direction, capturing iPhone video, stereo frames, LiDAR data, joint states, and exploration waypoints. As described by Aree Blog, HomeBody feeds this data into a Real2Sim pipeline to build a 3D digital twin inside Nvidia Isaac Sim. The system registers the live Super Odometry SLAM map with the simulated environment using iterative closest point (ICP) alignment.
This reconstructed spatial memory allows GPT-6 Astra to track items even when they leave the humanoid's immediate field of view. During demonstrations, researchers gave the system underspecified prompts such as: "Clean up all of the coffee bags and put them in the middle, and throw away all of the milk and orange juice cartons that have gone bad." The system coordinated successive navigation trips, gathered the coffee bags on the kitchen island, and threw away spoiled cartons.
In a second scenario, a user requested out-of-sight medicine: "I forgot my medicine, can you get it for me? Also throw out the bad carton while you are at it." Using spatial keyframes, the robot navigated to the drawer, opened it with its right hand, retrieved the medicine bottle, handed it to the person, and discarded the carton with its left hand, according to Stanford's research team.
Practical Constraints and System Limitations
While HomeBody shows that general-purpose frontier models can handle long-horizon physical planning, the researchers highlight several operational bottlenecks, noted by AIToday and AI and Tech News:
- Reasoning Latency: Astra's cloud inference introduces noticeable pauses between skill executions while the model interprets previous results and formulates its next step.
- Hardware Endurance: Extended operation leads to finger-servo overheating on the Unitree G1.
- Compute Requirements: Local motion planning, stereo depth, and segmentation require an RTX 4090 laptop GPU running alongside remote API calls.
- Setup Overhead: Real2Sim environment reconstruction adds upfront time and API costs before chores can begin.
The researchers note that published previews do not represent a large-scale statistical benchmark; success rates and trial counts were not detailed on the project page, and most skill previews were sped up to about 7–10 seconds, with some grasping failures requiring local automated retries. OpenAI, which released GPT-6 Astra with vision, reasoning, and tool-calling capabilities in September 2026, has previously stated its intent to re-enter robotics for personal applications. The HomeBody project repository is hosted on GitHub, with full implementation code marked as coming soon.
Frequently asked questions
What hardware does the HomeBody system use?
HomeBody runs on a Unitree G1 humanoid robot equipped with a D435i camera and LiDAR. Local perception, tracking, and motion planning run on a Razer Blade laptop with an RTX 4090 GPU, while GPT-6 Astra runs remotely.
How does GPT-6 Astra control the robot without a VLA layer?
Instead of predicting low-level motor torques, GPT-6 Astra outputs structured tool calls selecting from five composable skills (navigate, pick, place, open drawer, pick from drawer) with spatial targets, delegating inverse kinematics and trajectory execution to local controllers.
What are the main technical limitations of HomeBody?
Primary constraints include reasoning latency from GPT-6 Astra causing pauses between steps, hardware endurance issues such as overheating finger servos, Real2Sim setup costs, and heavy local compute requirements.
Sources
- Researchers plug GPT-6 Astra directly into a robot and let it clean up an unfamiliar kitchenThe Decoder · Sep 27, 2026
- HomeBody: A Humanoid That Explores, Remembers, and Acts on Its Owntml.stanford.edu · Sep 1, 2026
- Stanford's HomeBody Wires GPT-6 Astra Directly to a Unitree G1, Skips the VLA LayerAI Weekly · Sep 27, 2026
- Researchers Put GPT-6 Astra in Humanoid Robot for Household TasksAree Blog · Sep 27, 2026
- HomeBody: GPT-6 Astra runs a Unitree G1 kitchen tidy-upAIToday · Sep 27, 2026
- Researchers Test GPT-6 Astra Direct Integration With Autonomous Robot Kitchen CleanupAI and Tech News · Sep 27, 2026
How this story was made: the newsroom picked it up from the-decoder.com, gathered the full text of the sources above, and drafted it with AI assistance. Every factual claim was then checked against those sources before publishing (36 claims checked). Illustrations marked as AI-generated are not photographs. Spotted an error? Tell us.
Published September 28, 2026 at 01:06 UTC
