Reflection AI Releases Open Weights and Recipes for 501B Beam Model
The sparse mixture-of-experts model activates 23 billion parameters per token and launches with day-zero support across AWS, vLLM, and SGLang.
In short
- Reflection AI released model weights and training recipes for Beam-1.0-501B-A23B under the Apache 2.0 license on Hugging Face.
- Beam employs a 52-layer sparse MoE architecture activating 23 billion parameters out of 501 billion total parameters per token.
- A hybrid attention structure limits KV cache memory footprint to 36.08 GiB across a 1 million token context window.
- Deployments are supported natively on AWS SageMaker JumpStart, HyperPod Inference, SGLang, and vLLM across BF16, FP8, and NVFP4 formats.
On October 16, 2026, Reflection AI released the model weights and post-training recipes for Beam-1.0-501B-A23B under the permissive Apache 2.0 license on Hugging Face. The release brings the startup's flagship text-to-text model into public availability, providing organizations and developers with full access to customize its weights for agentic coding, multi-step software engineering, and scientific reasoning.
Beam operates as a sparse mixture-of-experts (MoE) model comprising 501.3 billion total parameters, but only routes 23.4 billion active parameters per token. The architecture was developed to challenge frontier reasoning systems while maintaining lower computational demands during inference. Alongside the core model weights, Reflection published fine-tuning code, evaluation harnesses, and speculative decoding drafter weights.
Sparse Mixture-of-Experts Architecture
Beam’s 52-layer design combines 3 dense SwiGLU layers followed by 49 MoE layers. According to the official Hugging Face technical overview, each MoE layer hosts 256 individual experts with an intermediate size of 2,560, representing roughly 39 million parameters per expert. Together, the experts account for 98.4% of the overall 501 billion parameters.
To balance expert assignments, the model adapts DeepSeek-V3's group-limited routing strategy. A sigmoid function scores all 256 experts, dividing them into eight groups of 32. The router isolates the top four groups by aggregating their leading two scores and selects eight experts in total to process each token. The expert outputs are combined with normalized router weights and scaled by 2.5 without relying on a dedicated shared expert.
The residual connections diverge from conventional transformer structures. Beam scales sublayer additions with depth: at step k, sublayer outputs are added with a weight of 1/sqrt(k), while the existing residual stream is scaled by sqrt((k-1)/k). Because the sum of the squares equals one, the residual stream maintains an even scale across all 104 additions without accumulating exploding variance.
Hybrid Attention and Long-Context Memory
Beam provides an effective context window of 1 million tokens, with a knowledge cutoff date of June 30, 2026. As documented in the Hugging Face technical blog post, the attention mechanism interleaves 43 sliding-window layers and 9 global attention layers.
The 43 sliding-window layers attend strictly to the preceding 512 tokens using Rotary Position Embeddings (RoPE). Every sixth layer, along with the final layer, handles global context without positional encodings, applying a dynamic scaler to query states that widens context capacity through temperature-like scaling. Both layer variants utilize grouped-query attention with 48 query heads and 8 key-value heads, bounded by a learned sigmoid gate before the final projection.
This hybrid configuration reduces memory consumption. In a standard 52-layer global setup, a full 1 million token context in BF16 precision consumes roughly 208 GiB per sequence in key-value cache memory. By restricting 43 layers to a fixed 512-token cache, Beam’s KV cache grows at 36 KiB per token, totaling 36.08 GiB at maximum context.
Pretraining Scale and Reinforcement Learning
Prior to releasing weights, Reflection detailed Beam's training pipeline. According to reporting by MarkTechPost, the base model was pretrained on 23.8 trillion tokens over under four weeks using a cluster of 6,144 NVIDIA GB300 NVL72 GPUs. Data filtering discarded 95% of raw web data while retaining roughly 1.8 trillion specialized tokens, including curated code and processed PDF documents.
Reinforcement learning formed the primary scaling phase, employing 10,500 NVIDIA GB300 GPUs across four weeks. The training infrastructure processed more than 100 million rollouts across 1.3 billion sandboxed execution environments, covering nearly 1 million coding and STEM scenarios. Operating an asynchronous policy gradient system, the run remained stable when learning against policies up to 107 weight versions behind the active deployment.
As reported by SiliconANGLE, Reflection used custom reliability algorithms that averted configuration mismatches between training and inference environments. The system automatically rolled back training checkpoints during data errors rather than restarting runs, which minimized cluster downtime.
Benchmark Performance and Tool-Use Features
Reflection reported evaluation metrics across standard SWE-bench and mathematical benchmarks for its BF16, FP8, and NVFP4 checkpoints. On SWE-bench Verified, Beam achieved 81.6% in BF16, 80.2% in FP8, and 80.7% in NVFP4. On Terminal Bench v2.1, it scored 80.1% in BF16, while reaching 97.7% on AIME 2026 and 90.6% on GPQA Diamond.
Earlier comparisons reported by Unite.AI noted that Reflection reported Beam achieved comparable scores to GLM-5.2 on reasoning evaluations while using three to four times less inference compute, an estimate based on active parameters and generated tokens. However, Reflection acknowledged that larger frontier systems such as Kimi K3 retain higher raw performance ceilings on select broad benchmarks.
Beam incorporates a response template inside its tokenizer, mapping reasoning, plain-text answers, and XML-style tool calls into discrete fields without regex post-processing. A user-configurable reasoning effort setting allows developers to select between five levels—low, medium, high, xhigh, and max—to adjust the model's compute budget according to task requirements.
Cloud Deployments and Ecosystem Integrations
Beam is released alongside deployment recipes across open-source frameworks and major cloud providers. According to Amazon Web Services, the model is available via Amazon SageMaker JumpStart in US East (N. Virginia) and US East (Ohio) as well as SageMaker HyperPod Inference running SGLang or vLLM.
Hardware footprint varies across checkpoint formats:
- BF16: Demands approximately 1 TB in storage and requires multi-node layouts, such as 16 NVIDIA H200 GPUs partitioned across tensor and expert parallelism.
- FP8: Fits on single 8-GPU nodes equipped with Hopper or Ampere hardware.
- NVFP4: Compresses weights to approximately 280 GB, enabling serving on four NVIDIA B200 or B300 Blackwell GPUs.
Along with cloud availability, Reflection introduced Mirror CLI, an open-source terminal harness featuring a Python frontend and a Rust execution core for orchestrating local coding agents. Day-zero support covers Hugging Face Transformers version 5.20.1 or higher, vLLM, and SGLang, alongside speculative decoding via the companion DFlash2 drafter model.
Questions people ask
What license governs Beam-1.0-501B-A23B?
Reflection AI released the weights, post-training resources, and model recipes under the permissive Apache 2.0 license.
What hardware is required to run the model?
The BF16 checkpoint requires multi-node setups such as 16 NVIDIA H200 GPUs. The FP8 checkpoint can run on an 8-GPU Hopper node, while the 4-bit NVFP4 checkpoint can fit on four NVIDIA Blackwell B200 or B300 GPUs.
How does Beam reduce memory usage for long contexts?
Beam uses 43 sliding-window attention layers with a 512-token limit and 9 global attention layers, reducing KV cache growth from 208 KiB to 36 KiB per token across its 1 million token context.
Sources
- Introducing Reflection Beam: an open-weight model for agentic coding on AWS | Amazon Web ServicesAmazon Web Services, Oct 16, 2026, official source
- reflection/Beam-1.0-501B-A23BHugging Face, Oct 7, 2026, official source
- Welcome Beam: a 501B open model you can own, customize, and scaleHugging Face, Oct 16, 2026, official source
- Reflection AI Introduces Beam: A 501B Open-Weight MoE Model With 23B Active Parameters for Coding and Agentic Workloadsmarktechpost.com, Oct 5, 2026
- Reflection AI open-sources Beam’s weights, post-training recipesSiliconANGLE, Oct 16, 2026
- Beam is now available — ReflectionReflection
- Reflection AI Unveils Beam, a 501B-Parameter Open-Weight ModelUnite.AI, Oct 5, 2026
How this story was made: the newsroom picked it up from aws.amazon.com, Reddit and Google News, gathered the full text of the sources above, and drafted it with AI assistance. Every factual claim was then checked against those sources before publishing (48 claims checked). Illustrations marked as AI-generated are not photographs. Spotted an error? Tell us.
Published October 17, 2026 at 00:07 UTC
