Aether-7B-5Attn: A Fully Open Sovereign Foundation Model

On 19 July 2026, VIDRAFT, a single Korean AI startup, released Aether-7B-5Attn — a foundation model that meets the most stringent definition of open source: not just the weights, but the data recipe, the training code, the full logs, intermediate checkpoints, and the complete architecture source, all under Apache 2.0.

What "Fully Open" Means

Most models labelled "open" release weights under a permissive license but keep the training pipeline, data composition, and intermediate states proprietary. Aether follows the standard that Allen Institute's OLMo established: every artifact required to rebuild the model from scratch is published alongside the weights.

Artifact	Available
Model weights	Apache 2.0
Training data recipe and mixing weights	Published
Tokeniser	Open
FSDP launcher and training loop	Open
All hyperparameters	Documented
Step-by-step training logs	Published
Intermediate checkpoints (110k, 115k, 162k)	Published
Architecture source	Apache 2.0

The claim is narrow and verifiable: among the world's fully-open sovereign foundation models, Aether is the only one built by a single AI startup rather than a national lab, a government consortium, or a university coalition.

The Latin Square Attention Experiment

Aether's 49 layers place five distinct attention mechanisms on a 7×7 Latin square — a design that converts a vague architectural intuition into a controlled experiment. Each row and column of the square contains each attention type exactly once, isolating the effect of heterogeneous placement from the effect of which layer receives which operator.

Attention Type	Layers Applied
Full (standard softmax)	Position (1,1), (2,5), (3,2), (4,6), (5,3), (6,7), (7,4)
Sliding window (local)	Position (1,2), (2,6), (3,3), (4,7), (5,4), (6,1), (7,5)
Dilated (strided)	Position (1,3), (2,7), (3,4), (4,1), (5,5), (6,2), (7,6)
Differential	Position (1,4), (2,1), (3,5), (4,2), (5,6), (6,3), (7,7)
Sparse (top-k)	Position (1,5), (2,2), (3,6), (4,3), (5,7), (6,4), (7,1)

The five operators are realised as five distinct computational mechanisms: standard softmax attention, gated SDPA, native sparse attention, sliding-window attention, and differential attention. To our knowledge, no released model has arranged heterogeneous attention on a Latin square — making Aether a genuine research instrument as well as a deployable model.

Architecture and Training

Aether-7B-5Attn is a 6.59B-parameter MoE model with approximately 2.98B active parameters per token. It uses 25 routed experts (top-7) plus one shared expert, with standard load-balancing loss.

Component	Specification
Architecture	Decoder-only Transformer, 49 layers
Total / active parameters	6.59B / ~2.98B
Attention heads	16 query / 4 key-value (GQA), head dim 128
FFN intermediate	6144 (dense)
Experts	25 routed (top-7) + 1 shared
Vocabulary	151,936 (BPE)
Context length	4,096 (RoPE θ = 10,000)
Pretraining	144.2B tokens, 162k steps, 16× B200, ~46 days
Precision	bfloat16
License	Apache 2.0

Training on 144.2B tokens across 162,000 steps on 16 NVIDIA B200 GPUs in a 2-node FSDP configuration completed in roughly 46 days. The released checkpoints span 110k, 115k, and 162k (annealed) steps, enabling training-dynamics research on a fully-open model.

A Methodological Lesson in Post-Training

The instruct variant, Aether-7B-5Attn-it, was selected not by training loss but by held-out benchmark accuracy. This is a deliberate methodological stance: for small models, training loss rewards format-fitting, not competence. Sweeping three learning rates (2e-6, 6e-6, 2e-5) and selecting on held-out accuracy produced the instruct variant that scores 34.9% average across the K-AI 4-benchmark suite — an 8.2 percentage point gain over the base model. The lowest-loss checkpoint (2e-5) collapsed on one subject entirely.

The Sovereignty Thesis

For enterprise architecture teams, Aether's primary contribution is not its benchmark score. It is an existence proof that a mid-size organisation can hold the entire means of production of a foundation model — and therefore rebuild, audit, and evolve it independently.

"Real sovereignty is the capacity to rebuild," the VIDRAFT team writes. "It requires the recipe, not just the dish." Aether's recipe is verifiably complete: download the code, run the pipeline, and the training binaries reproduce byte-for-byte.

For enterprises evaluating sovereign AI infrastructure, Aether-7B-5Attn represents a new baseline for what "open" must mean.