What Dream-Cubed is
Dream-Cubed is a large-scale dataset and family of generative models for generating Minecraft worlds at block resolution. Sakana AI developed it jointly with New York University and released it on July 29, 2026. The paper's formal title is "Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes," and the data, models, and code are all public.
Three things were released: the technical blog at pub.sakana.ai/dream-cubed, the paper on arXiv (2604.22847), and the code on GitHub at SakanaAI/DreamCubed.
We are excited to share our latest work, together with New York University: Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes. / Technical Blog: https://pub.sakana.ai/dream-cubed / Paper: https://arxiv.org/abs/2604.22847 / Code: https://github.com/SakanaAI/DreamCubed — From the passages on the collaboration with New York University, the paper's formal title, and where the releases live
Why Minecraft
The choice of subject matter tells you how the research thinks. Generative AI has come a long way in images, video, and text, but the interactive 3D worlds of video games have gone largely untouched. Sakana AI is aiming at that gap.
The reason Minecraft works is how its world is built. Minecraft's 3D world is made of voxels — discrete blocks with meaning, like sand, stone, and water, stacked together. That structure is convenient for a generative model, and that is the observation the work rests on.
Text is divided into words, which is what made language model training possible in the first place. A Minecraft world is divided into blocks the same way. A piece of the game world is nothing more than a collection of block tokens. Because that correspondence holds, the large Transformers used for language can be carried over directly.
Generative AI has made incredible progress in the fields of image, video, and text generation. Despite success in these modalities, the interactive 3D worlds of video games have received much less research attention. / The 3D world of Minecraft is made up of voxels: discrete, compositional building blocks like sand, stone, and water. These semantically meaningful block types provide a perfect abstraction for generative modeling: each piece of the game world is simply a collection of block tokens that come together to create a simulacrum of the natural world. — From the passages on 3D worlds as a research gap and on why voxel structure suits generative modeling
A dataset of 30 billion blocks
The composition of the data is public too. One part comes from Minecraft's procedural generation — the terrain the game builds by itself — yielding more than 1.8 million 32×32×32 chunks. The other comes from maps built by hand by human creators, at a scale of 200,000 (the source writes 200,00+, with a digit dropped). Together they add up to more than 30 billion blocks.
Composition of the Dream-Cubed dataset (per the official release)
200,00+, a digit dropped / used with creator permission)Procedural data is effectively unlimited, but collecting it as-is produces a skew. Villages are the most structurally complex areas of the game world and the least common. So targeted pipelines were built to go after specific biomes and secure enough of them. The experiments below show how much that balancing matters downstream.
Human data is the scarce side. Sharing sites like Planet Minecraft exist, and some professional creators earn income selling hand-crafted maps. The team contacted one of them and obtained permission to use several large, high-quality maps. The release itself is stated to be made in accordance with the map creators.
We create automated data collection and curation pipelines, collecting 1.8+ million 32³ chunks of procedurally generated Minecraft worlds, and 200,00+ chunks from human-authored maps, comprising over 30 billion total blocks. / We collect chunks from the surface level of the world and assign a class label based on the biome they are in, creating a dataset for biome-conditional generation. / Villages, for example, are both the most structurally complex and the least common areas of the game world. To overcome this data balancing problem, we set up targeted collection pipelines to identify and collect specific biomes to ensure adequate representation, which we show can have a very significant impact on downstream models. / Biome-conditioned 32³ chunks generated by our top-performing discrete MD4 model and continuous DDPM model for each of the 15 biome classes. / We reached out to one such professional map creator, who granted permission to use several of their large and high-quality maps for scraping. We release all our data and models in accordance with the map creators for the larger research and Minecraft community. — From the passages on dataset scale, biome labeling and the number of classes, handling the imbalance, and permission for human-authored maps
Trading efficiency for fine-grained control
The heaviest technical decision sits here. Understanding it shows what the research chose to prioritize.
Training at native resolution, uncompressed
Diffusion models of the kind used in image generation normally operate on representations compressed by something like a variational autoencoder (VAE). Shrinking the data first buys a large gain in compute efficiency.
Dream-Cubed does not do this. It explicitly adopts the design decision not to compress the spatial dimensions, feeding each 32×32×32 chunk straight into a 3D diffusion Transformer of roughly 280 million parameters.
That means giving up the efficiency gains. Even so, the official write-up says this choice was crucial for enabling fine-grained control over generation. Operability over speed.
A key design decision is not to compress the spatial dimensions of our data. Commonly, diffusion models operate on the compressed representations from models like Variational Autoencoders, which can massively improve the efficiency of diffusion models. We choose to train directly at the native block resolution, feeding each 32³ chunk of the world into a 3D Diffusion Transformer with ~280M parameters. While we miss out on the performance gains of compressed representations, we find this choice is crucial for enabling fine-grained control over generation. — From the passage on the decision not to compress spatial dimensions, its cost, and its purpose
Two diffusion formulations, compared
A Minecraft block is a categorical value drawn from a vocabulary of thousands, not a continuous number. So which kind of generative model suits it? The team compared two formulations on an identical Diffusion Transformer backbone.
The first is MD4, a discrete masked diffusion. A special [MASK] state is added to the vocabulary, and real blocks are progressively replaced with [MASK] to corrupt a chunk. Generation runs the process backwards, predicting the original block type at each [MASK] position and filling it back in.
The second is DDPM in embedding space. Each block type is mapped to a 16-dimensional vector by passing its in-game name (say, "stone") through a frozen text embedding model. Gaussian noise is added to corrupt, noise is removed to restore, and a nearest neighbor lookup converts the result back to a discrete block type.
Both handled the training task well, generating believable chunks matching a target biome. That alone does not settle which is better. What decided it was the property described next.
We implement two diffusion formulations: / MD4 discrete masked diffusion: We add a special [MASK] state to the vocabulary, and corrupt chunks by gradually replacing real blocks with [MASK] tokens. To generate new chunks, the model "unmasks" blocks by predicting the original block type to replace the [MASK] tokens. / DDPM in embedding space: We map each block type to a 16-dimensional vector obtained by a frozen text embedding model, using the block's in-game name (e.g., "stone"). Chunks are corrupted by gradually adding Gaussian noise, and we generate chunks by gradually removing noise back to clean embeddings. We perform a nearest neighbor lookup to recover discrete block types from denoised embedding vectors. / With our carefully curated data and after hyper-parameter tuning, we show that both models perform well on the training task, generating believable 32³ conditioned on a target biome, as seen below. — From the passages describing the two diffusion formulations and the finding that both performed well on the training task
Inpainting and outpainting come free
The reason discrete masked diffusion won is that the interactive generation workflows arrive as a byproduct of the training objective.
The mechanism is simple. Generation fills [MASK] positions one at a time, and each filled result conditions the rest of the process. Because of that, writing any pattern of blocks as the initial state turns it into a hard constraint the model must work around. You can also write during generation.
Since nothing is compressed, this reduces to "place a handful of Minecraft blocks in the shape you want." You can have part of a chunk taken from the game world regenerated, or hand-craft a pattern and let the model fill in the rest. In the demonstrations, even a suggestive pattern like a volcano — which does not occur naturally in Minecraft — extrapolates successfully.
Push the same property further and the 32×32×32 box stops being a limit. Shifting the generation window with overlap is enough to build worlds of any size.
This simple property gives us a fine-grained block-conditioning on top of class-conditioning as a free byproduct of training. / In practice, this means we can write any pattern of blocks as an initial state and even during the generation process, which becomes a hard constraint that the model must work around. Because we work at the native resolution using the in-game block vocabulary, this translates to simply placing a handful of Minecraft blocks in a desired pattern. / We can use a simple sliding window procedure to generate arbitrarily-sized worlds, generating overlapping chunks in sequence and stitching them together: — From the passages on block-conditioning arriving free from the training objective and on generating arbitrarily sized worlds
What the evaluation showed
The evaluation section rewards reading too. With no established metrics for 3D voxel worlds, the team started by validating the metric itself.
Data composition drove the results
First, an experiment on how much the makeup of the data matters. Three variants were built, each of one million chunks.
The three dataset variants compared (per the official release)
| Variant | Contents | Result |
|---|---|---|
| Balanced | Even distribution across the 15 biomes | The reference composition |
| Boosted | Village samples doubled | Large improvement in village FID |
| Natural | Collected at the game's natural spawn rate | Good on ocean (35%), drops on village (<1%) |
The naively collected Natural variant does well on the ocean biome, which is 35% of the data, but drops on the village biome that accounts for under 1%. Boosted, which doubles village samples, shows a significant improvement in FID. What you show a model more of is what it gets good at — an obvious relationship, now confirmed with numbers.
Patch sizes 2 and 4 for the DiT were compared as well. The patch 4 model drops sharply on the village biome, suggesting finer resolution matters for handling complex data.
To test how important data composition is, we create three variants of our dataset, each with 1 million chunks. The Balanced dataset represents even distribution between the 15 natural biome classes, while the Boosted dataset doubles the proportion of village samples. The Natural dataset represents the naive approach, collecting chunks at their natural spawn rate in the game world, leading to large imbalances between biome labels. / We can see a significant performance drop in our patch 4 models in the village biome, suggesting that the finer resolution at lower patch sizes is important for modeling this more complex data. Similarly, we see differences regarding dataset composition: the Natural dataset model performs well on the overrepresented ocean biome (35% of the dataset), and underperforms in the village biome (< 1% of the dataset). We find that doubling the number of village samples compared to the rest in the Boosted model, we see a significant improvement in FID. — From the passages on the three dataset variants, the effect of patch size, and the performance differences by dataset composition
Players picked the generated ones
This is the result most likely to get quoted. The samples look convincing at a glance, but how close are they to the real Minecraft world? The team handed the judgment to the experts on this 3D voxel world: Minecraft players.
The setup is simple. Show two sets of chunks for the same biome, and pick the better one. The comparison pool included two discrete models, one DDPM model, and real samples taken from the training data.
The result: across all three models, generated chunks were preferred over real ones. The margins vary, though. MD4 (patch 2) won 67.1% of 173 trials, MD4 (patch 4) 57.1% of 163, and DDPM (patch 2) 55.2% of 192 — two of the three are close to even.
Sakana AI does not take the result at face value either. It notes that this may be partly due to the data curation and generation process, and frames it carefully as evidence that the output is at least on par with the distribution being modeled. Not "AI beat humans," but "we can reproduce the distribution we aimed at."
We create an experiment where humans view two sets of chunks for the same biome, and pick the one they think is better. We include samples generated by two of our discrete models, our DDPM model, and real samples taken from our training data. / MD4 (patch 2) Real 67.1% 173 / MD4 (patch 4) Real 57.1% 163 / DDPM (patch 2) Real 55.2% 192 / Model vs. real. Win rate is the fraction of trials where participants preferred generated chunks from Source A over real Minecraft chunks from the same biome. / Across all three models, generated chunks were actually preferred to real ones. This may be partly due to our process of data curation and generation, but we view it as evidence that generated chunks are at least on par with the distribution we're trying to model. — From the passages on the design of the human comparison, the per-model win rates and trial counts, and the interpretation of the result
The metric itself was validated
The other careful touch is that the evaluation metric gets checked. With no established metric for 3D voxel worlds, FID (Fréchet Inception Distance) from image generation is adapted and applied to rendered images.
Whether that actually measures 3D chunk quality is not self-evident. So the human preference data was used to test whether lower-FID models win more often. The answer was a limited but significant relationship. In biomes where FID differences between models are small the relationship is weak; in more structured biomes where the differences are larger, metric and preference align well.
Lacking established metrics for 3D voxel worlds, we adapt Fréchet Inception Distance (FID) [4] to measure the differences between our models. / Using this data, we validate the render-based FID metric we use. If image-based FID is a good proxy for 3D chunk quality, we'd expect that lower-FID models would win more against higher FID models. / And we do find a limited but significant relationship between FID and win rate. In particular, we find that for biomes where there are small FID differences across models, this relationship is weaker. However, focusing on more structured biomes where the FID differences are larger, alignment of our metric with human judgment becomes stronger, with the lower-FID model winning 66.1% of the time when the gap is > 15 points. — From the passages on the absence of established 3D metrics, adapting FID to rendered images, and validating the metric against human preference
Human-authored maps trained on a reduced vocabulary
For training on human-authored maps, the thousands of block types present are compressed down to 171. Map creators use a much larger palette of blocks to capture small visual details, and narrowing the vocabulary simplifies the data so that the model can focus on the complex human structures instead. The write-up also acknowledges that learning to generate from data encoding human creativity brings new challenges and ethical considerations, and presents the results as a first step in that domain.
When training on human-authored maps, we treat each map as its own additional "biome", and compress the thousands of block types present in this data down to a smaller, more manageable vocabulary of 171 unique blocks. As map creators use a much larger palette of blocks to capture small visual details, compressing maps to a limited vocabulary results in data that is simpler, allowing our models to focus instead on capturing the complex and detailed human structures. We acknowledge that learning to generate from data that encodes human creativity comes with a new set of challenges and even ethical considerations, and we present our models and results as a first step in this domain. — From the passage on why the human-authored map vocabulary was compressed to 171 blocks and the ethical considerations involved
Where this sits in Sakana AI's research
Dream-Cubed is not a one-off headline; it extends a direction the company has been working in.
Carrying an existing framework onto a different subject
Taking a method established in language modeling and porting it wholesale onto a different unit — blocks — echoes the company's other work. The Picbreeder research on evolutionary search and CoffeeBench, which measures the practical competence of AI agents, share the same character: existing machinery aimed at a different question.
Some results become products, like the coding agent Fugu; others, like this one, ship as a dataset and code handed to the research community. On the infrastructure side, a partnership with NVIDIA has also been reported.
What "playable" means here
The point stressed repeatedly is that the output is immediately editable and playable. Not a 3D model to look at.
Because the models work with the in-game block vocabulary at native resolution without compression, the output is game content as-is. The decision to give up efficiency pays off right here. Placing a few blocks and letting the model fill in the rest is possible only because of that stack of choices.
We use our data to train a family of powerful transformers for efficient generation of interactive 3D environments at cube resolution. We show how our models allow players to mold the world around them by generating structures, terrain, and maps that are immediately editable and playable. — From the passage on the output being immediately editable and playable
Takeaway: the output arrives in a playable form
The headline from Dream-Cubed will be that players preferred the generated chunks over real ones. But that is a number the researchers themselves hedge — "partly due to our process of data curation and generation" — and reading it as "AI surpassed humans" goes too far.
What is doing the technical work is the quieter design decision: choosing not to compress. That is what makes it possible to inject a pattern of blocks mid-generation, and it is why inpainting and extension to arbitrary sizes arrived as byproducts of the training objective. A choice between efficiency and operability, resolved in favor of the latter, runs through the whole thing.
The other thing worth keeping is the data composition result. Collect at the natural rate and you lose the ability to build exactly the rare, complex things. That is not specific to Minecraft; it applies wherever data is gathered and trained on.
With data, models, and code all public, replication and adaptation are both open. Whether or not game development interests you, the work is worth a look for one reason: the same approach may apply to anything with discrete units. If you want to follow the details, converting the technical blog into a readable form makes the comparisons easier to hold side by side.
