Transformers in numerical simulation: what a community survey and an industrial benchmark tell us

Three figures from the joint NAFEMS and Teratec survey on AI applied to numerical simulation, launched on June 10, 2026 during the NAFEMS and Teratec event "AI and Numerical Simulation" in Paris. All shares below are of the organizations that responded.

• 85% of responding organizations have begun testing or deploying AI in their simulation workflows.
• 26.2% of responding organizations report being operationally satisfied with what those initiatives have delivered.
• 5% of the engineers surveyed have experimented with transformer architectures in their simulation workflows.

If your organization has a pilot that shows promise but has never touched a real design decision, it has plenty of company. The third figure helps explain the first two. Over the same five years in which CAE teams were testing convolutional networks, graph networks and physics-informed models, transformers became the default architecture in nearly every other domain of applied AI. Simulation is close to the last field where they remain a curiosity.

Is that a blind spot, or justified caution? Transformers arrived late to simulation for real technical reasons. What follows lays out what survey respondents said is blocking them, what a transformer-based simulation model actually offers on those specific points, and what happened when the question was put to an open industrial benchmark.

Asked what is blocking their AI initiatives, respondents produced a clear hierarchy. The barrier at the top of the list is worth considering, because it is not compute, and it is not explainability. It is data. High-fidelity results are expensive to produce, rarely archived in a reusable form, and never available in the volumes conventional machine learning assumes.

Every industrial team that has scoped a surrogate-model project knows the resulting dead end. If training the model requires two hundred high-fidelity runs, and two hundred runs would already answer the design question, the model is pointless.

Where AI would matter most, if it worked

• Hybrid models combining field measurements with simulated data
• Topology optimization
• Fast surrogate models
• Real-time simulation

All four require a model that generalizes across geometries and operating conditions from limited data. In other words, the applications the community wants most are the ones the data barrier blocks first. The claims that follow are about the architecture family, not any single implementation, and most are documented in the open literature [1, 2].

From solving to inferring

Every AI surrogate, whatever its architecture, makes the same basic trade. A conventional solver decomposes the design into a computational mesh, often tens of millions of cells, and iterates a coupled system with at least that many unknowns until it converges, often thousands of cycles on an HPC allocation.

A trained surrogate instead takes the mesh and boundary conditions as input and infers the converged field in one forward pass, in seconds, because the mapping from problem to solution was learned offline. The speed-up does not come from solving faster, it comes from not solving at all.

After a decade of research the open question is no longer whether this works, it is which architecture survives contact with industrial scale.

Where the earlier architectures stopped

The 26.2% satisfaction figure has an architectural history behind it, and practitioners across organizations tell it in remarkably similar terms.

• Physics-informed networks embed the governing equations in the training loss. They remain difficult to train and hard to scale beyond simple geometries, and few industrial teams still consider them a durable route.
• Convolutional networks need structured Cartesian grids, which industrial meshes are not.
• Graph networks solved the geometry problem by operating on mesh connectivity and delivered useful results on components, but message passing becomes prohibitively expensive at full-assembly scale and information propagates only as far as the network is deep.
• Neural operators learn mappings between function spaces with elegant theory behind them, but published results repeatedly show fragility outside the training distribution and an appetite for data that industrial practice cannot feed.

Each family performs well on academic cases of a few thousand points and then hits a wall of scalability, data hunger, or both. An 85% experimentation rate combined with a 26.2% satisfaction rate is roughly what one would expect a community to report after running into those walls at scale.

Attention, tokens, and why scale is the point

A transformer represents its input as a sequence of tokens and updates each token by weighing its relationship to every other token through self-attention. For simulation, two consequences of that design matter.

Long-range coupling is native. In language, attention is what connects a pronoun to a name forty words back. In a physical field, the same mechanism lets a disturbance at the inlet inform the prediction at the outlet directly, without the information hopping through local neighborhoods the way it must in a graph network. Global coupling is the defining feature of elliptic and mixed PDE systems, and here it is a property of the architecture rather than something approximated by stacking depth.

Transformers scale, and scale is where the others failed. They are the only architecture with a sustained record of absorbing large heterogeneous datasets and improving as they grow. A model pre-trained across a broad corpus of physical simulations has already seen how fields behave near walls, how boundary layers develop, and how stress concentrates at features. Adapting it to a new industrial case becomes a fine-tuning problem of tens of cases rather than hundreds. The expensive data requirement moves out of each project and into a pre-training effort amortized across all of them.

The catch, and how the field is resolving it

There is a genuine reason nobody simply used a transformer on a mesh years ago. Self-attention is quadratic in the number of tokens, and standard transformers were built for 1D text sequences or regular 2D image grids. An unstructured industrial mesh is neither, and treating each of ten million mesh points as a token is computationally out of the question.

Several groups, working independently, arrived at the same insight: a token has to represent more than a single mesh point. The fix is essentially domain decomposition. The mesh is partitioned into a far smaller number of regions, each region is condensed into one token carrying its geometry and local fields, and attention operates on regions rather than raw points. Transolver, developed at Tsinghua University, groups points by physical state rather than spatial position and computes attention over those groups in linear time [1]. Other published designs make different choices at each step [2] and land in the same place.

The quadratic cost is gone. Mesh sizes that disqualified AI methods two years ago are becoming routine. Physics is governed by consistent laws, so these models do not need to memorize the ambiguity of human language: published models in this family run from a few million to a few tens of millions of parameters. The training bill looks like a workstation invoice rather than a data-center one, and it is small next to the HPC allocation the model then displaces.

• 100M+ cells per industrial CFD case, on a single GPU in a matter of days
• 1000x smaller than a large language model

Set the community's reported barriers against the architectural properties above.

• Lack of usable data (51%, the #1 barrier). Fine-tuning a pre-trained model needs tens of cases per new application, not hundreds. Cheap low-fidelity runs become useful training signal.
• Production deployment at industrial scale (49%). Linear-cost tokenization handles meshes of tens to hundreds of millions of cells, versus the roughly 100k ceiling of earlier methods.
• Result validation concerns (47%). Open benchmarks with held-out geometry variations. Error measured against reference solvers. Uncertainty quantification increasingly standard.
• Multi-physics, globally coupled problems. Long-range dependencies carried by attention rather than approximated by network depth.
• One bespoke model per problem. Generalization across geometries and operating conditions. Each fine-tuned application starts from shared pre-trained physics rather than from zero.

On the validation row

Trust, not speed, is the objection engineers actually raise. The sensible standard is not perfection, it is parity. A surrogate whose error against physical reality falls within the error distribution of the reference solver is a legitimate tool for design exploration, while final certification stays where it belongs, with validated solvers and physical testing. AI simulation earns its place upstream of that safety net, not instead of it.

On the last row

Most of today's applications of AI to simulation focus on single-purpose surrogates: one component, one physics, one model trained from scratch on data that cost more than the model gives back. A model built this way is often obsolete by the time it is ready, and the next project starts again from zero. It is not surprising that only a quarter of teams call the result satisfying. What disappointed them is not the accuracy of the predictions, it is the economics of building models one at a time.

Architecture arguments are eventually settled by measurement. Safran Tech has assembled and published an open benchmark suite for AI-based simulation, built on the PLAID datasets: industrial-grade cases spanning structural mechanics and turbomachinery aerodynamics.

Each case defines a training set, a held-out test set that includes geometry variations, and published error metrics against reference high-fidelity solutions.

The suite is hosted on Hugging Face, where anyone can consult the standings or submit a model [3]. To date it has received submissions from industrial and academic teams covering roughly ten distinct model families, including physics-informed networks, graph networks, neural operators and several independent transformer variants. It is the community's architecture debate, run as a controlled experiment on shared industrial data.

Three patterns stand out in the results:

• Pattern 01. Transformer-based submissions hold the top positions across the benchmark cases. Not one model but several, from independent teams with different tokenization strategies. The pattern points to the architecture family, not to any single group's engineering.
• Pattern 02. The margin is material. On the published metrics, the leading transformer submissions reduce prediction error significantly compared to the best non-transformer entries.
• Pattern 03. The gap widens on the geometry-variation test cases: exactly the regime that matters for design exploration, and exactly where the older architectures' out-of-distribution fragility shows up.

It was not assembled by an AI vendor selecting favorable ground. It was created by the simulation specialists of a major aerospace OEM, on cases drawn from their own engineering practice, because they wanted a rigorous answer to the question of where to place their own investment.

An organization with fifty years of simulation heritage chose to build open evaluation infrastructure for these methods, measured the alternatives on its own problems, and is directing its investment accordingly. For readers weighing the same decision with fewer resources, that is a more useful signal than anything a supplier could put in front of them.

None of this settles the question, and real issues remain open. How do these models fit into certified V&V processes? What does deployment look like under the sovereignty and data-governance constraints that dominate defense and energy? How does on-premise training compare with hyperscaler-hosted foundation models? Where does the transformer approach hit its own out-of-distribution limits?

These are the questions the 95% should be asking rather than waiting out, and they will get answered the same way the scalability question was: by measurement in the open.