When you design an mRNA therapeutic, billions of different sequences can encode exactly the same protein, and they don’t behave the same way. Stability, translation efficiency, expression in the right tissue and manufacturability: all of these and others depend on sequence choices you can’t test exhaustively at the bench.
That challenge is the focus of one of our research projects at Sanford Labs, where our AI/ML scientists work alongside wet-bench biologists to build an RNA design platform. It’s also why we care how well RNA foundation models actually capture regulatory sequence features.
In a new preprint, our team looked at something easy to overlook. To evaluate these models, you first have to collapse thousands of per-nucleotide embeddings into one vector per transcript, and the standard method averages across the entire transcript. That weights every region by its length. The 5′ UTR, which governs translation initiation, is only about 5–10% of the nucleotides, so its signal is diluted before any predictor sees it.
We tried pooling within the 5′ UTR, coding sequence, and 3′ UTR separately, then concatenating the three. Same frozen models, same forward pass, same linear probe. No retraining, no added parameters.
Across 20 models from 18 families, region-aware pooling improved translation and stability prediction by 11–25%, with no model significantly worse on any regression task. The improvement depends on using the real region boundaries, and the resulting probes respond to upstream AUGs and Kozak sequences in a way that the standard readout largely does not.
To be clear about scope: this is a change to how models are read out, not a new model, and it doesn’t help everywhere. The improvement concentrates where a regulatory signal sits in one short region, the upstream AUGs above being the clearest case. For signals spread across a long stretch of sequence, like codon usage across the coding sequence, the standard readout holds its own.
Preprint: https://doi.org/10.64898/2026.11.12.766253
Code: github.com/Sanford-Labs/rnaregionpooling
Work by Saiyang Na, Zhijie Qi, Deevanshu Goyal, Junzhou Huang (UT Arlington), Gene Yeo and Wenhao Jin.
If you’re building RNA foundation models or working on mRNA design, we’d be glad to compare notes — reach out to corresponding author Wenhao Jin at wjin@sanfordlabs.org.