This specific study sounds relatively benign, but generative models for DNA sequences sounds like an incredibly dangerous technology to develop. If someone were to reproduce their study while including animal viruses in the training data, what would it be capable of? And what about after a few years of progress?
Their phages have only 5400 base pairs, but the hepatitis virus has around 3000, and ebola only had ~19,000. That doesn't mean making a novel lethal virus is easy, but if anyone pulls it off, it would make Covid look like a picnic.
We have 5 known antibiotics. There are "MRSA" bacteria (which is a property of bacterial species, not a name for a particular species (anymore)) which don't care or react to about 4 of them and the 5th one is showing slightly reduced effectiveness already.
Why? Presumably because, of course, the new government rules imposed on doctors for antibiotics are about saving money and NOT about fighting resistance. The 5th antibiotic is ... not expensive in an absolute sense, but compared to the other 4 it's pretty expensive. The guideline from scientists is that if you decide to use it you need to nearly overdose the patient on it, despite that that is not necessary to cure them. Doing that is much more expensive (not that much more expensive really, it just looks impressive in % and presumably in excel sheets), but fights resistance. We're not doing that, and so we have ... probably not even a decade ... and then we have no treatment anymore against bacterial infection.
And in a decade, we will still be surrounded by lethal bacteria. Look at your hand. It's overwhelmingly likely you can't see them but you're looking at lethal pneumococci, several species. This is a clean hand we're taking about. Even a freshly scrubbed hand of a surgeon will still have quite a few bacteria. Grab a handful of soil. You know, the kind plants grow in. Your hand now has botulinum-producing bacteria, as well as many other toxins, tuberculosis (definitely if you're anywhere near a farm), tetanus, and the list goes on and on), some of which we have no vaccine against at all, and many we don't vaccinate for because it's not practical.
So the choice is between developing DNA sequences for viruses, or having a pandemic 100x worse than COVID every 30 years or so.
You should hear about the war that occurs in the ocean non stop. Quadrillions of bacteria, bacteriaphages, and viruses are in a constant battle and evolving to one up each other.
Really I don't get how multicellular life survived it.
I asked a family member biologist about concerns over open weight models helping to create bioweapons:
I can imagine a situation where a bio weapon pathogen could potentially be engineered to evade the immune system by creating a pathogen that is both extremely virulent and has a rapid mutation rate. Rapid mutation rates make it difficult to create an effective vaccine and also difficult for the immune system to learn to recognize the pathogen.
But whether open weight AI models can allow people to create such bioweapons - it does seem far-fetched to me. I think it's far-fetched because biological systems contain so many elements that we don't even know about, do not exist in any data set, and so they could not be included in the model.
In order for a pathogen to be successful, there are a number of environmental barriers that it has to endure in order to reach the host, survive in the host, proliferate in the host, and damage the host. Environmental barriers outside the host and inside the host.
I know that AI can be used to design genes to encode synthetic proteins. So could an open weight AI model result in the creation of genes that encode harmful proteins and then bioengineer pathogen to produce the proteins. But there are so many unknowns that can affect whether this pathogen can survive at all. So I would say from my limited knowledge, I would say it's far-fetched.
As noted in this paper⁽¹⁾, ”Publicly releasing model weights is an act of irreversible proliferation. Once weights are released, they can never be recalled.”
Do you really want to risk it on your off-the-cuff reasoning that it’s “far-fetched”?
I have been diving into / reading up this area lately. A few quick notes.
Worth specifying that these were hand selected candidates. Out of these human selected ones, 5% were successful.
The Stanford researchers picked the most promising 302 AI designs and synthesised them in the lab. Of these, 16 proved effective at killing E. coli bacteria.
"picked the most promising 302 AI designs" out of a n that isn't mentioned in this article.
Based on my quick reading, it feels like the BBC left out a ton of context that makes this work feel more novel than it is. There's a ton of prior art here (including some from the same team)
For example, older iterations of the same model family were used to generate novel genes and proteins. Including anti-CRISPR proteins that were tested against phages. That work is honestly more interesting.
Anti-CRISPRs (Acrs) are proteins used by phages to neutralize bacterial CRISPR–Cas systems (Fig. 3a). Many Acrs represent striking examples of rapid protein evolution, appearing as novel innovations without detectable similarity to other protein families [..]
[..] After filtering for size, complexity and structure (Extended Data Fig. 5b), we next used PaCRISPR, a machine learning model trained to identify potential Acr proteins, to evaluate our generated candidates [..]
[..] we further identified five proteins (EvoAcr1–5) that demonstrated strong protection against SpCas9 cleavage in both liquid culture survival assays (Fig. 3f and Extended Data Fig. 5d) and phage infection experiments (Fig. 3g and Extended Data Fig. 5f), while maintaining normal host growth
Of note, semantic design is not limited to Evo 1.5 and can leverage any language model trained on prokaryotic or phage genomes. Improvements in genomic language models, as well as a better understanding of prokaryotic gene synteny, should therefore directly translate to improvements in semantic design.
Also the same project has been covered before.
Prior pre-publication / release from 2025 called "AI-Designed Phages," the piece is really well written and I recommend reading it, https://www.asimov.press/p/ai-phages
edit - as flobosg notes one of them describes the model,
The first synthetic organism was Venter's mycoplasma laboratorium / Synthia in 2010.
The first organism to be synthetically assembled (not designed, but assembled ikea style) was a phage in 2003 also by Venter & co.
In 2021 another team made entities that can "find and combine building blocks into self-copies," Kinematic self-replication in reconfigurable organisms
https://pmc.ncbi.nlm.nih.gov/articles/PMC8670470/
> Based on my quick reading, the BBC article doesn't bring up a prior publication / release from 2025 called "AI-Designed Phages," https://www.asimov.press/p/ai-phages
That link is a commentary about the preprint of the article mentioned in the BBC article.
All those publications describe the methodology and modelling process to generate genomes, without testing them experimentally. This article shows that their designed genomes actually work in the wetlab. Your last two links are both preprints.
That's a fair point! I think I've found a few more things, but reading through / verifying first. Will update accordingly and leave notes. Thanks for helping!
edit - I updated to delete my over claim. Thank you!
This specific study sounds relatively benign, but generative models for DNA sequences sounds like an incredibly dangerous technology to develop. If someone were to reproduce their study while including animal viruses in the training data, what would it be capable of? And what about after a few years of progress?
Their phages have only 5400 base pairs, but the hepatitis virus has around 3000, and ebola only had ~19,000. That doesn't mean making a novel lethal virus is easy, but if anyone pulls it off, it would make Covid look like a picnic.
We have 5 known antibiotics. There are "MRSA" bacteria (which is a property of bacterial species, not a name for a particular species (anymore)) which don't care or react to about 4 of them and the 5th one is showing slightly reduced effectiveness already.
Why? Presumably because, of course, the new government rules imposed on doctors for antibiotics are about saving money and NOT about fighting resistance. The 5th antibiotic is ... not expensive in an absolute sense, but compared to the other 4 it's pretty expensive. The guideline from scientists is that if you decide to use it you need to nearly overdose the patient on it, despite that that is not necessary to cure them. Doing that is much more expensive (not that much more expensive really, it just looks impressive in % and presumably in excel sheets), but fights resistance. We're not doing that, and so we have ... probably not even a decade ... and then we have no treatment anymore against bacterial infection.
And in a decade, we will still be surrounded by lethal bacteria. Look at your hand. It's overwhelmingly likely you can't see them but you're looking at lethal pneumococci, several species. This is a clean hand we're taking about. Even a freshly scrubbed hand of a surgeon will still have quite a few bacteria. Grab a handful of soil. You know, the kind plants grow in. Your hand now has botulinum-producing bacteria, as well as many other toxins, tuberculosis (definitely if you're anywhere near a farm), tetanus, and the list goes on and on), some of which we have no vaccine against at all, and many we don't vaccinate for because it's not practical.
So the choice is between developing DNA sequences for viruses, or having a pandemic 100x worse than COVID every 30 years or so.
There is no choice here.
Jesus, how are the other living things existing at all?
You should hear about the war that occurs in the ocean non stop. Quadrillions of bacteria, bacteriaphages, and viruses are in a constant battle and evolving to one up each other.
Really I don't get how multicellular life survived it.
Why go to the ocean, because that happens everywhere, non-stop. If your immune system stops working, then you are a dead man.
Earlier: https://news.ycombinator.com/item?id=49200213
I asked a family member biologist about concerns over open weight models helping to create bioweapons:
I can imagine a situation where a bio weapon pathogen could potentially be engineered to evade the immune system by creating a pathogen that is both extremely virulent and has a rapid mutation rate. Rapid mutation rates make it difficult to create an effective vaccine and also difficult for the immune system to learn to recognize the pathogen.
But whether open weight AI models can allow people to create such bioweapons - it does seem far-fetched to me. I think it's far-fetched because biological systems contain so many elements that we don't even know about, do not exist in any data set, and so they could not be included in the model.
In order for a pathogen to be successful, there are a number of environmental barriers that it has to endure in order to reach the host, survive in the host, proliferate in the host, and damage the host. Environmental barriers outside the host and inside the host.
I know that AI can be used to design genes to encode synthetic proteins. So could an open weight AI model result in the creation of genes that encode harmful proteins and then bioengineer pathogen to produce the proteins. But there are so many unknowns that can affect whether this pathogen can survive at all. So I would say from my limited knowledge, I would say it's far-fetched.
You hope it’s “far-fetched”.
As noted in this paper⁽¹⁾, ”Publicly releasing model weights is an act of irreversible proliferation. Once weights are released, they can never be recalled.”
Do you really want to risk it on your off-the-cuff reasoning that it’s “far-fetched”?
⁽¹⁾ https://www.lesswrong.com/posts/qmQFHCgCyEEjuy5a7/lora-fine-...
I'm sure you have thought the same thing about CRISPR and cloud labs in the past. But this is real and will probably doom us all.
I have been diving into / reading up this area lately. A few quick notes.
Worth specifying that these were hand selected candidates. Out of these human selected ones, 5% were successful.
"picked the most promising 302 AI designs" out of a n that isn't mentioned in this article.Based on my quick reading, it feels like the BBC left out a ton of context that makes this work feel more novel than it is. There's a ton of prior art here (including some from the same team)
For example, older iterations of the same model family were used to generate novel genes and proteins. Including anti-CRISPR proteins that were tested against phages. That work is honestly more interesting.
From, "Semantic design of functional de novo genes from a genomic language model," https://www.nature.com/articles/s41586-025-09749-7
Also the same project has been covered before.Prior pre-publication / release from 2025 called "AI-Designed Phages," the piece is really well written and I recommend reading it, https://www.asimov.press/p/ai-phages
edit - as flobosg notes one of them describes the model,
"Genome modelling and design across all domains of life with Evo 2," March 2026, https://www.nature.com/articles/s41586-026-10176-5
pre-print of the asimov press piece, "Generative design of novel bacteriophages with genome language models," September 2025, https://www.biorxiv.org/content/10.1101/2025.09.12.675911v1
All research comes from somewhere. There's an entire sub-field / world here that goes back decades at this point.
Leon Chan refactored a bacteriophage by hand in 2005 https://link.springer.com/article/10.1038/msb4100025
The first synthetic organism was Venter's mycoplasma laboratorium / Synthia in 2010.
The first organism to be synthetically assembled (not designed, but assembled ikea style) was a phage in 2003 also by Venter & co.
In 2021 another team made entities that can "find and combine building blocks into self-copies," Kinematic self-replication in reconfigurable organisms https://pmc.ncbi.nlm.nih.gov/articles/PMC8670470/
From 2019 / 2020, algorithmically designed synthetic life / "xenobots", https://www.nytimes.com/2020/04/03/science/xenobots-robots-f... https://en.wikipedia.org/wiki/Xenobot
Or, "Team builds first living robots—that can reproduce" https://wyss.harvard.edu/news/team-builds-first-living-robot... , 2021
Zurich in 2019, https://web.archive.org/web/20191008040536/https://www.eurek...
And if you expand computing to include molecular processes for genes, then there's stuff like PACE that uses phages as a process, https://en.wikipedia.org/wiki/Phage-assisted_continuous_evol...
(final edit - thanks flobosg!)
> Based on my quick reading, the BBC article doesn't bring up a prior publication / release from 2025 called "AI-Designed Phages," https://www.asimov.press/p/ai-phages
That link is a commentary about the preprint of the article mentioned in the BBC article.
After your editing:
> Same team, prior publication / release from 2025 called "AI-Designed Phages," https://www.asimov.press/p/ai-phages
It is the same work, just not published in Science.
Apologies was editing the comment, but I looked into it more and it wasn't a preprint. They published it. Multiple times.
"Genome modelling and design across all domains of life with Evo 2," March 2026, https://www.nature.com/articles/s41586-026-10176-5
However, this was a preprint, "Generative design of novel bacteriophages with genome language models," September 2025, https://www.biorxiv.org/content/10.1101/2025.09.12.675911v1
Even older conference paper, I think, February 19th, 2025, https://arcinstitute.org/manuscripts/Evo2
All those publications describe the methodology and modelling process to generate genomes, without testing them experimentally. This article shows that their designed genomes actually work in the wetlab. Your last two links are both preprints.
That's a fair point! I think I've found a few more things, but reading through / verifying first. Will update accordingly and leave notes. Thanks for helping!
edit - I updated to delete my over claim. Thank you!
The publication: https://www.science.org/doi/10.1126/science.aec2657
It is paywalled, but this seems to be the corresponding preprint: https://www.biorxiv.org/content/10.1101/2025.09.12.675911v1....
Anyone else assume this was about computer viruses and exploits...
A-choo Intelligence!
Now it just sticks to using the old ones?