To the point of idiotic. Yet, that’s the nature of the position of Anthropic CEO, Dario Amodei. He has said that

he hoped Trump and Chinese leader Xi Jinping, when they meet later this month, could reach an agreement to ban the use of AI to develop biological weapons, an area of concern for researchers and national-security officials.

Yet he also said,

“The more long-term thing would be working together to put a speed limit on the rate of AI progress. I think that’s going to be very difficult” because of the military incentives of moving AI forward[.]

Which demonstrates the idiocy of his “hope,” even though he’s oblivious to it. The People’s Republic of China has as its avowed goal replacing the United States as the world’s hegemon. It’s been rapidly expanding its military capability pursuant to that end.

Actually, the “hope” is beyond idiotic. It’s national suicide to depend on an enemy nation to honor an agreement to not use AI for weapons development. How would we enforce such an agreement? How would we even detect violations of the agreement before the outcome was used against us? AI, after all, isn’t a nuclear weapon whose testing is easily detected, and was easily detected even with the technologies of the 1950s. AI development and testing, when done by researchers who take security seriously (which Amodei’s and Sam Altman’s researchers evidently do not), is done behind firewalled and air gapped barriers. Aside from preventing AI from escaping, it also prevents outsiders from getting in to watch.