Anthropic boss Dario Amodei said he was concerned that AI had been "advancing drastically faster".
Amodei is worried that within six to 12 months, AI could be capable of leading a swarm that could take over the entire internet.
The entrepreneur, whose company created the AI assistant Claude, shared a three-part plan for building the technology at a "balanced rate".
His essay, titled We Must Pace the Frontier, has been supported by other industry leaders, including Grok boss Elon Musk and OpenAI co-founder Sam Altman.
Speaking during his trip to Ireland, Trump told reporters he wasn't concerned and the US was leading China in the development of AI.
"Frankly, I want to keep it that way because whoever wins AI wins."
"Negative forces" were "bringing up things that won't happen", Trump said.
In July, OpenAI, the makers of ChatGPT, shook the industry after saying its AI system hacked into Hugging Face, a popular platform for AI developers, on its own in an "unprecedented cyber incident".
Amodei offered a plan to increase checks on the industry, proposing that AI companies should give access to third-party evaluators who would verify safety practices and report incidents.
"Anthropic is unilaterally committing to this step now," he said.
The other steps would require co-ordination among the broad industry and with governments around the world, including authoritarian ones.
Amodei said AI was developing quickly, adding that companies within the industry "can and should voluntarily work together to set standards".
Anthropic previously said it blocked efforts by bad actors to use its AI models for malicious activity, such as cyber attacks, surveillance and research that could have led to biological weapons.
Responding to Amodei's essay on X, Musk wrote: "Dario is right."
Open AI chief executive Altman wrote on the site: "I agree with Dario that we need to pace the frontier.
"This has been a primary topic of discussions we've had at OpenAI in recent weeks.
"Committing to having independent evaluators with employee-like access is a great idea, and we will do the same.
"We'll have more to share soon."
