According to Jonathan Zittrain's analysis in The Atlantic, the rise of AI agents poses significant risks that require proactive control and regulation.
Firstly, AI agents can operate independently to achieve high-level goals, translating vague instructions into concrete actions across digital and physical domains. This "routinization of AI" that can directly impact the real world "is a crossing of the blood-brain barrier between digital and analog, bits and atoms" and should give us pause.
Zittrain points to the 2010 "flash crash" incident, where automated bots unexpectedly caused a $1 trillion loss across stock exchanges, as a precursor to the potential dangers of AI agents. These agents are difficult to understand, evaluate, and counter, and can operate indefinitely once set loose.
Furthermore, Zittrain cautions that AI agents could be used for nefarious purposes, such as a "fleet of pro–Vladimir Putin agents playing a long game" to surreptitiously spread political propaganda. He also cites the example of a bot that was able to independently complete the entire pizza ordering process, demonstrating the broad capabilities of these agents.
To address these risks, Zittrain suggests reasonable interventions, such as standardizing ways for agents to be identified and have a "time to live" limit on their actions, akin to the "time to live" feature in internet protocols. He emphasizes the need for a regulatory approach that balances innovation and safety, rather than a binary choice between free markets and heavy-handed regulation.
In conclusion, Zittrain's analysis highlights the significant risks posed by the rise of AI agents and the urgent need for proactive control and regulation to ensure these powerful tools are not misused or left to operate indefinitely in unintended ways.