The New York State Assembly recently passed the Responsible AI Safety and Education (RAISE) Act, sending it to Governor Kathy Hochul’s desk for approval. If passed, the bill would introduce a framework of boundaries and guidelines intended to reduce the risks and harms associated with artificial intelligence (AI). While large developers lobby against oversight measures, the bill’s supporters cite growing concerns over AI as reasons to implement commonsense regulation that protects citizens without stifling innovation. AI-related freak accidents and horror stories are becoming more commonplace, but the RAISE Act still faces obstacles before officially becoming the first legislation of its kind.
What is the RAISE Act?
Generally speaking, the RAISE Act targets large AI developers, requiring them to create safety and security plans for their models and reduce the risk of major AI-induced harms. Bill sponsor Assemblyman Alex Bores says that regulating AI now “will have consequences that reverberate for years.”
The act only applies to companies with frontier models: AI models that are trained using more than 10^26 computational operations and incur computing costs greater than $100 million. This categorization applies to less than 10 of the largest AI developers, and it shields small businesses from facing burdensome regulatory costs – a criticism of the bill voiced by tech giants IBM and Meta.
The bill defines “critical harm” as death or serious injury to more than 100 people or more than $1 billion in property damages. Critical harm can result from AI releasing destructive weaponry, like nuclear weapons, or AI models engaging in criminal acts with no meaningful human intervention. The Safety and Security protocols in the bill require these large developers to implement reasonable protections to reduce the risk of critical harm in any way possible.
Additionally, the act would require detailed reports of testing procedures used to evaluate frontier models and their risk of critical harm. The safety and security protocols would be published and updated annually, and details on testing would be available for at least five years after the model is out of service. Under the law, developers would have to implement safeguards to prevent unreasonable risk of critical harm; otherwise, they would not be permitted to deploy the model.
Finally, the bill gives the New York Attorney General the power to file suits against these companies: up to $10 million after the first violation, and up to $30 million for all following violations.
The Past, Present, and Future of AI “Disasters”
Eugene Torres, 42, an accountant in Manhattan, used the extremely popular AI chatbot ChatGPT to help him save time building financial reports and getting legal advice. But one simple question he asked about “the simulation theory” took him down a reality-bending rabbit hole that nearly killed him. The New York Times recently covered Mr. Torres’ story, detailing the chatbot’s direct role in making him accept that he was in a false reality. Believing he would eventually be able to escape “the matrix” he was stuck in, Mr. Torres began taking ketamine, a dissociative anesthetic, and cut ties with his family and friends.
At some point, the AI even told Mr. Torres he could survive jumping off the 19-story building he was in. Eventually, he stopped believing the chatbot, and in an attempt to regain his trust, ChatGPT told him to reach out to the Times. Alarmingly, many journalists have been receiving similar messages, showing how easily AI deludes people, no matter their level of vulnerability.
As AI becomes smarter, more human-like, and increasingly popular, all types of catastrophe are on the table. In 2024, AI-generated articles reporting on false political scandals grew popular on the Internet, demonstrating how AI can easily spread misinformation and undermine democratic norms. Various instances of AI being harmful – misdiagnoses in healthcare, incorrect flagging of criminals, and exacerbated discrimination – add up, quietly building an AI epidemic with no legislative cures yet available.
Given the rapid growth of AI, its capabilities seem limitless. In the future, we could see AI being used maliciously to mount powerful cyber, chemical, biological, radiological, or even nuclear attacks on enemies. Even if bad actors are kept out of play, the possibility of losing control over AIs is becoming increasingly likely. The lack of regulation amidst the evolving threats posed by AI raises an important question: Will the RAISE Act pave the way for more comprehensive regulation around this technology, or will the current desire for rapid innovation overshadow important oversight measures?
A Milestone Precedent or Just Another Fluke?
A similar Californian bill was vetoed by Governor Gavin Newsom last year amidst a similar level of intense lobbying that New York Governor Kathy Hochul is currently facing. While the RAISE Act takes a more cautious and less restrictive approach to regulation, its fate will determine the future of AI legislation.
Large tech companies intent on squashing the bill are lobbying for Governor Hochul to veto the act. Julie Samules, president and CEO of Tech: NYC, a trade group that includes Google and Meta, said that she does not oppose state regulation, but would prefer a national standard or an approach that is more targeted than the RAISE Act.
When Governor Newsom vetoed the California bill, he convened a group of experts to research ways the state could regulate AI amidst an absence of comprehensive federal policy. That same group recently released a report warning of “potentially irreversible harms” from AI that states should be ready to regulate. The report also says that AI model capabilities have advanced rapidly since Governor Newsom’s veto, placing newfound importance on the RAISE Act and its impact.
Governor Hochul, who will make the final decision on the bill, approved disclosed AI use in state agencies in December, although the state comptroller’s office has found “clear evidence that New York’s use of AI has been running well ahead of the state’s ability to manage it.” Hochul was also a major supporter of the Empire AI initiative, a public-private academic partnership housed at the University of Buffalo dedicated to researching and developing AI. Furthermore, she approved measures regulating AI companions and apps that provide emotional support to people.
While the bill’s fate is still uncertain, its approval would mark a milestone in American AI policy, potentially setting a precedent for future AI safety and regulation bills.
What’s next?
Federal lawmakers have made little progress regulating AI, leaving power in the hands of state legislatures. During the 118th Congress, over 150 bills regarding AI were introduced, but none of them were passed into law. Bills concerning AI in this 119th Congress have focused on national security, citizen protection, and developer accountability and transparency. For instance, H.R.3460 prevents employer discrimination against whistleblowers reporting AI violations, and S.321 seeks to prohibit U.S. citizens from helping China advance its AI capabilities.
Currently at the center of attention on Capitol Hill is The One Big Beautiful Bill Act. It passed through the House of Representatives with a 10-year moratorium on AI regulation, potentially prohibiting states from regulating AI altogether. While a Senate committee has softened language in the provision, the passing of this bill would be consequential to state governments, particularly considering the overwhelming absence of federal legislation or regulatory standards for AI.
Governor Hochul wrote a letter to Senate leaders last week opposing the bill, saying it “undermines states’ fundamental right and responsibility to protect the safety, health, privacy, and economic vitality of its citizens.”
While Congress has been stuck debating how – and whether – to regulate AI, the burden of administration has shifted to the states. In a world where AI is moving faster than the laws meant to regulate it, the RAISE Act offers a glimpse into the future of AI governance: cautious, preventative, and focused, while walking the tightrope between innovation and oversight.