California’s ambitious attempt to regulate large-scale artificial intelligence (AI) systems is advancing through the State Legislature, but only with significant modifications and intense debate within the tech industry.
Senate Bill 1047, introduced by State Sen. Scott Wiener, D-San Francisco, cleared the Assembly Appropriations Committee last week with substantial amendments. The bill aims to establish safety standards for robust AI systems when concerns about the potential risks of unchecked AI development are growing. Industry experts fear that the bill’s provisions could hamper innovation.
“Companies that drive AI innovation need a regulatory environment that balances safety with the ability to grow and develop cutting-edge solutions,” Daniel Christman, co-founder of the AI company Cranium, told PYMNTS. He warns that overregulation could “hinder the natural benefits of generative AI and stifle the innovative potential that these models offer.”
The initial draft of SB 1047 included potential perjury charges for noncompliance, a provision that drew sharp criticism from AI developers. The revised bill now relies solely on civil penalties.
“We can advance both innovation and safety; the two are not mutually exclusive,” Wiener said in a news release defending the bill’s evolution.
Dev Nag, CEO of software company QueryPal and former CTO at Wavefront, sees the changes to the bill as a step in the right direction. “Removing the perjury penalty will help reduce the disincentive for AI in California, but the penalty was unworkable in its initial form,” Nag told PYMNTS.
Another revision is the elimination of the proposed Frontier Model Division (FMD), a new regulatory body that would have overseen the bill’s implementation. Instead, the existing Government Operations Agency will absorb some of the FMD’s intended functions.
Nag expressed reservations about this change: “Removing the proposed new regulatory body might streamline the bureaucratic process somewhat, but analyzing AI models is not a skillset that the state Attorney General’s office has previously had to demonstrate or hire for.”
The bill’s legal framework has also shifted. Developers will now be held to a “reasonable care” standard, replacing the more stringent “reasonable assurance” requirement. This change aligns the bill more closely with established legal precedents.
In a nod to concerns from startups and smaller AI firms, the revised bill now includes a $10 million threshold. Only models fine-tuned at a cost exceeding this amount will fall under the bill’s purview, effectively exempting many smaller players in the AI field.
Enforcement mechanisms have also been narrowed. The state Attorney General’s office will now only be able to seek civil penalties in cases where harm has occurred or where there’s an imminent threat to public safety, limiting the scope for preemptive action.
The bill’s focus has drawn scrutiny from industry experts. Nag pointed out a potential misalignment in priorities: “The legislation itself seems more interested in far-off speculative risks vs. the risks that are here today, such as misinformation and deepfakes.”
This concern echoes a broader debate within the AI community. A recent survey found that 70% of AI researchers believe safety should be prioritized in AI research, while 73% expressed “substantial” or “extreme” concern that AI could fall into the hands of dangerous groups.
Despite these reservations, the bill has garnered support from some heavyweight figures in the AI community. Geoffrey Hinton, often referred to as one of the “Godfathers of AI,” has voiced his approval, stating that “SB 1047 takes a very sensible approach to balance those concerns.”
The global race to develop and regulate AI has put California’s longstanding tech leadership under the microscope. Industry experts warn that overly stringent regulations could jeopardize the state’s competitive edge.
“Maintaining a competitive balance with other states and countries is crucial for making sure that California remains the hub of AI development, especially as other regions relax their regulatory environment,” Nag emphasized. He pointed to California’s historical leadership in various tech sectors, including hardware, software, biotech and aerospace, attributing this success to a more innovation-friendly environment.
The tech industry is watching closely as the bill heads to the Assembly floor for a vote on Tuesday (Aug. 20), with a deadline to pass by Aug. 31. With federal legislation on AI regulation stalled, California’s efforts could set a precedent for other states and shape the future of AI development nationwide.
“GenAI is a very multipurpose tool, much like a hammer,” Christman said. “The state is effectively trying to regulate hammer manufacturers, rather than those using a hammer for malicious activities.”
For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.
We’re always on the lookout for opportunities to partner with innovators and disruptors.
Learn More