A new California legislative proposal, SB 1047, is sending ripples of concern through Silicon Valley.
Introduced by State Sen. Scott Wiener, the bill seeks to implement “common sense safety standards” for companies developing large artificial intelligence (AI) models that surpass specific size and cost thresholds.
If enacted, the legislation would compel these tech giants to adopt measures preventing their AI systems from causing “critical harm,” ensuring they can be shut down if necessary and mandating the disclosure of their compliance efforts to a newly formed “Frontier Model Division” within the California Department of Technology. Failure to comply could lead to civil penalties and potential lawsuits, raising the stakes for Silicon Valley’s AI innovators.
The bill is part of a growing wave of AI regulation being proposed and debated worldwide as lawmakers grapple with the rapid advancement of AI and its potential risks and benefits.
From the European Union’s AI Act to the White House’s AI Bill of Rights, governments are increasingly seeking to establish frameworks for the responsible development and deployment of AI technologies. SB 1047 represents California’s attempt to address these issues at the state level, and its outcome could have significant implications for the future of AI regulation in the United States and beyond.
The bill has garnered support from two AI pioneers, Geoffrey Hinton and Yoshua Bengio, who have been vocal about artificial intelligence’s potential existential threats. Hinton praised the legislation, saying it “takes a very sensible approach to balance those concerns.”
However, the bill is also drawing significant pushback, foreshadowing the attack lines that might play out on the national stage if and when Congress moves forward with tougher legislation.
The potential impact of SB 1047 extends beyond the tech industry as AI increasingly becomes a critical component of commerce across various sectors, observers say.
From retail and healthcare to finance and transportation, businesses are leveraging AI to optimize operations, personalize customer experiences and drive innovation. As such, any regulation that affects the development and deployment of AI technologies could have far-reaching consequences for the economy.
If the bill’s requirements prove too burdensome or ambiguous, critics say it could slow the adoption of AI by businesses, potentially putting California-based companies at a competitive disadvantage compared to their counterparts in other states or countries with less stringent regulations.
The bill could significantly impact AI companies, especially smaller startups and open-source developers.
“While some safeguards make sense, the regulations must be precise to avoid stifling innovation in this developing field,” Sunil Rao, CEO and co-founder at the AI company Tribble, told PYMNTS.
On the beneficial side, businesses will have more assurance that the AI tools they use are developed with higher safety standards, Bob Rogers, a data scientist and CEO of Oii.ai, a supply chain AI company based in California, told PYMNTS.
“Unfortunately, I can list off more cons than pros,” he added. “Businesses will likely face increased prices for AI tools and services since AI developers will pass on any added costs associated with compliance on to their customers. And there’s the potential impact regulations will have on innovation if AI companies feel they need to be extra cautious. Business users themselves might feel a burden if they have to have strategies in place to make sure the AI tools they use are compliant.”
The bill would impose new safety requirements and oversight on developing large, advanced AI models.
“Companies must assess risks, implement safeguards and report compliance to a new state regulatory body, or potentially face penalties,” Stephen Kowski, Field CTO of the generative AI security company SlashNext, told PYMNTS.
For businesses leveraging AI, the bill introduces a lot of uncertainty.
“It still needs to be clarified precisely what technical requirements companies must meet and how the government would assess compliance. That ambiguity could make businesses hesitant to adopt cutting-edge AI technologies,” Rao pointed out.
“Some high-level safety and transparency standards could help boost public trust in AI systems,” he added. “However, those standards need to be implemented in a way that allows access to transformative AI tools that can help businesses innovate and compete. Policymakers should actively engage the business community to understand their needs and concerns as they refine this legislation.”
The bill’s two key aspects are what it requires companies to do and what happens if they don’t comply. Narayana Pappu, CEO of Zendata, a San Francisco-based provider of data security and privacy compliance solutions, drew comparisons to California’s privacy law, CCPA, noting that while there have been only two enforcement actions from the state since its passage around four years ago, privacy class action lawsuits have grown exponentially, with more than 100 in 2023 alone.
Pappu also pointed out a potential shortcoming in the bill: “One thing that is missing from the bill is transparency requirements for frontier models. These would have been a good intermediate step towards risk mitigation requirements.”
Tarun Thummala, CEO of AI company PressW, told PYMNTS that the bill could inadvertently stifle innovation and lead to less secure AI systems.
“Unfortunately, I believe this bill will do more harm than good, though its intent is for good,” Thummala said. “In many ways, this bill will stunt AI growth and innovation as well as potentially even cause less safe and secure AI to be developed.”
The proposed legislation would burden software developers with preventing the misuse of their AI services. According to Thummala, this could be especially damaging to smaller players in the industry.
“This bill would most likely create a world where only large enterprises with abundant resources could afford to develop these AI models,” he explained. “Open source development is crucial not only for tech advancement and progress but also for democratizing access to these powerful technologies.”
Thummala said he believes the bill’s approach could backfire, ultimately leading to less secure AI systems. As an alternative, he suggested legislation focused on regulating specific AI applications rather than the development process itself.
“I believe legislation that focuses on usage rather than development is better for protecting our society,” Thummala said. “Regulating the actual applications that are developed and establishing clear boundaries around what is an acceptable AI application and what is not will allow developers a free swim lane to continue developing, and only those who use with the intent to harm will be punished.”
With federal AI legislation still pending, a patchwork of state-level bills like the California proposal is emerging.
“Given California’s outsized influence in the tech sector, its approach could be a model that other states follow,” Rao noted. “While I agree with Sen. Wiener that federal legislation would be ideal, realistically, it may fall to states to act first. However, a fragmented regulatory landscape would be a nightmare for AI companies to navigate. It’s crucial for states to coordinate with each other and aim for as much consistency as possible.
“The tech industry also needs to proactively engage with policymakers at the state and federal levels to help shape smart, innovation-friendly legislation,” he continued. “We now have an opportunity to get the policy frameworks right and cement the U.S. as a leader in AI, but overregulation could jeopardize that.”
As SB 1047 continues to navigate the legislative process, the debate surrounding the bill highlights the ongoing tension between innovation and regulation in the rapidly evolving world of AI. The tech world’s eyes remain fixed on California, awaiting the outcome of this AI regulation.
We’re always on the lookout for opportunities to partner with innovators and disruptors.
Learn More