California Enacts Legislation to Regulate Use of Generative AI in Healthcare

California Governor Gavin Newsom signed Assembly Bill 3030 (AB 3030) into law, marking a significant step in regulating the use of generative artificial intelligence (GenAI) in healthcare settings. This new law, known as the Artificial Intelligence in Health Care Services Bill, will come into effect on January 1, 2025, and is designed to enhance patient transparency while addressing potential risks associated with AI in clinical communications.

AB 3030 aims to regulate how healthcare providers, including hospitals, clinics, and private medical practices, use AI to generate patient communications related to clinical information. Per the new law, AI-generated communications—whether written, verbal, or visual—must include a clear disclaimer informing patients that the content was created by AI. Additionally, these communications must provide clear instructions for patients on how to contact a human healthcare provider for further information or clarification.

The bill is part of a broader push by California lawmakers to mitigate the risks associated with GenAI technologies, particularly as AI systems become more integrated into healthcare practices. According to a statement from the California legislature, AB 3030 is designed to ensure that patients are fully aware when AI is used in their care and that they are given clear paths to seek human interaction should they need it.

While the law introduces new safeguards, it also clarifies that AI-generated communications reviewed and approved by licensed healthcare professionals are exempt from the disclosure requirements. This exception was supported by various medical associations, which expressed concerns that overly restrictive rules could hinder the use of AI to streamline clinical processes, such as documentation, which are often time-consuming for healthcare providers.

The law does not apply to AI-generated communications regarding administrative matters such as appointment scheduling or billing. Its focus is squarely on patient clinical information, where errors can have more serious consequences. This means that AI tools can still be used for non-clinical tasks without the stringent disclosure requirements.

Read more: UnitedHealth and Amedisys Prepare for Key Antitrust Meeting with DOJ

Defining GenAI as “artificial intelligence that can generate derived synthetic content,” the law specifically targets AI systems that create original content, such as large language models (LLMs) that produce written text. By focusing on synthetic content, the law aims to address the unique risks associated with AI-generated material, which can sometimes introduce inaccuracies or biases into clinical communication.

AB 3030 also introduces accountability measures for healthcare providers who violate the law. Physicians found in violation of the new regulations will be subject to oversight by the Medical Board of California or the Osteopathic Medical Board of California. Health facilities and clinics could face enforcement under California’s Health and Safety Code.

As the law is set to take effect, California regulators have emphasized the need to balance the benefits and risks of AI in healthcare. According to a statement from the California Senate, AI tools can help reduce the administrative burden on healthcare workers, offering more time for patient care. However, there are concerns about the potential for AI to introduce errors, such as “hallucinations,” where the AI generates plausible but false information, and biases stemming from training on incomplete or historically inaccurate data.

While AB 3030 does not directly regulate the accuracy of AI-generated clinical content, it seeks to provide transparency by ensuring patients are informed about the use of AI in their care. This is in line with broader efforts at the federal level, such as the White House’s Blueprint for an AI Bill of Rights, which emphasizes the right to know when automated systems are being used in a way that impacts individuals’ lives.

With California at the forefront of AI regulation in healthcare, healthcare providers across the state are now tasked with adapting to these new requirements. Experts suggest that medical facilities should start preparing for the implementation of AB 3030 by updating their communication systems, ensuring that AI-generated content is appropriately flagged, and reinforcing their oversight processes to maintain the quality of care. This proactive approach will help mitigate the risks of relying on AI while ensuring compliance with California’s stringent new rules.

Source: Nat Law Review