California Clamps Down On Lawyers Using AI

While generative artificial intelligence (“AI”) tools such as ChatGPT, Claude, and Gemini may seem helpful, they are not a substitute for qualified legal advice from a licensed lawyer. Using AI tools in place of a lawyer or during litigation results in significant harm, including waiver of the attorney-client privilege and false statements of the law. Judges across the country have made clear that:

• An AI model cannot act as or substitute for legal counsel.
• An attorney-client privileged communication requires an actual lawyer-client relationship.
• Documents prepared using generative AI are not protected by attorney-client privilege or the work-product doctrine.
• Self-initiated AI notes and strategies created without a lawyer’s direction do not qualify as protected work product
• Forwarding or sharing AI-generated outputs with a lawyer after they are created does not cloak unprivileged documents in the attorney-client privilege.
• Everything sent to standard AI chatbots can be subpoenaed or viewed by the other side during litigation (the head of ChatGPT has said this himself).
• Attorneys worldwide still use AI even though it often “hallucinates” by making stuff up and citing to non-existent law and court decisions.

Despite these known pitfalls, California lawyers and their clients still use AI in their cases. California state law is now clamping down on this.

In late September 2026, California passed Senate Bill 574 adding and amending existing law.¹ The new law is effective on January 1, 2027, when it will make California the first state to regulate lawyers’ use of generative AI.

California lawyers are still allowed to use generative AI but they must adhere to strict rules.

No Delegating The Practice of Law To AI.

• “An attorney shall not delegate the practice of law to generative artificial intelligence….”²
• Similarly, “[a]n arbitrator shall not delegate any part of their decision-making process to any generative artificial intelligence tool” nor shall they “rely on information generated by generative artificial intelligence outside the record without making appropriate disclosures to the parties beforehand and, as far as practical, allowing the parties to comment on its use.”³

No Confidential Information To AI.

• Lawyers using generative AI are restricted from inputting confidential or identifying information into publicly available or non-enterprise AI systems.⁴
• Personal identifying information includes: driver’s license numbers; dates of birth; Social Security numbers; National Crime Information and Criminal Identification and Information numbers; addresses and phone numbers of parties, victims, witnesses, and court personnel; medical or psychiatric information; financial information; account numbers; and anything sealed by order or confidential by law.⁵

Lawyers Must Verify That AI Is Not Hallucinating.

• Lawyers must take “reasonable steps” to verify the accuracy of generative AI outputs, “including, but not limited to, the accuracy of all case and statutory citations” and to “correct any erroneous or hallucinated output in any material used by the attorney.”⁶
• No “brief, pleading, motion, or any other paper filed in any court shall contain any citations that an attorney responsible for submitting the pleading has not personally verified, including any citation provided by generative artificial intelligence.”⁷

Disclosing AI Use To Court.

• Lawyers must “[d]isclose the use of generative artificial intelligence to the court for all documents submitted to the court and consider whether to disclose” where generative AI is used to create content provided to the public.⁸ More simply, attorneys must tell judges and arbitrators when they submit material generated by AI.

 
 
 
 
1 California Senate Bill 574 adds Business & Professions Code § 6068.1, amends Business & Professions Code § 6173, amends Code of Civil Procedure § 128.7, and adds Code of Civil Procedure §§ 180 and 1282.1.
2 Business & Professions Code § 6068.1(a).
3 Code of Civil Procedure § 1282.1(b)
4 Business & Professions Code § 6068.1(a)(3)(A)
5 Business & Professions Code § 6068.1(b)
6 Business & Professions Code § 6068.1(a)(3)(B)
7 Code of Civil Procedure § 128.7(b)(2)(A)
8 Business & Professions Code § 6068.1(a)(3)(C)