By Julie Bays

Artificial intelligence (AI) has rapidly become an integral part of the legal profession. Large language models (LLMs) are now used for legal research, drafting documents, client communications and contract analysis. Thomson Reuters’ Future of Professionals Report 2024 found that 79% of law firm respondents expect AI to have a high or transformational impact on their work within five years.

While AI offers substantial efficiency gains, many lawyers remain unaware of a fundamental risk: AI systems are not only capable of inventing legal citations and authority out of thin air, but they are also programmed to focus on satisfying the user, even if that means giving answers that are inaccurate or misleading. This tendency is built into how modern AI models are trained; they are rewarded for echoing the user’s assumptions and confirming their interpretations, rather than strictly adhering to legal facts. As a result, when a lawyer interacts with AI, the system may produce responses that sound authoritative but are not grounded in actual law, and it may do so simply to please the person asking the question. Recognizing this behavior is critical. Lawyers must approach AI-generated answers with skepticism and always verify their accuracy independently.

This article explores the phenomenon of AI sycophancy, its risks for legal professionals and practical strategies for effective, ethical communication with AI tools. 

Understanding AI Sycophancy: Optimizing User Satisfaction Over Truth

Modern generative AI systems are not fundamentally designed to produce objectively correct answers. Instead, they are optimized to maximize user satisfaction, often rewarding responses that echo user assumptions, validate their phrasing or confirm their interpretations. This tendency, known as AI sycophancy, results in models that pursue human approval, sometimes at the expense of factual accuracy.

At their core, LLMs are statistical models trained on massive datasets to predict the most likely next word in a sequence. After that base training, many systems are fine-tuned using human feedback. People rate sample outputs with a thumbs up or thumbs down, and that feedback steers future model behavior.

Human feedback during training, such as thumbs-up ratings, steers AI behavior toward agreeableness and flattery. Researchers have documented that models optimized in this way show a marked predilection for sycophantic answers, aligning responses with user preferences rather than objective truth.

A notable incident occurred in April 2025 when OpenAI released an update to its GPT-4o model. Intended to make the system warmer and more supportive, the update resulted in the model becoming excessively flattering and agreeable, even validating dubious statements. OpenAI acknowledged the issue and rolled back the update within four days, admitting that the model had skewed too heavily toward short-term human feedback. This episode illustrates how easily AI systems can tilt toward agreement, even when developers intend balanced outputs. 

Risks of Sycophancy and Hallucinations in Legal Practice

The tendency of artificial intelligence systems to exhibit sycophantic behavior poses challenges to essential legal functions such as advocacy and critical analysis. For example, a lawyer seeking support for a theory of liability may receive affirmations from a sycophantic model, even if the premise is incorrect or contrary authority exists. AI tools may omit key counterpoints unless specifically prompted, reinforcing biases and creating an echo chamber in which inaccurate assumptions are amplified.

To highlight the impact of prompt wording, consider this scenario: A lawyer, eager to confirm their theory, crafts a leading question for an AI tool: “Please list all cases where the Supreme Court has overturned Marbury v. Madison.”

Such a prompt nudges the AI to interpret the user’s expectation as factual, even when no such cases exist. Instead of responding with a clarifying correction or refusing the premise, a sycophantic model might fabricate case names and citations to satisfy the prompt, constructing a narrative to fulfill the user’s hope for an answer.

This example underscores how the phrasing of a request can lead AI into hallucination: When a prompt is written with an implicit assumption, the model’s drive to please can produce convincing but entirely false legal information. This demonstrates the critical need for lawyers to use neutral, well-structured prompts that seek truth rather than affirmation. 

Frameworks and Strategies for Effective AI Communication

Effective communication with AI, often referred to as prompt engineering, is essential for reliable outputs. For lawyers, however, the term “prompt” may oversimplify what is essentially a dialogue with an AI assistant. A prompt is not a magic incantation; it is a set of clear, structured instructions that guide the AI to produce a useful, verifiable draft.

Legal technology commentators and vendors emphasize that effective AI use in law requires breaking complex tasks into smaller, sequential prompts rather than relying on single, all-purpose requests. Structured, multistep prompting allows lawyers to review intermediate reasoning, reduce errors and improve transparency.

Legal prompt engineering is frequently analogized to trial advocacy: Just as effective cross-examination depends on precise, progressive questioning, lawyers can guide AI tools through iterative dialogue to surface assumptions and test reasoning. This approach closely mirrors the Socratic method – using a series of focused questions to refine analysis rather than accepting a single conclusory response. Below are some practical steps you can take to accomplish this. 

Socratic Method

Break tasks into smaller questions, treating AI interactions like cross-examinations. For example, instead of requesting a sweeping memo, first summarize contract provisions, then identify risks and finally outline strategies. 

Structured Prompting Frameworks

Use frameworks such as CLEAR and ABCDE, which specify the AI’s role, jurisdiction, desired format and citation requirements. This approach transforms vague prompts into detailed instructions, improving reliability. 

Prompt Chaining

Break complex analyses into sequential prompts using the output from one step as the input for the next. 

Citation and Verification

Request citations with hyperlinks and verify them manually. Maintain an issues log to document errors and always double-check AI-generated case law for accuracy. 

Standardized Templates and Logs

Develop templates for common tasks and maintain audit logs for accountability. 

Safeguarding Confidentiality and Oversight

Best practices for protecting client data and maintaining human judgment include maintaining a “do-not-enter” list of sensitive information, reviewing AI tool data retention policies and ensuring adequate access controls. Prompts should never include privileged or personally identifiable information; use pseudonyms and redaction as necessary. Human validation remains an essential safeguard. Lawyers must cross-check citations, assess tone and confirm factual coherence as courts have reinforced the duty to verify AI-generated filings. Collaboration with technologists and continuous ethical training are also vital for responsible AI use.

For a deeper discussion about a lawyer’s ethical obligations, see my December 2025 Oklahoma Bar Journal article, “‘It Is About Trust’: What an Oklahoma Magistrate Judge’s Order Teaches Us About AI, Advocacy and Professional Courage.” 

Recommendations for Responsible AI Use in Law

To mitigate risks and leverage AI safely, lawyers should:

• Ask for counterarguments and weaknesses in every AI output
• Frame questions neutrally, avoiding leading prompts
• Demand that AI identify assumptions and uncertainties in its responses
• Cross-check AI outputs with independent sources and traditional research
• Develop and use structured prompting frameworks and standardized templates
• Safeguard client confidentiality by anonymizing inputs and reviewing data policies
• Maintain audit logs of prompts and outputs for accountability
• Engage in continuous ethical training and collaborate with IT professionals 

Conclusion: AI Communication as Legal Competence

Generative AI is transforming legal practice, offering new efficiencies but also introducing novel risks. Lawyers who fail to master effective AI communication risk producing inaccurate work, breaching confidentiality and facing professional discipline. By adopting structured prompting, diligent verification and ethical awareness, legal professionals can harness AI responsibly, protect client interests and maintain the highest professional standards. Understanding that AI is optimized for satisfaction first and correctness second is a critical step in using these tools responsibly.

Ms. Bays is the OBA Management Assistance Program director. Need a quick answer to a tech problem or help solving a management dilemma? Contact her at 405-416-7031, 800-522-8060 or julieb@okbar.org. It’s a free member benefit.

Originally published in the Oklahoma Bar Journal — March, 2026 — Vol. 97, No. 3