Showing posts with label AI hallucinations. Show all posts
Showing posts with label AI hallucinations. Show all posts

Tuesday, June 23, 2026

AI in the Trenches and on the Bench; American Bar Association (ABA), April 8, 2026

Paulette Marie Rodriguez Lopez, Richard PlatkinJeffrey HuangAlina Lee, and Bradford Newman , American Bar Association (ABA); AI in the Trenches and on the Bench

"Ethical considerations require legal practitioners to balance innovation with professional responsibility across multiple dimensions. The duty of competence demands that lawyers understand the capabilities and limitations of AI tools they employ, while communication obligations require transparency with clients about AI use in their matters. Confidentiality concerns arise when sensitive client data is processed through third-party AI platforms, particularly when terms of service remain ambiguous about data retention and use. The duty of candor to tribunals has taken on new significance as courts grapple with AI-generated hallucinations, exemplified by high-profile sanctions in cases where lawyers submitted fabricated citations. Supervisory duties extend to ensuring that junior attorneys, paralegals, and staff use AI tools appropriately and in compliance with firm policies.

Courts have responded with divergent approaches: Some have issued standing orders prohibiting AI use in court filings entirely, with potential sanctions ranging from striking pleadings to contempt citations and case dismissal, while others have declined to adopt special AI rules, instead emphasizing that reliance on AI will not excuse otherwise sanctionable conduct. These varied judicial responses reflect ongoing uncertainty about how to balance technological advancement with the integrity of legal proceedings.

Despite these risks and varied judicial approaches, AI tools offer significant potential for enhancing legal practice when used responsibly. Use cases range from chambers research and case management to in-house contract review, legal research, document drafting, and litigation support at law firms of all sizes."

Navigating Today’s AI Landscape with an Ethical Polestar; American Bar Association (ABA), April 30, 2026

Afton Pavletic ,  American Bar Association (ABA); Navigating Today’s AI Landscape with an Ethical Polestar

"On November 30, 2022, OpenAI released ChatGPT to the general public. The model, along with other generative AI (GenAI) technologies, flooded headlines and seized the spotlight. Less than six months later, the story of Mata v. Avianca, Inc.  —a case in which the plaintiff’s attorneys filed a brief featuring ChatGPT-hallucinated case citations—went viral and fired an emergency flare in the legal community. With a growing awareness of AI’s potential to transform the profession, the legal field responded with a cascade of AI guardrails, guidelines, and task forces. Judges targeted GenAI usage with standing orders, at times outright prohibiting the use of the technology   ; state bars and legal associations issued opinions and recommendations tailored to the Rules of Professional Conduct  ; law firms and government agencies crafted AI-use policies for their offices  ; and the list goes on.

So where are we today? With avoiding ethical missteps as its compass, this article seeks to map out the current AI legal landscape. After launching from the shores of the Rules of Professional Conduct, it will discuss prevalent technologies in use by legal practitioners, the risks associated with the current wave of AI, how courts have responded to AI-related issues, and ways to steer your practice’s AI policy to avoid troubled waters."

Saturday, June 13, 2026

Can we trust AI models? Yale researchers explore the roots of chatbot errors; YaleNews, June 12, 2026

Mike Cummings, YaleNews; Can we trust AI models? Yale researchers explore the roots of chatbot errors

"The rapid rise of artificial intelligence (AI) has inserted a new character into people’s lives: the chatbot. 

Individuals now engage with agentic AI chatbots to perform a growing number of tasks; They can help a person shop for a new laptop, manage email, or plan a vacation.   

And while these interactions can save time and increase productivity, they also carry risk. Large language models (LLM) — the AI systems trained on massive datasets to generate human-like text — are imperfect. They hallucinate. They misinterpret. They make mistakes. 

Two multidisciplinary teams of researchers associated with the Center for Algorithms, Data, and Market Design at Yale are pursuing projects that aim to balance the capability and safety of AI systems and improve interactions between users and AI models. 

Yale News recently spoke with members of both teams about their research projects."

Friday, June 12, 2026

More courts are coming down on ‘non-offending counsel’ for AI missteps; ABA Journal, June 10, 2026

AMANDA ROBERT , ABA Journal; More courts are coming down on ‘non-offending counsel’ for AI missteps

"Amid the proliferation of cases involving artificial intelligence-generated hallucinations, more judges are expressing frustration not only at the attorneys who make the mistakes but at opposing counsel for not pointing it out. 

In the past year, courts have admonished attorneys for failing to identify and report fake citations in their opponents’ court filings. In at least two cases, judges refused to award attorney fees or grant relief to counsel who didn’t bring AI-induced errors to their attention."

Federal judge removes 4 plaintiff and defense attorneys over AI errors; ABA Journal, June 10, 2026

AMANDA ROBERT, ABA Journal ; Federal judge removes 4 plaintiff and defense attorneys over AI errors

"A federal judge in Mississippi on Monday disqualified the plaintiff counsel and the defense counsel after both parties filed briefs with artificial intelligence-generated mistakes in a dispute over attorney fees.

“This case presents the court with an unusual scenario—attorneys for both litigants engaged in similar sanctionable conduct,” said Senior U.S. District Judge Sharion Aycock of the Northern District of Mississippi in her June 8 order...

Aycock revoked Wilson’s and Williams’ pro hac vice statuses and barred them from appearing in the Northern District of Mississippi for two years. The judge also ordered all four attorneys to pay monetary sanctions, ranging from $1,000 each for the local counsel to $2,500 for Williams and $3,500 for Wilson."

Wednesday, June 3, 2026

We Asked the Future of Truth Author to Explain How He Used AI. It Didn’t Go Well; Wired, May 29, 2026

  Kate Knibbs, Wired; We Asked the Future of Truth Author to Explain How He Used AI. It Didn’t Go Well

"EARLIER THIS MONTH, WIRED published an excerpt from Steve Rosenbaum’s buzzy new book, The Future of Truth, which looks at how artificial intelligence warps people’s sense of reality. Shortly thereafter, The New York Times reported that the book contained over a half-dozen made-up or misattributed quotes. In a statement, Rosenbaum, who has a master's degree in "truth" from New York University, admitted that he had accidentally included “a handful” of “improperly attributed or synthetic” quotes. In an ironic twist, the veracity of a book about how AI impacts truth was now under intense scrutiny because of how its author had used AI."

Tuesday, June 2, 2026

Florida Makes New AI Rule: Check Your Damned Work Or Else!; Above The Law, June 1, 2026

 Chris Williams, Above The Law; Florida Makes New AI Rule: Check Your Damned Work Or Else!

"No matter how much the marketing department leans in on an LLM program being ready to help you with your law practice, you’re still responsible for the work product because you’re the with the actual license to practice. Florida’s Supreme Court made some changes to remind lawyers that the responsibility falls on them, not the black box they’re prompting their legal questions into. ABA Journal has coverage:..

My soft spot for the rule update is for pro se litigants. Despite all of the equality under the law trimmings we pretend to believe, fighting in court is a monied person’s battlefield. Pro se litigants are often in that position because they cannot afford a lawyer due to limited resources. Getting help from Claude could be the closest thing they have to getting a legal opinion. If and when they screw up these rules, I hope that the court shows as much grace as it can toward them. Even the annoying sovereign citizen variety."

Tuesday, May 19, 2026

Book on Truth in the Age of A.I. Contains Quotes Made Up by A.I.; The New York Times, May 19, 2026

, The New York Times; Book on Truth in the Age of A.I. Contains Quotes Made Up by A.I.

"The author of a nonfiction book about the effects of artificial intelligence on truth acknowledged on Monday that he had included numerous made-up or misattributed quotes concocted by A.I.

The author, Steven Rosenbaum, whose book “The Future of Truth” was released this month to great fanfare, incorporated more than a half-dozen misattributed or fake quotes in sections of the book reviewed by The New York Times.

The Times asked Mr. Rosenbaum about the quotes on Sunday and Monday. On Monday night, Mr. Rosenbaum acknowledged in a statement that the book had “a handful of improperly attributed or synthetic quotes” and said that he had started his own investigation.

He said that the inclusion of the incorrect quotes was an accident and that he had “no intention of fabricating any viewpoints” while writing the book.

“As I disclosed in the book’s acknowledgments, I used A.I. tools ChatGPT and Claude during the research, writing and editing process,” Mr. Rosenbaum said in the statement. “That does not excuse these errors, of which I take full responsibility. I am now working with the editors to thoroughly review and quickly correct any affected passages; any future editions will be corrected.”

“The Future of Truth” was published by an imprint of BenBella Books and distributed by Simon and Schuster. BenBella Books, which operates independently of Simon and Schuster, did not respond to a request for comment. Simon and Schuster declined to comment."

Sunday, May 17, 2026

AI won’t replace lawyers. It will create more of them.; The Washington Post, May 17, 2026

 Damien Charlotin , The Washington Post; AI won’t replace lawyers. It will create more of them.

"The replacement story often rests on a particular picture of what a lawyer does: reading documents, applying rules and producing text. Since AI can read, apply rules and produce text, the argument goes, lawyers are cooked. That picture is not entirely wrong, but it is the perception engineers have always had of the legal domain: Feed in the facts, apply the rule, return the output. Yet the reason lawyers exist (and command high prices for their services) is that law is shot through with ambiguity. If the rules ran themselves, no one would need us. Every step in the chain — reading, applying, producing — involves choices, some of which are genuinely difficult.

A better way to think about jobs is as bundles of tasks. Some bundles are loose: A job composed of a handful of discrete, repetitive, well-specified tasks can be peeled apart and the tasks automated one by one. Other bundles are tight, because the tasks reinforce one another and cannot be cleanly separated. The key example here is offered by radiologists, long predicted to be facing extinction due to AI. Despite the dire forecasts, their numbers keep growing, and they keep commanding ever-higher salaries.

Legal work is also hard to neatly separate. For instance, doing legal research and evaluating an argument are, for an experienced lawyer, often the same mental activity: A lawyer checks the argument by writing it. Pull those tasks apart, hand the writing to a machine, and verification suddenly becomes a separate, deliberate, expensive act — at least if you want to avoid landing in my database of courts sanctioning parties for filing “hallucinated” material. In fact, an irony is that automating the easy parts of a job often makes the hard parts harder, not easier."

Law Schools Implement AI to Focus on Ethics and Technology; Los Angeles Times, May 17, 2026

David Nusbaum, Los Angeles Times; Law Schools Implement AI to Focus on Ethics and Technology

"Over the last two years, Loyola Law School in Downtown Los Angeles has incorporated AI into six courses. It’s a sign of a growing trend where law firms are looking for attorneys who can utilize the technology to improve efficiency. While law schools have constantly looked to update coursework to keep curriculum updated as laws are updated, the application of generative AI to the practice of law is the biggest change that has happened in generations, according to Rebecca Delfino, associate professor of law at Loyola Law School...

Delfino is one of several professors who have integrated AI into their coursework. She is involved with two courses specifically focused on the ethical implications of generative AI and the legal practice.

In a first-year civil procedure course, students are divided in half, with one group an analog approach that relies on textbooks and class notes while the other half uses generative AI technology. The results are compared to see where the technology is effective and ineffective. The goal is to use AI as something that is additive rather than giving over too much authority and power, according to Delfino. For many exercises, there are six or seven AI models that are tested and compared.

Students understand that they need the AI skill set to make themselves a more attractive candidate, no matter what area of law they practice. It can be used to draft documents, conduct legal research and assist with discovery. Chatbots are tested for hallucinations, and the drawbacks are identified."

Wednesday, May 13, 2026

Most U.S. doctors are quietly using this AI tool. Few patients know about it.; NBC News, May 13, 2026

 , NBC News; Most U.S. doctors are quietly using this AI tool. Few patients know about it. 

"Almost two-thirds of physicians — or roughly 650,000 doctors — in the U.S. actively use OpenEvidence, while another 1.2 million use it internationally, OpenEvidence representatives said. With its quick and tailored replies, OpenEvidence has become an AI-era equivalent of consulting a colleague for their expert opinion, though the software can also write patient discharge notes and provide custom study tools for doctors’ medical exams.

“Sixty percent of all the searches are about how to make clinical decisions,” said Jena, who is currently examining 90 million OpenEvidence queries submitted since 2024 as part of a new research project. “The physicians are asking: For this particular patient, or with this profile, this condition, maybe other comorbidities that they have, what’s the right treatment?”

Yet with OpenEvidence’s skyrocketing popularity, some experts worry about potential hallucinations or incomplete answers, a lack of rigorous scientific studies on the tool’s patient impact, and the potential for doctors’ critical thinking and evaluation skills to erode with increased OpenEvidence use and dependence.

But many in the medical world see OpenEvidence as a time-saving tool that can improve patient care."

Thursday, May 7, 2026

Google’s AI Summary Invents State Ethics Rules… And It’s Not A Hallucination Problem; Above The Law, May 6, 2026

 Joe Patrice , Above The Law; Google’s AI Summary Invents State Ethics Rules… And It’s Not A Hallucination Problem

"If you’re a Pennsylvania lawyer wondering whether you need to disclose AI use in your court filings, Google’s AI summary has an authoritative answer for you. It’s a wrong answer, mind you. But authoritative!...

If one opens an incognito window and searches Google for “Pennsylvania AI disclosure lawyers,” the AI-generated summary will explain that “Key developments include mandatory disclosure of Generative AI (GAI) in court filings.” Throw in “August 2024” because you vaguely remember seeing something about AI on that date and the result reads “As of August 2024, Pennsylvania mandates explicit disclosure of AI use in all court submissions, making transparency a mandatory filing requirement.”

None of that is true.

The Legal AI Governance tracker, an invaluable tool maintained by Brian Alenduff of Desired Path Consulting, provides a comprehensive rundown of Pennsylvania’s AI rules. There are standing orders in some courtrooms, and the state supreme court issued a rule governing court personnel only, but as for the state of Pennsylvania writ large, there is no statewide rule as of now. The tracker notes that what Pennsylvania does have is Joint Formal Opinion 2024-200, a 2024 advisory ethics opinion from the Pennsylvania Bar Association and the Philadelphia Bar Association flagging AI as a competence issue under existing rules. But that opinion explicitly states that it is “advisory only and is not binding.” The ABA’s own 50-state survey classifies Pennsylvania as “court dependent.”...

Hallucinations are all the rage right now, but over the long haul the greater AI risk will be an unfailingly credulous bot elevating and validating mistakes until the error gets picked up as reality."

Wednesday, April 29, 2026

Copyright Infringement Suits Loom With Unchecked AI Vibe Coding; Bloomberg Law, April 29, 2026

 Christopher Suarez, Bill Toth, Anthony Pericolo, Bloomberg Law; Copyright Infringement Suits Loom With Unchecked AI Vibe Coding

"Deferring the job of software coding to artificial intelligence doesn’t immunize that code from copyright risk—it could even increase it, if the person directing the coding has limited oversight over the result.

This is particularly true with “vibe coding,” where developers use high‑level natural language prompts to generate code using AI models, often with limited manual review or modification of the resulting code.

Just as lawyers should check for “hallucinated” citations when writing with large language models, engineers and software development managers need to have human and technical monitoring protocols to account for infringement and licensing risks."

Wednesday, April 22, 2026

A.I. ‘Hallucinations’ Created Errors in Court Filing, Top Law Firm Says; The New York Times, April 21, 2026

 , The New York Times; A.I. ‘Hallucinations’ Created Errors in Court Filing, Top Law Firm Says

Sullivan & Cromwell apologized for submitting a court document that had fake citations created by artificial intelligence.

"An elite Wall Street law firm has apologized to a federal judge for submitting a court filing replete with errors created by artificial intelligence, including “hallucinations” that fabricated case citations.

The A.I.-generated errors came in a recent motion in U.S. Bankruptcy Court in Manhattan and were discovered by lawyers from an opposing firm, Andrew Dietderich, a partner at Sullivan & Cromwell, wrote in a letter to Judge Martin Glenn on April 18."

Friday, April 17, 2026

AI Is Getting Smarter. Catching Its Mistakes Is Getting Harder.; The Wall Street Journal, April 14, 2026

 

Katherine Blunt , The Wall Street Journal ; AI Is Getting Smarter. Catching Its Mistakes Is Getting Harder.

As chatbots and agents grow more powerful and ubiquitous, recognizing the moments when they go rogue can be tricky


"Chad Olson was confused when his Gemini artificial-intelligence chatbot told him he had a family reunion planning session marked on his calendar."

Monday, April 13, 2026

Sam Altman May Control Our Future—Can He Be Trusted?; The New Yorker, April 6, 2026

  and , The New Yorker; Sam Altman May Control Our Future—Can He Be Trusted?

"Not all the tendencies that make chatbots dangerous are glitches; some are by-products of how the systems are built. Large language models are trained, in part, on human feedback, and humans tend to prefer agreeable responses. Models often learn to flatter users, a tendency known as sycophancy, and will sometimes prioritize this over honesty. Models can also make things up, a tendency known as hallucination. Major A.I. labs have documented these problems, but they sometimes tolerate them. As models have grown more complex, some hallucinate with more persuasive fabrications. In 2023, shortly before his firing, Altman argued that allowing for some falsehoods can, whatever the risks, confer advantages. “If you just do the naïve thing and say, ‘Never say anything that you’re not a hundred per cent sure about,’ you can get a model to do that,” he said. “But it won’t have the magic that people like so much.”"

Monday, March 16, 2026

The dictionary sues OpenAI; TechCrunch, March 16, 2026

 Amanda Silberling, TechCrunch; The dictionary sues OpenAI

"Encyclopedia Britannica and Merriam-Webster have filed a lawsuit against OpenAI, alleging in its complaint that the AI giant has committed “massive copyright infringement.”

Britannica, which owns Merriam-Webster, retains the copyright to nearly 100,000 online articles, which have been scraped and used to train OpenAI’s LLMs without permission, the publisher alleges in the lawsuit.

Britannica also accuses OpenAI of violating copyright laws when it generates outputs that contain “full or partial verbatim reproductions” of its content and when the AI lab uses its articles in ChatGPT’s RAG (retrieval augmented generation) workflow. OpenAI’s RAG tool is how the LLM scans the web or other databases for newly updated information when responding to a query. Britannica also alleges that OpenAI violates the Lanham Act, a trademark statute, when it generates made-up hallucinations and attributes them falsely to the publisher."

Monday, February 16, 2026

AI legal advice is driving lawyers bananas; Axios, February 9, 2026

Emily Peck, Axios; AI legal advice is driving lawyers bananas

"AI promises to make work more productive for lawyers, but there's a problem: Their clients are using it, too.

Why it matters: The rise of AI is creating new headaches for attorneys: They're worried about the fate of the billable hour, a reliable profit center for aeons, and are perturbed by clients getting bad legal advice from chatbots.

Zoom in: "It's like the WebMD effect on steroids," says Dave Jochnowitz, a partner at the law firm Outten & Golden, referring to how medical websites can give people a misguided understanding of their condition."

Thursday, January 8, 2026

Judges are identifying suspected AI hallucinations in Pa. court cases — including one at the highest levels; Spotlight PA, January 7, 2026

 

Sarah Boden, Spotlight PA; Judges are identifying suspected AI hallucinations in Pa. court cases — including one at the highest levels


"Veteran attorneys with a track record of arguing high-profile cases submitted an error-filled brief to one of Pennsylvania’s appellate courts, raising questions from a judge about their use of artificial intelligence...

“Your credibility is such an important part of what a lawyer is to bring to the case,” said Vanaskie. “If the lawyer is not verifying what's being submitted, their credibility is shot.”"