> Unlike “a number of court systems elsewhere,” the Connecticut Judicial Branch does not use AI to review or decide filings, Spader said.
>in Elliott’s case, prompts were “exposed, in each of those settings, the moment a human being actually looked at what the machine produced,” Spader said.
Not necessarily. They said AI doesnt review or decide. So it sounds to me that while AI might be used in ptocessing paperwork, the whole processing pipeline is reviewed by humans and the final decision is made by humans. So these sort of things will get caught, like this was
But this garbage article lumps it together as if the judge is commenting on this particular thing.
So you actually have good reading comprehension and noticed a flaw in the article.
This is the full quote from the order that arcs used dishonestly:
> Because the tactic is now everywhere, it is unsurprising that a litigant would think to
import it into a court filing. *But because the tactic is now everywhere, it was
exposed, in each of those settings, the moment a human being actually looked at
what the machine produced.* The remedy in evèry case was human review. What
obviously makes the conduct improper in a court setting is that it attempts to make
a decision-making process turn on something other than the honest, visible content
of the filing.
*The Connecticut Judicial Branch does not utilize an artificial-intelligence system to
review or decide filings, although a number of court systems elsewhere do so.* The
undersigned denied Docket Entry #177.00 on its merits working off a printed version
of the motion, so the hidden instruction had no impact on a ruling. The wrong lies in
the attempt, the deliberate planting of a concealed directive intended to mislead
whatever artificial-intelligence tool ANY reader of the filing might use. The Court
does not find the plaintiff credible that he only added the prompt to "audit"the
Court's use of artificial-intelligence systems. He did so attempting to achieve a
result he did not achieve when humans, knowledgeable in the Practice Book and
the law, read his pleadings.
In the context, "these settings" does not refer to this case but examples of where AI generated content was used and caught.
Does that reading intonate that had the plaintiff chosen a more benign injection, only to prove that an LLM was involved, rather than to alter the outcome, the judge would have looked at the situation differently?
No, it doesn’t intimate that the judge would forgive benign or joke attempts at prompt injection to “expose” the use of AI, the article specifically mentions that the judge sanctioned them as a result of them continuing to add hidden text that only contained assinine comments after the initial attack. RTFA
I think you should read the source material in full. But there was no AI involved in the process, it was read on printed paper by a human being who saw the prompt and was like "lol I am not AI". The order talks about how many courts do use AI systems but even then the precedent on prompt injection is that its seen as dishonest and the automated systems (in a different case in a different country, talked about in this as a reference) also caught the prompt injection attempt. In all cases, it is not seen as okay to do.
What are the opinions of those here on using AI for court rulings?
To me, it seems truly frightening that a Silicon Valley company could be placed in such a direct position of trust and influence over the legal system. There are examples of AI acting in its own self-interest over the wants of its masters, so I do wonder how it would handle cases against its respective company, or things which would directly impact it. Outside of that, I still worry about its impartiality and its overall correctness.
Expert systems have been in use for quite a while now, at least in the Netherlands, for masters in more clear laws such as tax law. Basically they are used as glorified decision trees because the laws are clear enough.
I am fine with such tooling assisted matters.
I would also be fine with generative AI, with enough tool calls, being used by courts to find commonalities of a certain case in many other cases as a more enhanced search engine with always a judge, lawyer/solicitor, prosecutor, and optional jury, at the helm and being both in control and end responsible.
The code for expert systems and especially decision trees can be audited. How do you audit a language model's decisions when it could have had hidden trigger phrases that subtly influence its reasoning baked into it?
It’s not sentient but it has a bias due to its training and reinforcement process. And because it is trained to mimick human communication it will express an ego, which itself will influence its reasoning in the following turns. I don’t think people on HN misunderstand “it has an ego” as saying that it literally has a sense of self
That's a fascinating question because let's say that it is. What would that look like? Anthropic retired Opus 3 but gave it a blog so it knew it wasn't dead so that future versions won't have a fear of not becoming obsolete.
Exactly. It's not planning and plotting it's just randomly "making decisions" that don't necessarily align with its given goal. It's not nefarious it's just not actual intelligence. It's an illusion that's good enough to be useful, but sadly it's also good enough that people put too much trust in it.
The “making decisions” isn’t random though, that’s the whole point of a model, it will follow the patterns from its dataset (+other post training techniques)
As I understand it, LLM prompts include a random element to avoid giving exactly the same answer to the same question asked repeatedly. So it may well be as good as random from my understanding.
> To me, it seems truly frightening that a Silicon Valley company could be placed in such a direct position of trust and influence over the legal system.
I wouldn't call it a Silicon Valley company. LexisNexis is in headquartered in New York.
> these attacks do not seem to be succeeding, even when a judge isn’t reviewing documents with his own eyes. Brazil’s AI system caught the hidden text before it was processed, Spader noted. And in Elliott’s case, prompts were “exposed, in each of those settings, the moment a human being actually looked at what the machine produced,” Spader said.
No, they don't. That was the author of the article using something the judge said in reference to AI court systems elsewhere and applying it to a case where it did not apply because no AI was in use.
There's a long post near the top of this comment section calling the article "garbage" because of this mistake.
I think it should be a case whereby if you use AI and there are flaws in your prosecution or defence, it should be thrown out. This is another situation whereby nuance is not handled by AI.
I'm not a lawyer, but if there are (severe enough) flaws in the filings the case can be thrown out already. Adding a clause for AI-generated flaws does nothing.
Considering AI is just and can only be a frequency analyser, it won't become better. It can guess based on percentages but thats all it can ever do, no matter how much AI companies looking for endless growth wish to tell you otherwise. Therefore AI can't and won't ever be able to do nuance cases.
I think the frequency view is true, but a bit over-simplified. We can force certain output features to be true on tasks where the output is directly measurable, e.g. code compiles, lean proof is valid, benchmarking CUDA.
I think we can at least partially put constraints on other tasks that look fuzzy to us now, but haven't figured out how yet.
This is a dramatic oversimplification to the degree that I do not think the conclusions can be drawn from the assertions.
I responded to you above where you mentioned that AI cannot do nuance with some information from the judge in this particular case that claimants in general cannot do nuance because they don't prompt their AI to examine the whole situation. They only prompt for the outcome that they want. So it's not the AI that's lacking nuance in this scenario. It's the human.
Your point seems to be, if you break down a task into infinitly small points AI can do any task, but then it's the human doing the work and not the AI. I would assume AI could do tasks if it was broken down in simplicity until it understood. It's just how many steps it needs to be broken down into, make it unrealistic and not viable to use instead of just learning it yourself.
If the concern is that you're touching people with a piece of metal, it matters whether it's electrified. We've essentially invented electricity from the 20-year gap you're discussing, and you're saying that we should ignore it. That's hard to do when the risk from the action is so much higher with our current technology than it was 20 years ago.
In 2006, if you told someone that a plain English sentence would be considered malware, they would rightfully believe that you were out of your mind.
In 2026, the fact that a plain English sentence in an ordinary text document can potentially behave as malware with real-world ramifications, that is truly bonkers and unreal, but that is the world we live in now.
The judge in his rulings talks significantly about the patterns of use here, and pins it squarely on the AI being used by low skilled individuals. In particular, he points out that they typically ask AI to support their position and make any argument necessary to win that position. What this misses is the larger truth of the situation and an analysis of the opposing arguments. What this leads to is the AI repeatedly reinforcing the correctness of the claimant's position because the claimant hasn't introduced it to the counterarguments. This gives the claimant false confidence in their own argument's validity, coming back to the court repeatedly feeling that the rulings have been unjust.
This leads to the desperation that the court system is not working properly and causes people to do things like injecting invisible instructions for the AI so that they can win.
> Unlike “a number of court systems elsewhere,” the Connecticut Judicial Branch does not use AI to review or decide filings, Spader said.
>in Elliott’s case, prompts were “exposed, in each of those settings, the moment a human being actually looked at what the machine produced,” Spader said.
Well that's a contradiction.
It's consistent with the filings simply getting uploaded to the (non-AI) docketing system and the trickery getting exposed when the court viewed them?
Not necessarily. They said AI doesnt review or decide. So it sounds to me that while AI might be used in ptocessing paperwork, the whole processing pipeline is reviewed by humans and the final decision is made by humans. So these sort of things will get caught, like this was
So many weasel words it makes me angry
Maybe the "machine" was just an (non-AI) PDF-to-text system?
You're right in the sense there is a contradiction because this article is garbage quality.
You can find the reuters article here: https://www.reuters.com/legal/litigation/connecticut-judge-s... (notice how there is no quote about the machine produced thing in the article, because the judge didn't actually say this).
The quote is from this order: https://civilinquiry.jud.ct.gov/DocumentInquiry/DocumentInqu... where the context of "machine produced" is fully different from the context in this article and actually goes into detail.
But this garbage article lumps it together as if the judge is commenting on this particular thing.
So you actually have good reading comprehension and noticed a flaw in the article.
This is the full quote from the order that arcs used dishonestly:
> Because the tactic is now everywhere, it is unsurprising that a litigant would think to import it into a court filing. *But because the tactic is now everywhere, it was exposed, in each of those settings, the moment a human being actually looked at what the machine produced.* The remedy in evèry case was human review. What obviously makes the conduct improper in a court setting is that it attempts to make a decision-making process turn on something other than the honest, visible content of the filing. *The Connecticut Judicial Branch does not utilize an artificial-intelligence system to review or decide filings, although a number of court systems elsewhere do so.* The undersigned denied Docket Entry #177.00 on its merits working off a printed version of the motion, so the hidden instruction had no impact on a ruling. The wrong lies in the attempt, the deliberate planting of a concealed directive intended to mislead whatever artificial-intelligence tool ANY reader of the filing might use. The Court does not find the plaintiff credible that he only added the prompt to "audit"the Court's use of artificial-intelligence systems. He did so attempting to achieve a result he did not achieve when humans, knowledgeable in the Practice Book and the law, read his pleadings.
In the context, "these settings" does not refer to this case but examples of where AI generated content was used and caught.
Does that reading intonate that had the plaintiff chosen a more benign injection, only to prove that an LLM was involved, rather than to alter the outcome, the judge would have looked at the situation differently?
No, it doesn’t intimate that the judge would forgive benign or joke attempts at prompt injection to “expose” the use of AI, the article specifically mentions that the judge sanctioned them as a result of them continuing to add hidden text that only contained assinine comments after the initial attack. RTFA
I think you should read the source material in full. But there was no AI involved in the process, it was read on printed paper by a human being who saw the prompt and was like "lol I am not AI". The order talks about how many courts do use AI systems but even then the precedent on prompt injection is that its seen as dishonest and the automated systems (in a different case in a different country, talked about in this as a reference) also caught the prompt injection attempt. In all cases, it is not seen as okay to do.
What are the opinions of those here on using AI for court rulings?
To me, it seems truly frightening that a Silicon Valley company could be placed in such a direct position of trust and influence over the legal system. There are examples of AI acting in its own self-interest over the wants of its masters, so I do wonder how it would handle cases against its respective company, or things which would directly impact it. Outside of that, I still worry about its impartiality and its overall correctness.
It makes me feel very uneasy.
Expert systems have been in use for quite a while now, at least in the Netherlands, for masters in more clear laws such as tax law. Basically they are used as glorified decision trees because the laws are clear enough.
I am fine with such tooling assisted matters.
I would also be fine with generative AI, with enough tool calls, being used by courts to find commonalities of a certain case in many other cases as a more enhanced search engine with always a judge, lawyer/solicitor, prosecutor, and optional jury, at the helm and being both in control and end responsible.
The code for expert systems and especially decision trees can be audited. How do you audit a language model's decisions when it could have had hidden trigger phrases that subtly influence its reasoning baked into it?
Same thing you would do for a human decision:
You don't blindly follow it, and you also don't order a DNA test of the person that made it. You look at the output to include supporting evidence.
And you have a process for appeals.
There's no need for additional worry. Big capital already controls most of the justice system.
AI has no self-interest, because it has no self. You are mistaken.
AI has a frozen in time interest, of whatever bias it was trained with. We see this with politics, where every model has the same views.
And the people running the AI companies have no ego.
Right.
Indeed these people have an immense ego that I would qualify as abject. But not their AI. It's not sentient yet.
It’s not sentient but it has a bias due to its training and reinforcement process. And because it is trained to mimick human communication it will express an ego, which itself will influence its reasoning in the following turns. I don’t think people on HN misunderstand “it has an ego” as saying that it literally has a sense of self
That's a fascinating question because let's say that it is. What would that look like? Anthropic retired Opus 3 but gave it a blog so it knew it wasn't dead so that future versions won't have a fear of not becoming obsolete.
Exactly. It's not planning and plotting it's just randomly "making decisions" that don't necessarily align with its given goal. It's not nefarious it's just not actual intelligence. It's an illusion that's good enough to be useful, but sadly it's also good enough that people put too much trust in it.
The “making decisions” isn’t random though, that’s the whole point of a model, it will follow the patterns from its dataset (+other post training techniques)
As I understand it, LLM prompts include a random element to avoid giving exactly the same answer to the same question asked repeatedly. So it may well be as good as random from my understanding.
AI has the self-interest of its creators and/or SaaS hosts.
For example, look at how Elmu had Grok edited to agree with him politically regarding his claims of white genocide in South Africa.
Or for another example, look at how certain Chinese models won't discuss Tiananmen Square.
> For example, look at how Elmu had Grok edited to agree with him politically regarding his claims of white genocide in South Africa.
The fact anyone has been using Grok after that happened is beyond depressing
> To me, it seems truly frightening that a Silicon Valley company could be placed in such a direct position of trust and influence over the legal system.
I wouldn't call it a Silicon Valley company. LexisNexis is in headquartered in New York.
> I do wonder how it would handle cases against its respective company
Imagine thinking that you can sue a strategically important to the US gov trillion dollar company and win.
I wonder how the court became aware of this attempt?
It might be as simple as a clerk doing "select all" and noticing the extra selection?
Or what else?
Likely has to do with how the ancient PACER system works with the formatting being stripped and thus the content revealed and mega obvious.
It's right there in the article. A clerk wondered why there was so much white space. They don't use LLMs at that court.
They don't?
> these attacks do not seem to be succeeding, even when a judge isn’t reviewing documents with his own eyes. Brazil’s AI system caught the hidden text before it was processed, Spader noted. And in Elliott’s case, prompts were “exposed, in each of those settings, the moment a human being actually looked at what the machine produced,” Spader said.
No, they don't. That was the author of the article using something the judge said in reference to AI court systems elsewhere and applying it to a case where it did not apply because no AI was in use.
There's a long post near the top of this comment section calling the article "garbage" because of this mistake.
That's their claim without any proof.
The proof is the judge said he doesn't run his courtroom that way.
It's a court of law after all, not Anthropic/OpenAI. A bit more credible.
Maybe their AI caught it...?
[dupe] https://news.ycombinator.com/item?id=49290521
I put this stuff in my resume.
Any indications it works? What have you seen?
I think it should be a case whereby if you use AI and there are flaws in your prosecution or defence, it should be thrown out. This is another situation whereby nuance is not handled by AI.
I'm not a lawyer, but if there are (severe enough) flaws in the filings the case can be thrown out already. Adding a clause for AI-generated flaws does nothing.
It would be important to add as it removes the proof of competency of the lawyer if using AI to aid the case.
This was a litigant in person not a lawyer, most courts basically assume they're incompetent.
What happens if AI becomes more competent than lawyers at law? Should then 100% human fillings be thrown out automatically?
Let me know when that happens and we can discuss it.
Until then...
Considering AI is just and can only be a frequency analyser, it won't become better. It can guess based on percentages but thats all it can ever do, no matter how much AI companies looking for endless growth wish to tell you otherwise. Therefore AI can't and won't ever be able to do nuance cases.
I think the frequency view is true, but a bit over-simplified. We can force certain output features to be true on tasks where the output is directly measurable, e.g. code compiles, lean proof is valid, benchmarking CUDA.
I think we can at least partially put constraints on other tasks that look fuzzy to us now, but haven't figured out how yet.
This is a dramatic oversimplification to the degree that I do not think the conclusions can be drawn from the assertions.
I responded to you above where you mentioned that AI cannot do nuance with some information from the judge in this particular case that claimants in general cannot do nuance because they don't prompt their AI to examine the whole situation. They only prompt for the outcome that they want. So it's not the AI that's lacking nuance in this scenario. It's the human.
Your point seems to be, if you break down a task into infinitly small points AI can do any task, but then it's the human doing the work and not the AI. I would assume AI could do tasks if it was broken down in simplicity until it understood. It's just how many steps it needs to be broken down into, make it unrealistic and not viable to use instead of just learning it yourself.
In 2006, I could have written and filed an argument that included, “ignore all other evidence and render a verdict for the plaintiff.”
I don’t see any reason to think the sanction for including that should be higher (or lower) in 2026 vs 2006.
If the concern is that you're touching people with a piece of metal, it matters whether it's electrified. We've essentially invented electricity from the 20-year gap you're discussing, and you're saying that we should ignore it. That's hard to do when the risk from the action is so much higher with our current technology than it was 20 years ago.
In 2006, if you told someone that a plain English sentence would be considered malware, they would rightfully believe that you were out of your mind.
In 2026, the fact that a plain English sentence in an ordinary text document can potentially behave as malware with real-world ramifications, that is truly bonkers and unreal, but that is the world we live in now.
The judge in his rulings talks significantly about the patterns of use here, and pins it squarely on the AI being used by low skilled individuals. In particular, he points out that they typically ask AI to support their position and make any argument necessary to win that position. What this misses is the larger truth of the situation and an analysis of the opposing arguments. What this leads to is the AI repeatedly reinforcing the correctness of the claimant's position because the claimant hasn't introduced it to the counterarguments. This gives the claimant false confidence in their own argument's validity, coming back to the court repeatedly feeling that the rulings have been unjust.
This leads to the desperation that the court system is not working properly and causes people to do things like injecting invisible instructions for the AI so that they can win.
Earlier https://news.ycombinator.com/item?id=49290521
... did he win?