Australis13@fedia.io
on 01 Aug 2025 10:31
nextcollapse
LLMs are being shoved into so many bits of software (office suites, programming tools, etc.) it doesn't surprise me that something like this has happened.
unexposedhazard@discuss.tchncs.de
on 01 Aug 2025 10:45
collapse
Law people have a duty to be up to date on this stuff tho. If you dont know how to avoid LLMs from seeing or interacting with your stuff then you shouldnt be allowed to practice law.
To really be sure would require knowing what software is actually doing - not just taking the claims made by the programmers (or more likely, the mere owners) of the proprietary software.
That sounds doubly difficult job.
unexposedhazard@discuss.tchncs.de
on 01 Aug 2025 11:20
nextcollapse
Pretty sure there is such a thing as legally certified software where the liability would then lie with the software vendor.
jbloggs777@discuss.tchncs.de
on 01 Aug 2025 11:58
nextcollapse
Judges usually don’t know this stuff, but they primarily work with systems and software supplied by the state…whose experts should know what they are doing.
My bet is that this guy decided to work on personal equipment, probably in violation of the rules. Being a judge, he’s unlikely to be sanctioned for it, and will certainly learn from the experience. If anything, there may be some internal discussions which we’ll never hear about.
Law is an area where AI can add value, though… searching through past rulings and legal opinions is tedious, and anything that can assist to find needles in haystacks would be welcome. It shouldn’t be used to write legal judgements or arguments though…
14th_cylon@lemmy.zip
on 01 Aug 2025 13:27
collapse
To really be sure would require knowing what software is actually doing
i am pretty sure you do know whether you wrote a text, or it just magically spawned in front of your eyes out of thin air - you don’t need degree in computer science for that.
Also no, you don’t know what it’s doing so you could be blindsighted by the latest AI update making unexpected changes. Not only from good-intentioned features but also bugs, or malicious anti-features after the CEO throws their toys out of the Twitter pram.
Consider that a program can edit the file while running at any point, not merely during user input. Like a virus with access to user’s files it could even edit a document that’s not even being displayed to the user on the screen.
This may be out of date but in this video by Lawful Masses lawyers are concerned that software AI tools which somehow (I don’t recall) help them understand a case. This issue is the AI should not use information sourced from another client’s confidential case/documents to inform them about another case but they don’t know how it works. Responses from Microsoft were not forthcoming.
I would argue they can’t know unless they have access to the source code to verify what any (local) AI can do (not personally do it, but a trusted 3rd party audit which isn’t behind closed doors).
perishthethought@piefed.social
on 01 Aug 2025 10:41
nextcollapse
“If an attorney does this, a judge can demand explanations, but it’s not true in the other direction,” Frohock said. “We will probably never know what happened, unless an appellate court demands it.”
That sucks. There should be some sort of consequence for doing this.
Quill7513@slrpnk.net
on 01 Aug 2025 13:11
collapse
the only place at this point that can come from is from the governed: the populace
sp3ctr4l@lemmy.dbzer0.com
on 01 Aug 2025 12:42
nextcollapse
threaded - newest
LLMs are being shoved into so many bits of software (office suites, programming tools, etc.) it doesn't surprise me that something like this has happened.
Law people have a duty to be up to date on this stuff tho. If you dont know how to avoid LLMs from seeing or interacting with your stuff then you shouldnt be allowed to practice law.
To really be sure would require knowing what software is actually doing - not just taking the claims made by the programmers (or more likely, the mere owners) of the proprietary software.
That sounds doubly difficult job.
Pretty sure there is such a thing as legally certified software where the liability would then lie with the software vendor.
Judges usually don’t know this stuff, but they primarily work with systems and software supplied by the state…whose experts should know what they are doing.
My bet is that this guy decided to work on personal equipment, probably in violation of the rules. Being a judge, he’s unlikely to be sanctioned for it, and will certainly learn from the experience. If anything, there may be some internal discussions which we’ll never hear about.
Law is an area where AI can add value, though… searching through past rulings and legal opinions is tedious, and anything that can assist to find needles in haystacks would be welcome. It shouldn’t be used to write legal judgements or arguments though…
i am pretty sure you do know whether you wrote a text, or it just magically spawned in front of your eyes out of thin air - you don’t need degree in computer science for that.
Creating text is not the only issue, it may be trained from your confidential files.
Also no, you don’t know what it’s doing so you could be blindsighted by the latest AI update making unexpected changes. Not only from good-intentioned features but also bugs, or malicious anti-features after the CEO throws their toys out of the Twitter pram.
no malicious update can force you to generate a text and file it in court as your own work.
Consider that a program can edit the file while running at any point, not merely during user input. Like a virus with access to user’s files it could even edit a document that’s not even being displayed to the user on the screen.
well that would be fucked up for sure. are you suggesting any existing program works like that, or are just speculating what if?
This may be out of date but in this video by Lawful Masses lawyers are concerned that software AI tools which somehow (I don’t recall) help them understand a case. This issue is the AI should not use information sourced from another client’s confidential case/documents to inform them about another case but they don’t know how it works. Responses from Microsoft were not forthcoming.
I would argue they can’t know unless they have access to the source code to verify what any (local) AI can do (not personally do it, but a trusted 3rd party audit which isn’t behind closed doors).
That sucks. There should be some sort of consequence for doing this.
the only place at this point that can come from is from the governed: the populace
Vibe based legal system, sure, why not?
That’s just called common law
This made me laugh. Not wrong though!
I know a law firm who creates garbage like this without even using AI
Should have judged the situation better
Another awesome Regan appendage
I SO want to love that state, but fuck me, this comes at zero surprise.