A woman sues her insurance company for terminating her disability benefits.

[wcbdata@vis.social]

@mjd Try me. There isn't a use case for it that isn't, at its core, fraud.

[marshray@infosec.exchange]

@mjd “41. On October 29, 2025, OPENAI amended the terms and usage policies of ChatGPT to prohibit users from using ChatGPT to provide tailored legal advice. Prior to the October 29, 2025 emendation, ChatGPT’s terms of use did not prohibit users from using ChatGPT to draft legal papers, conduct legal research, provide legal analysis or give legal advice.”

[wcbdata@vis.social]

@mjd Couldn't think of even one reasonable candidate in 15 minutes, even with your precious AI right there in front of you? I rest my case.

[divverent@social.vivaldi.net]

@mjd TBH I do not think OpenAI should be responsible. They're just providing a fancy random text generator to the public. And it's outright impossible to teach a random text generator to _not_ output a specific kind of text, as whatever you do, there is a way around it.

The woman should pay all costs, as per the usual "vexatious filings" or "frivolous lawsuits" standards.

Plus, the law in her state against practicing law without a license starts with "No person shall...". ChatGPT isn't a person.

[divverent@social.vivaldi.net]

@rowat_c @mjd This honestly just makes sense. Not an attorney, no attorney-client privilege.

[infoseepage@mastodon.social]

@mjd My guess is her thought process was that her insurance company lowballed her on the settlement offer, and she was forced to accept it anyways because she couldn't afford pursue an aggressively in court, versus the insurance company has all the resources in the world and can just sit there and bankrupt anybody trying to seek justice in that manner.

[sabik@rants.au]

@divVerent @mjd
OpenAI are certainly marketing ChatGPT as being useful, whatever the fine print says, so they do bear some responsibility there

[infoseepage@mastodon.social]

@mjd So, she turned the whole thing around and used an llm to generate a large quantity of motions and filings which the insurance company now had to analyze and rebut, costing them lots of time and money.

[divverent@social.vivaldi.net]

@sabik @mjd It did probably exactly what she asked for in the prompt, so where's the problem? Definitely "useful".

In case she was misinformed by ChatGPT and has to pay penalties for that reason, then _she_ should be the one suing OpenAI, not the insurance company.

[mjd@mathstodon.xyz]

@Infoseepage Yes, and which the judge and the court clerks also had to deal with, consuming public resources that should have been allocated to more deserving citizens.

[mjd@mathstodon.xyz]

@Infoseepage You made that up out of your head to come to the conclusion you selected beforehand.

I don't know what actually happened, and neither do you.

[gyrosgeier@hachyderm.io]

@mjd Wouldn't that be "tortuous inference"?

[falcennial@mastodon.social]

@GyrosGeier @mjd torturous interference

[falcennial@mastodon.social]

@mjd @krupo it will be a financial bubble pop, followed by what we will call 'the AI recession,' then limited, appropriate use. the shit is the dot com and GFC playbook all day.

[mjd@mathstodon.xyz]

@marshray I wonder if that will help get them off the hook. If not, it shows that they were aware that what they were doing could be a problem.

[gyrosgeier@hachyderm.io]

@falcennial @mjd I mean, because running an AI model is called "inference."

[mjd@mathstodon.xyz]

@falcennial @GyrosGeier They're all closely related. They're from the Latin verb “to twist”.

[jonoleth@mastodon.social]

@divVerent @mjd ChatGPT is not a person, which is why ChatGPT is not being sued. OpenAI sells a tool that gave her legal advice, and they certainly didn't say anywhere that it's actually just a "fancy random text generator"

[mjd@mathstodon.xyz]

@divVerent Except that there are laws against providing bogus legal advice to people, to prevent exactly this sort of situation.

And, as you pointed out, it was OpenAI, not ChatGPT, providing the advice.

[infoseepage@mastodon.social]

@mjd Nope, I don't know the particulars of the case, but I do have some experience with the ruthlessness of insurance companies when it comes to disabled people, including a friend who went blind from cancer as a child and another who was born with severe cerebral palsy.