A woman sues her insurance company for terminating her disability benefits.

[divverent@social.vivaldi.net]

@mjd TBH I do not think OpenAI should be responsible. They're just providing a fancy random text generator to the public. And it's outright impossible to teach a random text generator to _not_ output a specific kind of text, as whatever you do, there is a way around it.

The woman should pay all costs, as per the usual "vexatious filings" or "frivolous lawsuits" standards.

Plus, the law in her state against practicing law without a license starts with "No person shall...". ChatGPT isn't a person.

[divverent@social.vivaldi.net]

@rowat_c @mjd This honestly just makes sense. Not an attorney, no attorney-client privilege.

[infoseepage@mastodon.social]

@mjd My guess is her thought process was that her insurance company lowballed her on the settlement offer, and she was forced to accept it anyways because she couldn't afford pursue an aggressively in court, versus the insurance company has all the resources in the world and can just sit there and bankrupt anybody trying to seek justice in that manner.

[sabik@rants.au]

@divVerent @mjd
OpenAI are certainly marketing ChatGPT as being useful, whatever the fine print says, so they do bear some responsibility there

[infoseepage@mastodon.social]

@mjd So, she turned the whole thing around and used an llm to generate a large quantity of motions and filings which the insurance company now had to analyze and rebut, costing them lots of time and money.

[divverent@social.vivaldi.net]

@sabik @mjd It did probably exactly what she asked for in the prompt, so where's the problem? Definitely "useful".

In case she was misinformed by ChatGPT and has to pay penalties for that reason, then _she_ should be the one suing OpenAI, not the insurance company.

[mjd@mathstodon.xyz]

@Infoseepage Yes, and which the judge and the court clerks also had to deal with, consuming public resources that should have been allocated to more deserving citizens.

[mjd@mathstodon.xyz]

@Infoseepage You made that up out of your head to come to the conclusion you selected beforehand.

I don't know what actually happened, and neither do you.

[gyrosgeier@hachyderm.io]

@mjd Wouldn't that be "tortuous inference"?

[falcennial@mastodon.social]

@GyrosGeier @mjd torturous interference

[falcennial@mastodon.social]

@mjd @krupo it will be a financial bubble pop, followed by what we will call 'the AI recession,' then limited, appropriate use. the shit is the dot com and GFC playbook all day.

[mjd@mathstodon.xyz]

@marshray I wonder if that will help get them off the hook. If not, it shows that they were aware that what they were doing could be a problem.

[gyrosgeier@hachyderm.io]

@falcennial @mjd I mean, because running an AI model is called "inference."

[mjd@mathstodon.xyz]

@falcennial @GyrosGeier They're all closely related. They're from the Latin verb “to twist”.

[jonoleth@mastodon.social]

@divVerent @mjd ChatGPT is not a person, which is why ChatGPT is not being sued. OpenAI sells a tool that gave her legal advice, and they certainly didn't say anywhere that it's actually just a "fancy random text generator"

[mjd@mathstodon.xyz]

@divVerent Except that there are laws against providing bogus legal advice to people, to prevent exactly this sort of situation.

And, as you pointed out, it was OpenAI, not ChatGPT, providing the advice.

[infoseepage@mastodon.social]

@mjd Nope, I don't know the particulars of the case, but I do have some experience with the ruthlessness of insurance companies when it comes to disabled people, including a friend who went blind from cancer as a child and another who was born with severe cerebral palsy.

[falcennial@mastodon.social]

@mjd @krupo that MIT article demonstrated 5% of AI implementations are profit making. 95% are loss making. so when the investment goldrush mania ends or winds down, 95% of invested amounts will be wiped out. it's billions so it will have a disruptive and negative economic effect that I think we will likely experience as recession. and then that 5% of profit making implementations will be what carries forward, with further investment being modelled on those (anyone can do that right now).

[sabik@rants.au]

@divVerent @mjd
If ChatGPT misinformed her, that's not very useful

[divverent@social.vivaldi.net]

@jonoleth @mjd Pretty sure it's common knowledge that LLMs are nothing but random text generators.

OpenAI is a company, not a person. From what I understand, the law banning unlicensed legal advice bans _persons_ and gives them a penalty for doing so anyway.

But OpenAI, being a company, cannot commit crimes (after all, how to put a company in prison?). Only the employees can. So the question is which concrete employee committed a crime there. (Yes, some say companies _can_ commit crimes, but then solve the problems by making an employee / owner / ... actually criminally liable - but then they are the ones who have committed the crime)

The question is rather, have any employees of OpenAI committed a crime there? If any employee at OpenAI _knew_ that it tries to give legal advice, and did not implement any countermeasures, then that employee has committed a crime. That's the case no matter how the "random text generator" works.

If someone tries to get legal advice out of a magic 8-ball, AND the company producing the 8-ball does not implement any countermeasures (such as writing in the manual that responses it gives cannot be used as legal advice), then they can potentially be held liable. Except that in case of a mechanical device that works strikingly like a die it may not be necessary to put such a disclaimer ;)