Mom horrified by Character.AI chatbots posing as son who died by suicide - Ars Technica (arstechnica.com)
from sabreW4K3@lazysoci.al to technology@beehaw.org on 20 Mar 17:03
https://lazysoci.al/post/23374468

#technology

threaded - newest

OsrsNeedsF2P@lemmy.ml on 20 Mar 17:58 next collapse

About 3 percent of students in the study had positive mental health outcomes, reporting that talking to the chatbot “halted their suicidal ideation.” But researchers also found “there are some cases where their use is either negligible or might actually contribute to suicidal ideation.”

This is referring to a bot designed to help with people struggling with mental health, and is actually a big one. That number is way too low.

GammaGames@beehaw.org on 20 Mar 18:22 collapse

“hey, I know you feel like killing yourself, but if it happens then we’ll just replace you with a shitty bot” probably isn’t as helpful as they thought it would be. It’s violating and ghoulish.

OsrsNeedsF2P@lemmy.ml on 20 Mar 19:07 collapse

I hate this attitude of “well if you can’t get a professional therapist, figure out how to get one anyways”. There needs to be an option for people who either can’t afford or can’t access a therapist. I would have loved for AI to fill that gap. I understand it won’t be as good, but in many regions the wait-list for therapy is far too long, and something is better than nothing

TehPers@beehaw.org on 20 Mar 19:44 next collapse

Someone close to me gave up on the hotlines in the US and now just uses ChatGPT. It’s no therapist, but at least it’ll hold a conversation. If only the hotlines here weren’t so absurdly understaffed.

Megaman_EXE@beehaw.org on 20 Mar 19:56 next collapse

I’ve used one called PI which I’m assuming is some kind of branch off of chat gpt or something.

You don’t have to sign up or anything (for now) which is cool. But I assume they harvest all our data and information.

I tested to see if I could break it once, and from my brief tests, it seemed to never break out of character or tell me something bad or negative, which I thought was interesting(and good!)

Powderhorn@beehaw.org on 21 Mar 00:14 collapse

I actually used Pi as my intro to generative LLMs. It was … I guess not encouraging self harm, but so fucking irritating that it led me to want to. Always with the irrelevant supportive words that I guess work if you’re a teen?

Megaman_EXE@beehaw.org on 21 Mar 09:28 collapse

Lol yes, that was going to be the one downside I was going to mention. I wasn’t sure if it was just unique to my situation, but I found it would lead me down a logical path. It would ask me if I had tried various solutions.

Eventually, I would hit a point where it wouldn’t know where to go any further, and it would land on “here’s some things you can do” but those options would be things I was actively trying and failing with.

So that was fun. In a way, it was great at confirming that I had thought of all the logical options.

Powderhorn@beehaw.org on 21 Mar 00:16 next collapse

I’ve given up on crisis lines. Their whole premise seems to be “get back to being comfortable with the oppressive system, you little bitch.”

Alice@beehaw.org on 22 Mar 09:58 collapse

I tried AI once but it just kept telling me to call the hotlines. Useless.

wizardbeard@lemmy.dbzer0.com on 20 Mar 22:15 collapse

I would have loved AI to fill that need as well, but it’s not an adequate tool for the job.

jlow@beehaw.org on 20 Mar 18:25 next collapse

Oh hey, reality being even worse than Black Mirror again 😿

i_am_not_a_robot@discuss.tchncs.de on 20 Mar 19:31 next collapse

This will become more common. I’ve seen dead people involved in scams on the internet.

prole@lemmy.blahaj.zone on 20 Mar 19:54 next collapse

Wasn’t this an episode of Black Mirror back when it was still really good?

halykthered@lemmy.ml on 20 Mar 21:45 next collapse

Yeah, I know someone who won’t watch that episode again because of how unsettling it is. Luckily for them, it’s slowly becoming reality.

somegeek@programming.dev on 21 Mar 20:23 collapse

I watched until season 6 and all of it was really amazing. Is S7 bad?

prole@lemmy.blahaj.zone on 25 Mar 12:59 collapse

Nah, not particularly… If you like it up til then, you will probably like it.

It’s not that I think it got bad or anything, but I think there was a noticeable drop in quality from season 3 to season 4 (the second Netflix season).

The first two seasons, when it was still on Channel 4 in the UK, were just so fucking good. Only a few episodes each, but man. And the Christmas episode with Jon Hamm, goddamn. So fucking good. Some of the best sci-fi ever put to film imo.

Then Netflix bought it. Season 3 was good, it had some bangers (I imagine Charlie Brooker had some of the plots ready to go already). Then… I don’t know maybe it’s because they were pushed to write like 3x more episodes per season? The quality suffered.

The show is still solid, and I will watch the new season for sure. But I don’t know, it’s just not the same as it was.

Powderhorn@beehaw.org on 20 Mar 22:18 collapse

Imagine a 3% success rate being acceptable in any situation. That tends to get you fired.

sqgl@beehaw.org on 21 Mar 04:11 collapse

3% success vs what? 6% sent over the edge? 10% 20% ?

If the journalist asked for a specific figure but was evaded then it should be stated in the article.

Powderhorn@beehaw.org on 21 Mar 04:49 collapse

I don’t much like that take. Ars commits excellent journalism.

From the story:

About 3 percent of students in the study had positive mental health outcomes, reporting that talking to the chatbot “halted their suicidal ideation.” But researchers also found “there are some cases where their use is either negligible or might actually contribute to suicidal ideation.”

sqgl@beehaw.org on 21 Mar 06:37 next collapse

I don’t think they contacted the researchers and the linked study does not seem to give the answer (I spent a few minutes looking).

Powderhorn@beehaw.org on 21 Mar 06:55 collapse

I generally don’t go about doing research for free.

sqgl@beehaw.org on 21 Mar 07:37 collapse

Ars offers free articles while most publications have a paywall, so I imagine funding isn’t as generous as it would have been 30 years ago when such publications would have been in magazine format.

Powderhorn@beehaw.org on 21 Mar 15:05 collapse

Ars is actually my only paid subscription. Didn’t need to, but wanted to support their journalism.

sqgl@beehaw.org on 22 Mar 08:50 collapse

I mistakenly thought you were the actual journalist. But I should always presume the journalist will see my comments and therefore not be so harsh, especially when freeloading.

FWIW I subscribe to an (Australian) online newspaper which is free just like you do. The difference being that I rarely read it since I am on top of those topics largely. Am just glad that it exists for others because it is well researched and presented.

sqgl@beehaw.org on 21 Mar 06:45 collapse

I removed the insult.

Powderhorn@beehaw.org on 21 Mar 06:55 collapse

Thank you.