Authors Are Furious After Finding Their Works on List of Books Used To Train AI
(www.themarysue.com)
from stopthatgirl7@kbin.social to technology@lemmy.world on 29 Sep 2023 07:50
https://kbin.social/m/technology@lemmy.world/t/498078
from stopthatgirl7@kbin.social to technology@lemmy.world on 29 Sep 2023 07:50
https://kbin.social/m/technology@lemmy.world/t/498078
Authors using a new tool to search a list of 183,000 books used to train AI are furious to find their works on the list.
threaded - newest
I would be proud, but you do you.
But would you get paid?
Yes. People wouldn’t be able to pirate my story through an AI, it wouldn’t spit it out verbatim. They’d still need to buy or pirate it other ways.
They don’t need to, the AI just tells them what happens. Why are you against the author being able to consent for their work to be trained on and being compensated?
You think spoilers should be illegal?
?
You appear to be saying that:
Is a violation of the author's copyright.
I know, right? It’s weird.
I certainly hope that none of these authors have ever read a book before or have been inspired by something written by another author.
That would be a much better comparison if it was artificial intelligence, but these are just reinforcement learning models. They do not get inspired.
...like the naturally occuring neural networks are.
The brain does not work the way you think… (I work in the field, bio-informatics). What you call “neural networks” come from an early misunderstanding of how the brain stores information. It’s a LOT more complicated and frankly, barely understood.
Yeah, accurately simulating a single pyramidal neuron requires an eight-layer deep neural network:
www.cell.com/neuron/…/S0896-6273(21)00501-8.pdf
that was an interesting read, thank you
Yet you confidently state that the brain doesn't work the way LLMs do?
Obviously it doesn't work exactly the same way that LLMs do, if only because of the completely different substrates. But when you get to more nebulous concepts like "creativity" and "inspiration" it's not so clear.
The part where brain and neural net differ is in the learning via backpropagation, that seem to be done different in the brain, as there is no mechanism to go backwards through the network and jiggle the weights.
That aside, they seem to work very similar once they are trained, as the knowledge they are able to extract from data ends up being basically the same that a human would be able to extract. There is surprisingly little weirdness in AI and a surprising amount of human-like capabilities.
people have a definite fear of being defined as machines... not sure why we think were so special..
so its barely understood, but this definitely is not it. got it.
But you, random stranger on the internet, knows better than the guy that literally works in the field. Got it.
i do? where did i claim that?
Tell you what, you get a landmark legal decision classifying LLM as people and then we’ll talk.
Until then it’s software being fed content in a way not permitted by its license i.e. the makers of that software committing copyright infringement.
What exactly was not permitted by the license? Reading?
Using it to (create a tool to) create derivatives of the work on a massive scale.
Well when that happens we have laws. So no problems
Would you be okay with applying that argument for any crime?
Ever heard of the early 21st century classic Minority Report
You’re missing the point. I’ll make your example more specific.
Those things happen. Creating a LLM based on copyrighted material without permission happens - it’s not a hypothetical. But even then, giving a punishment after the fact does not make the initial crime “no problem”, as you put it.
I would be, and I don't understand why you think this would be a problem. I wouldn't want the government to be preventing activities that there weren't any actual laws prohibiting.
Wikipedia: In copyright law, a derivative work is an expressive creation that includes major copyrightable elements of a first, previously created original work.
I think you may be off a bit on what a derivative work is. I don’t see LLMs spouting out major copyrightable elements of books. They can give a summary sure, but Cliff Notes would like to have a word if you think that’s copyright infringement.
An AI model is not a derivative work. It does not contain the copyrighted expression, just information about the copyrighted expression.
Better tell that Google and their search index, book scanning project and knowledge graph.
I didn’t know those were LLMs, TIL.
More to the point: they replicate patterns of words.
So do humans.
That’s a Bingo!
These are machines, though, not human beings.
I guess I’d have to be an author to find out how I’d feel about it, to be fair.
What's the difference? On the most fundamental level it's all the same.
Wait. Are human beings machines?
Please do not take this as support of ai use of copyrighted works (I don’t), but as far as I can tell, yes we are machines. This rant is just me being aspie atm, so feel free to ignore it.
We are thinking machines programmed by our genetics, predispositions, experiences, and circumstances. A 2 part explanation of how humans are merely products of their circumstances was once put forward to me. The first part is that humans can do anything, but only the thing we want to do most.
For instance, a common rebuttal is that people can choose go to the gym even when they find the experience of exercise undesirable. However, when that happens, it’s merely a case of other wants out balancing the want to not go to the gym, typically they want to be fit.
We want to not spend money, but we want to not rush going to jail for stealing more, usually. We want to not work overtime, but sometimes we want the extra cash more than that.
The second part of the argument is that we can’t choose what we want. When someone talks themselves out of the slice of cheesecake, they aren’t changing what they want, they’re resolving said want against the larger want they have to lose weight.
And if we make decisions by our wants, while said wants are not decided by us, then despite appearances we are little more than complex automata.
Biological machines, yes.
Unless you think theres no difference between killing a person and closing a program, I think we can agree they should be treated differently in the eyes of the law.
And so theres a difference between a person reading a book and being inspired by it, and someone writing a program that automatically transforms the book in data that can create new books.
The same thing as with tooooooons of things: scale.
Nobody cares if one dude steals office supplies at work. Now, if everyone stats doing it, or if the single guy steals everything, then action is taken.
Nobody cares if a random person draws in the same style and with same characters as you, but if they start to sell them, or god forbid, out-sell you, then there is a problem.
Nobody cares (except police I guess) if a random driver drives double the speed limit and annoys people living next to the road on the weekends, but when tons of people do it, you get speed bumps.
Nobody cares if few people pirate movies, but when it gets to mainstream and companies notice that there might be money being lost. Then you get whatever we have now.
Nobody cares if the mudhill behind your house erodes a bit and you get mud on your shoes. Have a bunch of that erode and you realise the danger…
You have been fine-tuning your own writing style for a decade and random schmuck starts to write similarly, you probably don’t care. No harm done. Now, get an AI to write 10 000 books in a weekend and someone starts to sell them… well now you have a completely different problem.
On a fundamental level the exact same thing is happening, yet action is only taken after a certain threshold is step over.
Bingo.
A human, regardless of how many books they read, will have personal experiences that are undeniably unique to themselves. They will interpret the works they read differently from each other based on their worldly experiences. Their writing, no matter how many books they read and get inspired on, will always be influenced by their own personal lives. They can experience love, hate, heartbreak, empathy, sadness, and happiness.
This is something a LLM does not have, and in my opinion, is a massive distinguishing factor. So on a “fundamental” level, it is not the same. It is no where near the same.
So will every AI. ChatGPT will give you different answers than Bard or WizardLM, since they are all trained on different books. And every StableDiffusion model creates different images, different styles, different topics, etc. It’s all in the data they “experienced”.
do you really think we are that far off... from giving a foundational memory and motivation layers to these LLMs, that could mimic.. or even.. generate the generic thoughts youre indicating?
i dont think so. you seem to imply its impossibility, i expect its inevitability. the human brain will not be a black box forever... it still exists in a world of physics we can emulate, even if rudimentary.
But terminator said neural networks
Damn.
Machines that aren’t reproducing or distributing works
If an AI "reproduces" a work it was trained on it is a failure of an AI. Why would anyone want to spend millions of dollars and devote oodles of computing power to build something that just does what a simple copy/paste operation can accomplish?
When an AI spits out something that's too close to one of the original training set that's called "overfitting" and it is considered an error to be corrected. Most overfitting that's been detected has been a result of duplication in the training set - when you hammer an AI image generator in training with thousands of copies of the Mona Lisa it eventually goes "alright, I get it already, when you say 'Mona Lisa' you want that exact pattern!" And will try its best to replicate that pattern when you ask it to later. That's why training sets need to be de-duplicated.
AIs are meant to produce new things.
I don’t think anyone is faulting the machines for this, just the people who instruct the machines to do it.
Did you write a comment on Reddit before 2015? If so, your copyrighted content was used without your permission to train today’s LLMs, so you absolutely get to feel one way or another about it.
The idea that these authors were somehow the backbone of the models when any individual contribution was like spitting in the ocean and model weights would have considered 100 pages of Twilight fan fiction equivalent to 100 pages from Twilight is honestly one of the negative impacts of the extensive coverage these suits are getting.
Pretty much everyone who has ever written anything indexed online is a tiny part of today’s LLMs.
Thank you for your reply.
On a completely separate note, it’s funny to think that there exists Twilight fan fiction when
Twilight itself started as fan fiction work.Edit: I dun goofed.
Pretty sure it’s the other way around.
Fifty Shades of Gray started out as Twilight fanfiction before becoming its own thing.
AFAIK Twilight was always just its own pulp fiction.
Oh true! My memory was fuzzy on the details. Thanks for the correction.
Are you saying the writers of these programs have read all these books, and were inspired by them so much they wrote millions of books? And all this software is doing is outputting the result of someone being inspired by other books?
Clearly not. He’s saying that other authors have done the same as the software does. The software creators implemented the same principle into their llm. You are being daft on purpose.
It’s not the same principle. Large language models aren’t ‘inspired’ to write new works. Software can’t be inspired. It follows instructions. Even though large language models might feel like somebody is talking back to you and giving you new information, it’s just code following instructions designed to predict output based on the input provided and the data supplied. There’s no inspiration to be had, and to attribute inspiration to language models is a huge mischaracterization of what’s happening under the hood. Can a language model, without being told what to do, actually use any of the data it was fed to create something? No. Every single large language model requires some sort of input from a user to act as a seed before any sort of response can begin.
This is why it’s so stupid to call this shit AI, because people start thinking it’s actual intelligence. Really, It’s just a fancy illusion.
It is using the term as defined. Maybe stop being a stupid parrot just repeating crap you heard else where and use your brain for a moment. I am losing hope that humans are capable of thought reading all this junk.
They purchased their books to get inspiration from, the original author gets paid, and the author consented to selling it. That’s the difference.
Also the LLM can post entire snippets or chapters of books, which of course you’ll take at face value even if it hallucinates and makes the author look like a worse author then they are.
That’s an interesting take, I didn’t know software could be inspired by other people’s works. And here I thought software just did exactly as it’s instructed to do. These are language models. They were given data to train those models. Did they pay for the data that they used to train for it, or did they scrub the internet and steal all these books along with everything everyone else has said?
Well, now you know; software can be inspired by other people's works. That's what AIs are instructed to do during their training phase.
Does that mean software can also be afraid, or angry? What about happy software? Saying software can be inspired is like saying a rock can feel pain.
Software can do a lot of things that rocks can't do, that's not a good analogy.
Whether software can feel "pain" depends a lot on your definitions, but I think there are circumstances in which software can be said to feel pain. Simple worms can sense painful stimuli and react to it, a program can do the same thing.
We've reached the point where the simplistic prejudices about artificial intelligence common in science fiction are no longer useful guidelines for talking about real artificial intelligence. Sci-fi writers have long assumed that AIs couldn't create art and now it turns out it's one of the things they're actually rather good at.
If it is programmed/trained that way, sure. I recommend having a listen to Geoffrey Hinton on the topic (41:50).
The rock doesn’t do anything similar to pain. The LLM on the other side does a lot of things similar to inspiration. I can give the LLM a very trivial question and it will answer with a mountain of text. Did my question or the books it was trained on “inspire” the LLM to write that? Maybe, depends of course how far reaching you want to define the word. But either way, the LLM produced something by itself, that was neither a copy of my prompt nor the training data.
Here is an alternative Piped link(s):
Geoffrey Hinton on the topic
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source; check me out at GitHub.
Software cannot be “inspired”
AIs in their training stages are simply just running extreme statistical analysis on the input material. They’re not “learning” they’re not “inspired” they’re not “understanding”
The anthropomorphism of these models is a major problem. They are not human, they don’t learn like humans.
People attributing any kind of person hood or sentience is certainly a problem, the models are fundamentally not capable of that (no loops, no hidden thought). At least for now. However what you are doing isn’t really much better, just utterly wrong in the opposite direction.
Those models are very definitely do “learn” and “understand” by every definition of the word. Simply playing around with that will quickly show that and it’s baffling that anybody would try to claim otherwise. Yes, there are limits to what they can understand and there are plenty things that they can’t do, but the amount of questions they can answer goes far beyond what is directly in the training data. Heck, even the fact that they hallucinate is proof that they understand, since it would be impossible to make completely plausible, but incorrect, stuff up without having a deep understanding of the topics. Also humans make mistakes too and they’ll also make stuff up, so this isn’t even anything AI specific.
Yeah, that’s just flat out wrong
Hallucinations happen when there’s gaps in the training data and it’s just statistically picking what’s most likely to be next. It becomes incomprehensible when the model breaks down and doesn’t know where to go. However, the model doesn’t see a difference between hallucinating nonsense and a coherent sentence. They’re exactly the same to the model.
The model does not learn or understand anything. It statistically knows what the next word is. It doesn’t need to have seen something before to know that. It doesn’t understand what it’s outputting, it’s just outputting a long string that is gibberish to it.
I have formal training in AI and 90%+ of what I see people claiming AI can do is a complete misunderstanding of the tech.
Than why do you keep talking such bullshit? You sound like you never even tried ChatGPT.
Yes, that’s understanding. What do you think your brain does differently? Please define whatever weird definition you have of “understand”.
You are aware of Emergent World Representations? Or have a listen to what Ilya Sutskever has to say on the topic, one of the people behind GPT-4 and AlexNet.
Which is obviously nonsense, as I can ask it questions about its output. It can find mistakes in its own output and all that. It obviously understands what it is doing.
They weren’t given data. They were shown data then the company spent tens of millions of dollars on cpu time to do statistical analysis of the data shown.
A computer being shown data is a computer being given data. I don’t understand your argument.
The data is gone by the time a user interacts with the AI. ChatGPT has no access to any books.
AI isn’t software. Everything the AI knows is from the books. There is no human instructing the AI what to do. All the human does is build the scaffolding to let the AI learn, everything else is in the data.
Hey, computational linguist here who works with large language models. This is the most ridiculous thing I ever read.
Generally they probably bought the books they read though.
If George RR Martin torrented Tolkien, wouldn’t he be infringing on the copyright no matter how he subsequently incorporated it into future output?
I completely agree that the training as infringement argument is ludicrous.
But OpenAI exposed themselves to IP infringement by sailing the high seas in how they obtained the works in the first place.
I hate that the world we live in is one where so much data is gated behind paywalls, but the law is what it is, and if the government was going to come down hard on Aaron Swartz for trying to bypass paywalls for massive amounts of written text, it’s not exactly fair if there’s a double standard for OpenAI doing the same thing in an even more closed fashion.
But yes, the degree of entitled focus on the premise of training an AI as equivalent of infringing is weird as heck to see from authors drawing quite clearly from earlier works in their own output.
God that Aaron/jstor thing makes me see red every time. Swartz was scraping jstor to publish it for the benefit of everyone, openai is doing it to make billions of dollars. Don’t forget who the bad guys are (and donate to sci-hub)
I have to assume that openAI also paid for the books. if yes then i consider it the same as me reciting passages from memory or coming up with derivative text.
if no, then by all means, go after them and any model trainer for the cost of one book.
Asking an LLM to recite an entire novel isn’t even vaguely a thing yet.
Well, here’s straight from one of the suits against them:
I’m not even sure how they would have logistically gone about purchasing 294,000 books in bulk in digital form to be fed into training. Using the existing collections seems much more likely, but I suppose we’ll see what turns up in litigation.
Also, the penalty for downloading copyrighted material if willful infringement is up to $250,000 per work. So it’s quite a bit more than the cost of one book on the line…
I hope they can at least get compensated.
.
So where can I check to see if my book was used? I published a book.
Did you ever comment on Reddit before 2015? If so, your copyrighted material was used to train the modern LLMs even if your published book wasn’t used at all.
Yes I did my account is almost 11years old on Reddit. But I was talking about my novel that was never on Reddit.
The database is here. You’ll have to sign up for a free trial if you’re not a subscriber to The Atlantic already. www.theatlantic.com/technology/archive/…/675363/
they were compensated when the company using the book, purchased the book. you can’t tell me what to do with the words written in the book once I’ve purchased it. nor do you own the ideas or things I come up with as a result of your words in your book. of course this argument only holds up if they purchased the book. if it was “stolen” then they are entitled to the $24.95 their book costs.
Good point. I guess this aspect is much different from the AI Art scene, where the producers of the dataset are usually not compensated for their drawings.
That’s the thing – they weren’t.
The case has two prongs.
One is that training the AI on copyrighted material is somehow infringement, which is total BS and a dangerous path for the world to go down.
The other is that copyrighted material was illegally downloaded by OpenAI, which is pretty much an open and shut case, as they didn’t buy up copies of 100k books, they basically torrented them.
And because of ridiculous IP laws bought by industry lobbyists in the dawn of the digital age, the damages are more like $250,000 per book if willful infringement, not $24.95.
Had they purchased them, these cases would very likely be headed for the dumpster heap.
That said, there’s a certain irony to Lemmy having pirate subs as one of the most popular while also generally being aggressively pro-enforcement on IP infringement.
Training AI on copyrighted material is infringement and I’ll die on that hill. It’s use of copyrighted material to create a commercial product. Doesn’t get any more clear cut than that.
I know as an artist/musician/photographer I’d rather not put my creations out there at all if it means some corporation is going to be able to steal it.
You can stand wherever you like on any hill you’d like, but the question of nonprofit use vs commercial use is only one part of determining fair use, and where your stance is going to have serious trouble is the fact that the result of the training is extremely transformed from the training data, with an entirely different purpose and character and cannot even reproduce any of the works used in training in their entirety. And the areas where they can reproduce in part are likely not even the direct result of using the work itself in training, but additional reinforcement from other additional secondary uses and quotations of the reproducible parts of works in question.
And don’t worry. Within about a year or so (by the time any legal decision gets finalized or new legislation is passed) no one is going to care about ‘stealing’ your or anyone else’s creations, as training is almost certainly moving towards using primarily synthetic data and curated content creation to balance out edge cases.
Use of preexisting works was a stepping stone hack that acted like jumper cables starting the engine. Now that it’s running, there’s a rapidly diminishing need for the other engine.
Edit: And you’d have a very hard time convincing me that StableDiffusion using Studio Ghibli movies to train a neural network that can produce new and different images in that style is infringement while Weiden+Kennedy commercially making money off of producing this ad is not.
What about my Reddit history?
Arguably there’s more of my text there that was used to train these LLMs than most authors in that list.
The comment elsewhere in this thread about models built on broad public data needing to be public in turn is a salient one.
IP laws were designed to foster innovation, not hold it back.
I’d much rather see a world where we have open access models trained broadly and accelerating us towards greener pastures than one where book publishers get a few extra cents from less capable closed models that take longer for us to reach the heyday where LLMs can do things like review the past 20 years of cancer research in order to identify promising trends in allocation of future resources.
OpenAI should probably rightfully be dinged for downloading copyrighted media the same way any average user would be sued when caught doing the same.
But the popular arguments these days for making training infringement are ass backwards and a slippery slope to a far more dystopian future than the alternative.
.
This troll isn’t even good, why bother?
It’'s wacky, but some people don’t believe ideas are “owned” and need to be “rented” out for use and that no piece of art is an island, they all “steal” from prior works.
Can you believe that? Next thing you know there will be human sacrifices, cats and dogs living together, mass hysteria!
I agree with them that an idea can’t be owned but it’s a misunderstanding of “AI” to compare it to artistic inspiration
No it’s not, it’s exactly the same thing. What if you go to an AI image generator and tell it to make a cat in the style of Picasso, Van Gough, and Todd McFarlane? What then? Who’s art is it “stealing”? Is it not creating a transformative piece of work? That’s exactly how the human brain works.
I think the real issue is the greedy fucks hoarding the models behind closed source projects and trying to bleed everyone else for access to them. The models should be available to anyone for any reason since they used the common knowledge and general advancement of society. They’re like libraries, and they should be public like libraries.
Anyways, the real reason I responded in the first place was your dismissive tone that any opinion going against the mob mentality about AI is somehow trolling.
This isn't a troll, it's a valid counterargument. Dismissing it is not a useful response.
Here’s an idea, legally force companies like OpenAI to rely on opt-in data, rather then build their entire company on stealing massive amounts of data. That includes requiring to retrain from scratch. Sam Altman was crying for regulations for scary AI, right?
Would search engines only be allowed to show search results for sources that had opted in? They "train" their search engine on public data too, after all.
They aren’t reselling their information, they’re linking you to the source which then the website decides what to do with your traffic. Which they usually want your traffic, that’s the point of a public site.
That’s like trying to say it’s bad to point to where a book store is so someone can buy from it. Whereas the LLM is stealing from that bookstore and selling it to you in a back alley.
AI isn’t either. It’s selling statistical data about the books.
It literally shares passages verbatim
So does any site that quotes the book. Just being trained on a work doesn’t give the model the ability to cite it word for word. For most of the books in this set you wouldn’t even be able to get a single accurate quote out of most models. The models gain the ability to cite passages from training on other sources citing these same passages.
It shares popular quotes from books, it can’t reproduce arbitrary content from a book. The content needs to be heavily duplicated in the training data to stick around (e.g. from book reviews), and even than half of it might still end up being made up on the spot.
Also request for copyrighted content will be blocked by ChatGPT and just receive the stock “I can’d do that” response anyway.
If you have some damning examples that show the opposite, show them.
Being blocked by ChatGPT just means that the interaction layer you see doesn’t show the output, not that the output wasn’t generated.
Everything you see that’s public facing and interfacing with an AI is an extreme filtering layer for what is output. There’s tons of checks that happen to ensure that they don’t output illegal content or any of a million other undesirable things.
I’m too lazy and care too little but you can basically get it to roleplay as a book expert or something and to “remind” you of certain passages. It gets around the filter pretty easily, that’s how jailbreaks work.
That’s maybe an issue. I mirror speech a lot, though. How large are the passages?
That claim is disingenuous at best, and misinformed otherwise.
“I’m not reselling your book, I am selling a machine that holds a mathematical formula that partly represents your entire book word for word and can reprint it on command!”
I mean, yeah? They were running to a concrete description. That is not valid. My brain has most of Terry Pratchett’s works.
LLMs can't reprint their entire training data on demand. They rarely even remember quotes.
Don’t bother shouting into the AI misinformation void.
People aren’t going to put down their pitchforks and torches to brush up on basic ML principles and it’s just going to frustrate you engaging.
It’s going to be a non-issue within 24 months anyways.
No matter how the OpenAI court cases land, the writing is on the wall that the next generations of models are going to be built on the backs of synthetic data, which is inherently without copyright.
At best rulings against OpenAI mean a secondary market emerges in China for repackaging copyrighted data into synthetic data of equivalent value to help buffer SotA synthetic data in avoiding model collapse.
It’s not even going to end up amounting to a minor speedbump to progress by the time the court cases are finalized.
Let the armchair activists rant and rage and tire themselves out worrying about a fabricated version of reality, and just focus more on staying informed about actual reality for yourself when all this passes.
It will be years before people eventually drop the bias against AI we self-instilled from shortsighted Sci-Fi over the past few decades, and until then the average person online will be irrationally upset about something related to the tech. Might as well run themselves ragged over the misinformed “it just remixes copyright” in the meantime.
On the one hand, I agree with your estimation of how things will go overall.
On the other hand, though, I think there's value to be had in pushing back against the misinformation whenever it comes up. I don't think AI is going to be hindered by it in the long run, but it's possible that in the short run it's going to kill interesting projects and harm some of the people who are experimenting with it.
And I have seen technologies that have suffered from longer-term difficulties once the zeitgeist turned against them despite having technical merit. There are useful applications for NFTs to be had out there, for example, but just try mentioning them when the opportunity arises and see what sort of reaction you get.
Yes, though to be fair these things often move in pendulum swings back and forth and that’s a natural component of any system finding equilibrium.
Just as AI has many detractors raised to fear HAL or the Terminator and approaching any news of what’s perceived as the existential threat they were warned of with hyper caution, there’s also a ton of hypemasters packaging up snake oil with AI having become the new ‘quantum’ to slap on a pile of crap and claim is gold.
The two are going to balance out.
NFTs were hyped to shit on numerous get rich quick pyramid schemes, and a number of companies that were jumping on the bandwagon to try and catch a wave rightfully abandoned what was going to be a terrible idea (i.e. Square Enix).
That said, the technology isn’t going anywhere and I’m sure we’ll see peer to peer cryptography backed exchanges of goods and services continuing to work their way into future products where the technology makes sense on its own merits as opposed to hype cycles.
The utility of AI, and specifically LLMs, is so astounding right now even in its infancy that it’s not going anywhere no matter where public opinion sits. It just won’t necessarily be used as a selling point, like for a new Coke flavor. Which is ultimately going to be a good thing.
I agree that misinformation tends to be bad, and I do have legitimate concerns that the feverous anti-AI crowd is going to end up cutting off their nose to spite their face driving a technological revolution behind closed doors of international conglomerates rather than open access, but at a certain point pretty soon this ship is going to be out of anyone’s control, and just as the DCMA doesn’t actually prevent me from downloading The Matrix right now vs in 2000 outside of a few extra hoops, the likely eventual “let’s try to handicap AI Act” is probably not going to prevent me from running model weights published in Israel or Japan on a local GPU.
I used to get more stressed about the rhetoric online, but it’s reached a point where it’s clear 90% of people aren’t looking for facts or understanding, they are only seeking confirmation bias and down voting anything that doesn’t deliver it.
In that climate, why waste our time? Discussions where one stands to learn through contribution and formulating a comment are still probably worthwhile, but a lot of discussion of AI in more general forums have honestly just turned into tantrums where no one wants to have their outrage party rained on.
It’s become the equivalent of explaining the science of immunity to antivax crowds.
First: There are mechanisms to opt out (robots.txt and meta noindex)
Second: There is some foreknowledge on the part of the web author. Even in the early days of the web — before you could’ve predicted the concept of search engines — in order to distribute anything you had to understand the basics of hypermedia, among which is the idea that anything can link to anything else and clients can be users or machines alike.
Third: Even though you are correct that search engines are tokenizing text and doing statistical analysis to recombine the tokens into novel forms in order to rank against queries, those novel forms are never presented to the user. Only direct quotes. So a user never gets a false reference to the supposed content of a page (unless the page itself lies to crawler requests).
Fourth: All of the technical points above are pretty much meaningless, because we are social creatures and our norms don’t stem from a mechanical flow chart divorced from real-world context.
Creators are generally okay with their content being copied into search DBs, because they know it’s going to lead to users finding the true author of those words, which will advance their creative pursuits either through collaboration or monetary support.
Creators are complaining about content being copied into LLMs, because their work will be presented out of context, often cited incorrectly, keep people away from the author of those words, and undermine the lifeblood of their creative pursuits – be it attracting new collaborators or making sales.
Whether it technically counts as IP infringement or not under current law? Who really cares? Current IP law is a fucking scam, designed to bully creators out of their own creations and assign full control to holding companies who see culture as nothing more than a financial instrument to be optimized. We desperately need to change IP law anyway – something that I think even many strident “AI” supporters agree with – so using it as a justification for the ethics of LLMs reveals just how weak the group’s position truly is.
LLM vendors see an opportunity for profit, if they can get away with it. They are offering consumers a utopian vision of infinite access to content while creating an IP chokepoint that they can enshittify once it blows past critical mass. It’s the same tactics the social media companies used 15 years ago, and it weighs heavy on my heart that so many Lemmy users are falling for it once again while the lesson is still so fresh.
Everyone’s a fan of fair use until it’s their work that is transformed.
Either we make all art a common freely licensed good and pay artists a flat solidarity wage to feed them, or we don’t do that and keep it how it is, but having a loophole exception for some AI corporations is not the way to go.
Absolutely agree
Any AI model that uses publically available information for training should be open source by law.
This business where corporations (that includes authors, who are published by huge corporations) fight over who “owns” ideas is assinine. When it comes down to it, this is a fight about money being wrapped in an argument about “ideas.”
AI models were developed with the collective knowledge and wisdom of society. They’re like libraries and should be public like libraries. OpenAI, Google, all those fucks should be forced to open source their models, end of story.
You can do this open source right now
Trick is educating the octogenarians in the senate to understand any of what you just wrote.
One less to educate now. Hopefully replaced by someone that doesn’t need diapers.
Yup! My ideas about what should happen are so far removed from what will actually happen they could be Planet X.
But that doesn’t make me wrong, dammit!
That doesn’t make sense. The “source” of the AI model is the publically available information, which the creators have no right to redistribute.
The rules of Open Source simply do not work for AI models. You’d have to come up with some other rules.
Yeah, it ought to be owned by the people who contributed the work that trained it. But that’s socialism. … No really, that would literally be socialism.
My friend, there are already numerous open source models out there. It’s a thing.
The whole legal situation around AI models isn’t clear and common Open Source licenses are an ill fit for them because you aren’t distributing the source, but just a binary blob. You can’t just take any random accumulation of data and slap a Open Source license on it, especially when that accumulation is the result of proprietary data, incompatible licenses and all that.
Most people don’t care and just remix everything as they please, but just because you can download for free something doesn’t make it Open Source. Furthermore a lot of the models exclude commercial use or otherwise restrict the use in ways that are incompatible with the Open Source definition.
Has any of the model made it into Debian yet?
Oh, okay. Well you should tell IBM, they’re clearly confused.
They are, just like a lot of other people. For example they cite LLaMa 2 as a popular open source model, while it restricts commercial use.
What do you define as “source” for an AI model? Training code? Training data set?
I’d say they should have to follow the most-restrictive license of all of their training data, and that existing CC/FOSS licenses don’t count because they were designed for use in a pre-LLM world.
It seems like a pretty reasonable request. But people like free stuff, and when they think about who will get screwed by this they like to imagine that they’re sticking it to the biggest publishers of mass media.
But IRL, those publishers are giddy with the idea that instead of scouting artists and bullying them into signing over their IP, they can just summon IP on demand.
The people who will suffer are the independents who refused to sign over their IP. They never got their payday, and now they never will either.
I think we just need to ban the ability to copyright any AI output. Unless you can prove you created, and or paid for the rights for every piece of training data, I don’t see how it’s fair. Even then, there are still arguments against letting AI create IP.
The people I’m seeing outraged are artists and authors who did not sign their ideas over for public access or for disingenuous use. not a faceless publisher with cloth bags and dollar signs painted on them. Also I don’t think you understand what public and private ownership means. A person is allowed to privately own their own creation. They don’t owe that to the world. The world isn’t entitled to it.
do they also complain when their books are used to train wet networks in public schools? those networks are also later exploited by corporations who dont give back the writers. hmmmmmmm
They do get paid for that, however. They get a share of the value of each book sold. Those schools are paying for the books.
There is also the catch that those wet networks are of finite lifespan and are output throttled. This limits the losses caused. A lot of authors also consider improving those networks a big part of why they write.
It’s the difference between someone hand drawing a Micky mouse birthday card for their sibling, and hallmark mass producing them for sale. The former is considered acceptable, the latter is grounds for a law suit.
There’s an idea by Barath Raghavan about an AI dividend that companies pay each netizen a share for the data they use to train these models.
I am into this idea if companies can’t even do a simple opt-in mechanism.
Does this fall under fair-use part of copyright?
It hasn't been tested in court yet but I don't see why it shouldn't.
I don’t see why it should.
The creation of the AI model is transformative. The AI's model does not contain a literal copy of the copyrighted work.
No, but the training data does contain a copy. And making a model is not criticising, commenting upon, or creating a parody of it.
That list is not exclusive, it's just a list of examples of fair use.
The training data is not distributed with the AI model.
Yes, it’s a list of quite similar ways of commenting upon a work. Please explain how training an LLM is like any of those things, and thus, how Fair use would apply.
I'm not saying that training an LLM is like any of those things. I'm saying it doesn't have to be like those things in order for it to still be fair use.
Pay for every bit of information you’ve read and regurgitated on exams.
AI is not human and should not be treated like a human
It’s not. The humans that trained it (assumably) purchased the material used to train it. What’s the problem?
The use of the material to create a commercial product as well as the reality being that the humans training it never buy the data on an individual level.
Authors Guild, Inc. v. Google, Inc. decided that it is fair use to scan books and make large parts of them available verbatim on the net. What AI does is far more transformative than that, as very little of a book can be reproduced verbatim with AI (e.g. popular quotes), you really just get “knowledge” from the books. The sources are however lost in the process, unlike with Google, which by itself however also makes it difficult to argue for copyright violation, since you can’t point at what was actually copied.
The training argument is probably going to come up dry by the time the court works its way through expert testimony, as the underlying argument for training as infringement is insane.
But where OpenAI is probably in hot water is that torrenting 100k books in the first place runs afoul of existing copyright legislation.
Everyone is debating the training in these suits, but the real meat and potatoes is going to be the initial infringement of obtaining the books, not how they were subsequently used.
This is no different than every other capitalist enterprise. The whole system works on taking a public resource, claiming private ownership of it, and then selling it back to the public for profit.
First it was farmland, then coal and minerals, oil, seafood, and now ideas. Its how the system works and is the whole reason people have been trying to stop it for the past 150 years.
The people making the laws are there because they and/or their parents and/or grandparents did the exact same thing. As despicable and corrupt as it is you won’t change it by complaining and no-one is going to make a law to stop it.
God damned right. Every “new” thing tends to be stolen. In more event history, its stolen from other capital, or from innovation with a free license, rather than artwork. Publishers might actually be able to make a problem out of this.
Ok so it’s been stealing art now it’s coming for authors. At what point do we hold the coalition who started this shit culpable for numerous accounts of plagiarism?
TIL “culpable” is an English word too. Culpable means guilty in Spanish and I thought you were a Spanish speaker doing spanglish. Now I know you’re just a man of culture.
Curious if the AI company actually bought those books or if they just came across them by pirating.
Oh, they’re 100% pirated. Sorry this isn’t open, but the preview should give you enough information. The database is available elsewhere, IIRC. www.theatlantic.com/technology/archive/…/675363/