[Survey] Can you tell which images are AI generated? (forms.gle)
from popcar2@programming.dev to technology@lemmy.ml on 05 Oct 2023 18:27

Hey everyone. I made a casual survey to see if people can tell the difference between human-made and AI generated art. Any responses would be appreciated, I’m curious to see how accurately people can tell the difference (especially those familiar with AI image generation)


threaded - newest

virku@lemmy.world on 05 Oct 2023 18:41 next collapse

Question 11 didn’t have a correct answer.

popcar2@programming.dev on 05 Oct 2023 18:44 collapse

Fixed, thanks for reporting.

astanix@lemmy.world on 05 Oct 2023 18:51 next collapse

Well I got less than 50%, maybe I’m an AI. What is even real anymore!?

Danar@discuss.tchncs.de on 05 Oct 2023 18:51 next collapse

Questions 12 and 20 seem to have incorrect answers. The correct answer is “no”, but the comment says they were created by DALLE-3

popcar2@programming.dev on 05 Oct 2023 18:56 collapse

Fixed both right before seeing this comment, I’m really not awake enough for this :P

rbn@feddit.ch on 05 Oct 2023 18:58 next collapse

Questions 11, 12, and 20 we’re all graded incorrectly as the correct answer contradicts the specified source.

Based on the automatic grading I got 12 out of 20. Based on the feedback / comments I got 15 correct.

I’m quite proud that within the human photographies I classified 100% correct but I guess it’ll be impossible for me if the algorithms just improve a tiny little bit.

popcar2@programming.dev on 05 Oct 2023 19:01 collapse

They were fixed after posting but that may be after you opened the link, answers should be good now.

candyman337@sh.itjust.works on 05 Oct 2023 19:16 next collapse

I take issue with this because the devil is usually in the details with ai images and these are all low rez jpgs making it harder to tell with some of these.

SzethFriendOfNimi@lemmy.world on 05 Oct 2023 19:28 next collapse

Right? Are hands visible? Because those are a bear to get right

popcar2@programming.dev on 05 Oct 2023 19:36 next collapse

Unfortunately it seems like google forms resizes the image to fit the forms. If I had known this before I would’ve used something else, but oh well. I’ve stretched the images as far as they can go now, which seems to be around 740x740.

biddy@feddit.nl on 06 Oct 2023 19:27 collapse

True, but low rez web jpegs is a huge part of the market for images. AI will replace stock photos and that’s incredibly disruptive on it’s own.

baggins@lemmy.ca on 05 Oct 2023 19:27 next collapse

The link doesn’t work.

davidgro@lemmy.world on 05 Oct 2023 19:32 next collapse

Got 9/20.

That was a good selection of images, quite tricky.

I’m proud of getting both the LEGO minifig ones correct.

qdJzXuisAndVQb2@lemm.ee on 05 Oct 2023 20:36 collapse

Another 9/20er here. I did feel like I was guessing a lot, it was almost satisfying to get such a midway score.

cyberwolfie@lemmy.ml on 06 Oct 2023 20:51 collapse

I also got 9/20, feeling certain about only a handful, and completely thrown off by others. Since all questions were yes/no, expected score would be 10/20, so my score correctly reflects that I had no real idea what was AI-generated or not. I expect the average score to be close to 10/20, skewed somewhat higher by those who might have a keen eye for some telltale signs of AI-trickery.

KoboldCoterie@pawb.social on 05 Oct 2023 19:35 next collapse

Well, I did alright on the people, but I got almost all of the landscapes incorrect. I’ll also admit to guessing on a number of them; if I had to explain exactly what I was basing my answer on for ones I said were AI, I’d have missed 2 more, because I just couldn’t see anything that looked off, it was just a hunch.

ma11en@lemmy.world on 05 Oct 2023 19:40 next collapse


FiniteLooper@lemm.ee on 05 Oct 2023 20:39 collapse

11/20, I’m surprised. I thought I would do better at seeing that standard AI look and feel, but those other art aunties really threw me off. I wasn’t aware some of them could be generated from an AI so well!

apprehensively_human@lemmy.ca on 05 Oct 2023 20:57 collapse

AI generated photographs, or I guess AI generated images that are trying to appear photoreal are usually pretty easy to spot.

The stylized artistic ones are often really hard because you could have a human work of art that is trying to purposefully mimic an AI generator, which looks like the case with Strawberry Taiyaki Cat

starman2112@sh.itjust.works on 05 Oct 2023 19:46 next collapse

Wow, I got exactly half of them right. I reasoned my way through four of them–I assume AI can’t handle refraction or reflection very well, so I reasoned that 1, 2, and 12 were all fakes. On 17, the scarf is too detailed to be a Lego piece.

Every other guess was vibes based, and on my vibes-based guesses I got 6/16

Noved@lemmy.ca on 05 Oct 2023 19:57 next collapse

The one that got me was definitely the fruits, I didn’t realize that AI was able to generate decent text yet lol

popcar2@programming.dev on 05 Oct 2023 23:43 collapse

DALL-E 3 is the only model that gets text right. It usually yields consistent results but can still jumble on words if you ask it to say too much. It’s a big step forward regardless.

<img alt="AI generated photo of a cat saying “I’m king of the world!”" src="https://programming.dev/pictrs/image/4da79342-8857-4c47-8036-95b5033f1633.jpeg">

davidgro@lemmy.world on 06 Oct 2023 00:58 collapse

That’s incredible. I’m usually surprised when there’s a single correct word.

FooBarrington@lemmy.world on 05 Oct 2023 20:23 next collapse

Cool test, thanks for putting this together! I got 8/20 - this essentially proved to me that for many cases, AI generation is not really distinguishable anymore.

HipPriest@kbin.social on 05 Oct 2023 20:33 next collapse

8/20 here. Proud of getting some right. Really shocked about the answers to some!

A tricky test indeed

ram@bookwormstory.social on 05 Oct 2023 20:46 next collapse

12/20, most of my mistakes were in saying human-made work was AI generated.

gullible@kbin.social on 05 Oct 2023 21:07 next collapse

Detail plays a huge role in determining AI, and many of the pictures appear compressed which makes that criterion… difficult to consider. I’m not surprised that I got half right, regardless. The man on the bench really got me, why is his ankle thread-thin?

slazer2au@lemmy.world on 05 Oct 2023 21:23 next collapse


I have used SD so I can see some of the ai tell tale signs but haven’t used Dall-E at all.

hulemy@ani.social on 05 Oct 2023 21:36 next collapse

9/20 damnit, thought I’d be better at this

ArtyTester@artemis.camp on 05 Oct 2023 21:39 next collapse

I got all the “real life” pictures correct, but most of the drawings and paintings were in line with straight guessing.

There are things that stick out as “wrong” on a picture of life, but in a drawing or paining the question is “is this wrong or just what the artist decided to do?”

muhyb@programming.dev on 05 Oct 2023 22:08 next collapse

I got real life ones correct, almost all paintings too (10 was tricky) but charcoal drawings are kinda impossible to guess.

reflex_aliens@lemmy.ml on 05 Oct 2023 22:17 next collapse

3/20. I almost got everything perfectly backwards!

Codename_goose@sh.itjust.works on 06 Oct 2023 00:46 next collapse

11/20 I feel slightly upset by the fact that it’s still a 50/50 coin flip for me which was which.

mindbleach@sh.itjust.works on 06 Oct 2023 01:01 next collapse

Some of these seem unfair because - if they’re real images, they’re images that resemble common errors, and if they’re generated, they’re examples of those errors being situational enough to look ambiguous. I can tell you what I’m looking at in each image. I can tell you where I’ve seen that misplaced or overused in a ton of generated images. But I can also tell you what humans tend to scribble out that might’ve been picked up by machines without me noticing, and I can explain some that-looks-suspect locations as mundane physical artifacts.

You could argue that’s the point - demonstrating how far the technology has come in basically one year. But there’s some cases where damn near anything is plausible, so long as it’s locally sensible. Any close-up of a face might be from the “this person does not exist” kind of network, because with eight billion people on Earth, yeah, I’ll believe that’s a guy. But if you show me three pictures of the same alleged guy, I’m gonna know whether it’s a real dude or a machine hallucination. Nature photos are similarly hard because nature’s kinda anything-goes. Drawings, even moreso. There’s not much difference between an AI going nuts on waterfalls because it has poor segmentation and a human who wanted to draw a clusterfuck of waterfalls.

Here’s what I’m looking at in each image. Her thumb’s too good behind the glass, even if her fingernails are a little weird and the bench seat’s not quite the same color on either side. His glasses are the only thing that’s a little off, especially the gray-looking hairs on only his right temple, even though both could be perspective. His everything’s too smooth; if this isn’t generated then someone airbrushed a photo to death. Sketchy lines going nowhere and multiple approximations of a shape had me assume human over computer, but the bench’s third leg and janked-up shadow point to a computer or a shitty artist. This guy looks filtered instead of drawn, but it might just be scratched instead of drawn, and honestly his wonky hold on the book is less concerning than the other image’s bench. Perspective’s all fucked-up and I’m unsure why the mouse is in a bucket, but the most computery parts are the fine detail in distant waves and up-close spray, because the high frequency doesn’t match the drawing style. Except the next image has detailed asymmetrical elements and some smoke in front that only makes sense locally so I assumed these were human / generated pairs and marked the boat one as more-likely human. Fine stripey detail and repetition are suspect, as mentioned, make enough sense in this context that the distant foliage is almost more concerning. Rough painting originally had me mark this as human, versus the previous image, but where fine details appear (e.g. bottom left corner) they don’t make any sense for a human to have focused on. Either a person did a shit job drawing those horses and really scribbled out a city, or this is exactly the sort of disordered localized detail some models add. (Honestly the scale birds and bottom-left white scribble are the only things that look like ‘sloppy human’ versus ‘sloppy computer.’) God rays on craggy waterfalls are the hardest call because humans might also draw this geological uncertainty; I marked it as generated because the smaller fall to the right finishes plausibly but starts from nowhere. Soft glow forest mountains are a generated cliche at this point. Monotonic crisp layers are not. Only the English text and rounded speech-bubble tail are tells at this point. An ice cream cat seems like the kind of dumb shit you’d ask an AI to do, but this is a tough call: there’s three different kinds of “strawberry” here and they’re not bungled together, but the pay and cookie placement seem bizarre in light of the rim of the fish-cone, and the placement of the beads is either cause for criticism of a human artist or shockingly flexible for a network. Lego image one could go either way. But Lego image two has cliche composition, an impossibly detailed plastic scarf, and asymmetric nonsense prints on her legs. Cat one is painted with consistent brushstrokes on everything but the whiskers. Cat two is either a painting filter or a person drawing badly from a photo reference. Cat three is the same warm-glow cliche that’s easy to do on a computer, and if a human did that with actual paint, bravo.


Everything photographic, I nailed. You picked some some lackluster human art.

TonyTonyChopper@mander.xyz on 06 Oct 2023 04:23 next collapse

12/20, these new systems are way too good at what they do. With drawings it’s basically impossible to tell unless there are obvious mistakes. And the mugshots are spot on too.

RocketBucket@lemmy.ml on 06 Oct 2023 04:52 next collapse

15/20 - Nice survey, was quite tricky

Mr_Dr_Oink@lemmy.world on 06 Oct 2023 05:29 next collapse


Cool survey.

Some of the similar ones i got the wrong way around, but i was quite happy with my answers for others where i was quite confident there were (to me) obvious indicators of AI and got them right.

TheBlue22@lemmy.blahaj.zone on 06 Oct 2023 05:58 next collapse

10/20, got tricked by the horse in one of them, it looked really messed up, like something AI would make.

I guess the artist cant make horses

mindbleach@sh.itjust.works on 06 Oct 2023 15:51 collapse

That artist also forgot reins. And the front wheels of the carriage are goofy. And the city is a bunch of squiggles. And there’s a bizarre oversaturated smear of farmland in the distance. It’s the sort of human drawing that makes people go ‘did AI draw this?’ because a year ago they would’ve just said ‘this kinda sucks.’

The psychedelic waterfall one has the opposite problem, where the tree at left immediately had me go ‘that’s a robot,’ but it is also how humans draw when they’ve done quite a lot of drugs. Anywhere besides a landscape it would be inexcusable. But there’s every reason you might want to draw a tree, that way. An anime character eating ramen - not so much.

Cwiiis@kbin.social on 06 Oct 2023 08:10 next collapse

10/20 but I'm a little annoyed that what looks exactly like a panel from Berserk is apparently AI generated... Feels like the training data might just be replicated in entirety there, either that or someone asked it to generate "Guts from Berserk" 😛

Gullible@sh.itjust.works on 06 Oct 2023 16:39 collapse

You can always notice AI based on the random belts on armor clad figures. It’s a neat belt, but why did it begin and end at the center of their chest?

PrimalHero@kbin.social on 06 Oct 2023 08:45 next collapse

6/20 I am bad at this

ar0177417@lemmy.world on 06 Oct 2023 09:13 next collapse


Zeragamba@lemmy.ca on 06 Oct 2023 23:52 collapse

same score here

southernwolf@pawb.social on 06 Oct 2023 10:39 next collapse

14/20, nicely done! I will say, I maybe would have wanted to see more AI results than just from Dall-e 3, though given I still missed 6 of the, that speaks very highly of Dall-e 3’s capabilities. But some midjourney and SDXL images would have made for a wider guessing selection too.

MellowBright@lemmy.ml on 06 Oct 2023 18:02 collapse

I agree. I’ve only been using Stable Diffusion so I was surprised to see it’s lack of presence. I feel like half to ones I got wrong were because I’m too used to SD’s quirks.

Dall-e has apparently gotten really damn good lately. The talking fruit in particular. Being context aware enough to create accurate speech bubbles is crazy neat. One huge step closer to AI comic books.

Jimmycakes@lemmy.world on 06 Oct 2023 20:06 next collapse

I got 13.

crowfx@crowfx.web.id on 06 Oct 2023 22:29 next collapse

Got 7/20.

Those images are actually hard to distinguish for me. An eye opening experience.

motor_spirit@lemmy.world on 06 Oct 2023 23:41 next collapse

7/20 I think I’m supposed to seppuku or whatever now

Traegs@lemmy.world on 07 Oct 2023 00:49 next collapse

I got 12/20 and most of the ones I got wrong are the ones I was second guessing myself on.

NaturalViber@lemmy.world on 07 Oct 2023 01:59 collapse

I got 14/20. I am the real human that has gotten the high score. I am not a bot. Real human only.