Speculated to be responsible for the "AI restaurant-menu gore" that made the rounds on social media. Restaurants AI-generate the entire menu sheet from whole cloth, and while it looks "good" at first with modern image models, repeated edits to the menu text/layout transform the passable food "photos" into trypophobia-inducing horrors. (And somehow they don't notice or care)
Notably, the recent ChatGPT Images 2.5 announcement claimed to address this issue specifically ("multi-turn editing consistency")
With local models it's always been easy to edit only a part of an image (inpainting), either by painting a mask or automatically using a small segmentation model. It's actually pretty strange that commercial models have resorted to redrawing the entire image even in cases where the requested change is localized.
Depends. With or without quality control? Not any more than in a society that rewrites all of its knowledge over and over using using humans. We even have a term for doing so without quality control: A game of telephone.
Though you might well argue that the risk of us dropping the quality control gets higher the cheaper it gets to churn things out with AI.
Definitely already happening with many c-suites.
We unleashed Nyarlatothep…
“I do not recall distinctly when it began, but it was months ago. The general tension was horrible. To a season of political and social upheaval was added a strange and brooding apprehension of hideous physical danger; a danger widespread and all-embracing, such a danger as may be imagined only in the most terrible phantasms of the night. I recall that the people went about with pale and worried faces, and whispered warnings and prophecies which no one dared consciously repeat or acknowledge to himself that he had heard. A sense of monstrous guilt was upon the land, and out of the abysses between the stars swept chill currents that made men shiver in dark and lonely places. There was a daemoniac alteration in the sequence of the seasons—the autumn heat lingered fearsomely, and everyone felt that the world and perhaps the universe had passed from the control of known gods or forces to that of gods or forces which were unknown.
And it was then that Nyarlathotep came out of Egypt”
This reminded me of a story/vignette which my fallible memory tags as by someone like Twain or Poe, about an automatic rewriting machine. It was put to work on the same passage with an ever higher intensity setting, becoming more and more gothic to the point of parody -- iirc the final output was verse including the line "to atoms dash the doubly damned".
No memory, but searching says it was Lewis Carroll in Photography Extraordinary, published in 1885:
The Milk-and-Water School
Alas! she would not hear my prayer!
Yet it were rash to tear my hair;
Disfigured, I should be less fair.
She was unwise, I may say blind;
Once she was lovingly inclined;
Some circumstance has changed her mind.
The Strong-Minded or Matter-of-Fact School
Well! so my offer was no go!
She might do worse, I told her so;
She was a fool to answer "No".
However, things are as they stood;
Nor would I have her if I could,
For there are plenty more as good.
The Spasmodic or German School
Firebrands and Daggers! hope hath fled!
To atoms dash the doubly dead!
My brain is fire--my heart is lead!
Her soul is flint, and what am I?
Scorch'd by her fierce, relentless eye,
Nothingness is my destiny!
It would have been interesting to try something closer to the movie poster example; say have the AI rewrite the passage with the room toggling between the initial light-mode decor and a dark-mode decor, then back, repeatedly. Or even change the woman's name to "Mrs. Smythe" and back.
As the article explains: “[…] In other words, if you instruct a model to change a single word in a paragraph of text, it can almost always handle the task with no collateral damage.“
In the case of the movie poster, it probably doesn't help that the image was apparently stored in a lossy format. So after a few iterations the LLM can't tell the JPEG artifacts from the relevant image details and starts adding wrinkles to the faces and texture to the clothes etc. until it all goes to hell...
The existence of attractors (e.g. "stable at iteration 31") for AI models is one of the weirdest things in AI land. They are a wonderful space to explore, fascinating and full of horrors.
Meanwhile, my hat's off to the (currently) last comment on the page: "I like how the shear-sheathed legs still earned their keep — probably load-bearing for Mrs. Regan"
> PS. For obvious reasons, this post will fail Pangram tests.
Actual pangram to me says it's 100% human. Current pangram can flag different passages differently, but these are all flagged human by it, even the LLM written ones.
Notably, the recent ChatGPT Images 2.5 announcement claimed to address this issue specifically ("multi-turn editing consistency")
reply