TITAA #27: Cave Poems and Strange Horticulture
TITAA #27: Cave Poems and Strange Horticulture
Editing AI art with new tools - 3D image generation - story arcs - data & text vis
Last month I mentioned I had visited Chauvet caves in VR with the Dawn of Art app. Right after that, Jurie Horneman posted this fab LRB story by Philip Terry about a code breaker writing poetry with cave art symbols. One Jean-Luc Champerret had been working for the Resistance in 1940, living at a chateau in the Dordogne. His resistance cell asked him to visit the Lascaux caves discovered by “four schoolboys and their dog Robot” (hah) in case they could be useful to the Resistance. He examined markings near the paintings with the eye of a code breaker, and attached possible meanings to them.
A row of vertical lines might perhaps represent spears, or a forest, or even rain. An upturned ‘v’ sign (or two such signs, one on top of the other) might represent mountains, or huts. A line of dots might represent people, or a journey, or faces, or stars. A row of horizontal lines might represent mist or night. A sign resembling an upturned question mark might represent a club; a sign resembling three-quarters of a circle with a dot in the middle, an eye; a meandering line or group of lines a river, and so on. ‘Le signe,’ he remarks at one point, ‘n’est jamais arbitraire.’ These signs, he argues, could be linked together to form primitive sentences, or to carry messages if scratched on a stone or a piece of bark, or in the earth with a stick.
He further speculates that the 3x3 grids with marks might be a notation system for alerts, and might have evolved further to hold a kind of poetic visual writing form. He then tries himself to write in this form.
Champerret wrote (if we include his variants) more than six hundred of them. His method can be split into five stages: a) he fills the grid with signs; b) he ‘translates’ the signs minimally into French; c) he writes through the first translation, adding connector words, so that the poem reads more easily in modern French, translating the three by three structure of the grid into three stanzas of three lines each; d) he writes a first variation on the poem, elaborating some of the lines, and embellishing the detail, as a shaman or an oral poet might vary the bare outline of an inherited story; e) he repeats this process, continuing to elaborate and embellish the original, all the while maintaining the stanzaic pattern, although the lines are progressively indented to echo the original three by three structure as it is distributed laterally as well as vertically across the page.
For example, for the cave poem above, steps (b) and (e) might be:
B)
eye bison sun
horns bison spears
legs bison clubE)
The white eye
of the black bison
is like a star at nightthe curved horns
of the black bison
are like sharp spearsthe thick legs
of the black bison
are like heavy clubs
He experimented with many other grid form poems. With his paper scraps was found a letter from director of the Musée de l’Homme telling him, “Your work is pure fantasy” and that “The study of Upper Paleolithic parietal art is a science, and should be left in the hands of specialists.”
So sad. Treatises to be written about outsider perspectives. But this story is so awesome.
AI & Gen Art
Diffusion art only got more astounding this past month. I feel like some kind of milestone has been reached, and I think others feel the same. In fact, while writing this, Jack Morris (@jxmnop) posted an excellent overview of the evolution of the past year. Just go read that.
In just the past month, I have seen so much neural artwork on Twitter that is awesome. Here are some I have loved (credits follow):
(top row) @neuralbricolage, “`Begin at the beginning,' the King said gravely, `and go on till you come to the end: then stop.'" (No idea if this was actually the prompt to generate it.)
(second row) @quasimondo, “Ghost Riders in the Sky” (no idea if this was a prompt)
@ak92501, “rainy night at hogwarts | artstation | matte painting"
@ted_underwood, “the uncanny valley, tilt shift photography”
(third row) @RiversHaveWings, "the winds of madness by Gerardo Dottori" (yfcc_2 CLIP guided diffusion)
@logodaedalus, “Woodcut print of an alchemist creating a homunculus, I can't believe the detail”
@images_ai, “wilting” (from a starting photograph, disco diffusion)
If only there were a way to study the prompts that have been used as generative input over the past year or so! but it would be a hard scraping problem without hashtags on them anymore. Anyway, what people want to draw is as interesting to me as the output surprises. For instance, this is lovely:
As I was writing that, I discovered some folks have been collecting prompts for AI art and using them to generate new ones with GPT-3. Taking the human out of the loop! Here is @NeuralismAI’s prompt generator, based on @remi_durant’s madlib templates prompt generator. Related, VQGAN template prompts from dughogan. Here’s a handful from @NeuralismAI’s prompt gen colab:
spring refracting reservoir grouse settee | by Robert Henri | vaporwave iridescent boring London shark turret | by Paul Cezanne | abstract massive realistic ocean persian cat apricot | by Raphael | postmodern deadly looming rivulet retriever cantaloupe | by James Gurney | dutch golden age
Here’s a stab at a realistic if non-massive “ocean persian cat apricot” using Katherine Crowson’s fast Diffusion notebook (CC12M):
A few more image gen tools: Python Text-to-Image Tools, an annotated how-to and colab to run things, via Jacob Garbe. There is a lot of documentation, which supports the rather huge set of input options you are given.
Instance-Conditioned GANs from Facebook - generate alternate images from input source image. You can provide a pre-determined class like “castle” as a guide as well. It’s really fun. (Related, but using ruDALL-E, Looking Glass v1.1 with colab.)
Edit Your Image Gen: For those wanting UI to edit image contents in a form other than colabs, there’s now ProsePainter, from Joel Simon and team who did ArtBreeder. You can use masking, sketching, and prose to alter or create artworks. It uses ESRGAN (not the currently popular diffusion models), from the github peek I had. It’s a bit tricky to get right but has so much possibility. I think it’s being pounded this weekend, because it kept telling it was “waiting to start.” There is a useful video tutorial here.
Related in terms of UI concepts, NVIDIA’s GauGAN 2 is also kind of hard to use well. (See Janelle Shane’s diagnostic experiments in her article here.)
3D animated diffusion art: Outstanding Lovecraftian haunted house generated animated art render by @gandamu_ml.
Related, fantasy-themed short animation: Journey Across the Kingdom by @somnai_dreams. Also, see this video and thread by @xsteenbrugge on how he used different models to create a 3D diffusion animation. (It’s complex and uses Google’s frame interpolation lib.)
And finally, just today: Disco Diffusion v5 notebook has added 3D animation, from collaboration between @somnai_dreams and @gandamu_ml. (Via Jacob Garbe again! Who I owe a coffee!)
Artist Uses AI To Perfectly Fake 1970ies Pulp SF Book Covers and Titles.
CLIPasso: Turn images into SVG strokes of different levels of abstraction. Useful for lots of things. The images you feed it have to be in a strict format right now, which I don’t want to mess with this weekend.
Image Captions and Questions: BLIP is pretty incredible. It is very smart at image content recognition, for caption generation, visual content question answering, and search (image-text matching). You can try a demo of captioning or question answering with Gradio. When I tried it with a diffusion generated image from the prompt “A painting of a complicated castle with multiple levels and towers, by Xul Solar. Trending on artstation” (see last month’s newsletter), it identified it as “a painting of a castle in the middle of a mountain,” which is pretty damn good. It is surprisingly excellent at captioning paintings, although failed to get a bird out of an abstract Miró I tried.
3D City Blocks: Block NeRF from waymo - generating entire 3d city blocks from some still images. The video of driving around in SF is nice.
3D Crypto Mushrooms: Drawing Generative NFT Mushrooms with Three.js, including generating their names, by @ferluht.
3D Fashion: Infinite Passerella, an infinite fashion show by Lusion made with threejs. I don’t really know what’s going on here, but I love it :) The autonomous walking outfits with odd names, strutting without people in them, are perfect.
Proc Gen: An interview with Robert Hodgin about his organic growth v02 project, the pictures are great and the works are somewhat interactive.
Particles: Interesting particle agent artworks (like slime molds?), Becoming Ooze, by Sage Jensen (@mxsage). Interactive. (Thanks @mtchl.)
Text and Narrative Gen: Narrative Device, by grad student Rodolfo Ocampo. Input 2 concepts/themes, get amazingly coherent stories from them. Even more amazing, redoing will usually give you something quite different. Here are some different “haunted” and “cat” combos — I find it interesting that the last one only implies something ghosty?
“The family had always been haunted by the ghost of their cat. It was always there, lurking in the shadows, watching them from behind the bushes. No one could say for sure why the cat had chosen to haunt them, but everyone knew that it was because the cat loved them.”
“The cat's restless movements sent a shiver down the spine of the ghost. It had been watching the cat for hours, ever since it had seen the creature descending towards the river. The ghost had been trying to ignore it, but the cat's restless movements kept it company.”
“In the small hours of the morning, Sarah woke up to the sound of a cat scratching at the door. She bolted out of bed, making her way to the door to see if the cat was outside. When she opened it, the cat was gone, but the door itself was still open. Sarah wondered what could have happened to the cat.”
Text art: Stendahl, an NFT art project by Damien Seguin that uses sentiment analysis on the sentences in Le Rouge et Le Noir to orient the shapes in the pieces. (Thanks @mattdesl.)
Museums: Museum security guards get to curate an exhibit at Baltimore Museum of Art. What a great idea.
NLP and Narrative
Papers: Conversational Agents: Theory and Applications (arXiv paper).
Books: The Hugging Face Transformers NLP book is out, finally.
GPT-N: As of Feb 9, the GPT-NeoX-20B checkpoints are available from EleutherAI.
Keyword extraction from documents via KeyBERT (code, medium post). Related: PyTextRank for SpaCy.
Story Arcs: LIWC, the software for text analysis in spreadsheet format, has added topic modeling and “narrative arc analysis.” The narrative arc tools are based on this article, “The Narrative Arc: Revealing Core Narrative Structures Through Text Analysis” (2020, Boyd, Blackburn, Pennebaker).
First, they start by using lots of words that pertain to nouns and how they relate to one another: this is called "Staging" language.
Once the storyteller has set the stage, they often use less "Staging" language and begin to use more words that signal action, and words that imply a shared understanding of who is engaged in those actions, and how those actions are transpiring, and so on. This is the language that drives a story forward: "Plot Progression" language.
Importantly, all the while, most storytellers build and release psychological tension through some form of conflict: either by having characters struggle to attain their goals, or structuring situations in such a way to where it is uncertain whether characters' goals will be achieved. Traditionally, this "Cognitive Tension" rises and then peaks around the middle-to-later parts of a narrative.
You can try their analysis tools on a large number of Gutenberg Project public domain novels on the research website arcofnarrartive.com. This is Pride & Prejudice:
Story Generation Systems: I am converting my Medium posts to github markdown; my epic rant and exploration of the story combination book “Plotto” (my most popular post) is up here: “Plotto: Generating Truly Offensive Stories Since 1928.”
Data Vis
Dimension Reduction: Another python lib option for trying to reduce dimensionality and visualize, TopOMetry, for Topologically Optimized geoMetry. Supports several visualization reduction methods like t-sne and simplified UMAP.
Text Dimension Reduction: Also, fab demo of using Observable plus HuggingFace sentence-transformers in a single web page app to do UMAP or HDBScan clustering of sentence embeddings: Sentence Embedding Visualization. (I just did this with a lot more tools and steps for work, although this does seem a bit prone to bugs with sizable data.)
Old Manuscripts: A new phenomenal historical document recreation and design recap by Nicholas Rougeux, the Iconographic Encyclopedia of 1851. His graphic and interactive design detail are surpassed by no one, he is a treasure.
Data vis of Shakespeare’s Sonnets and sentiment in Tableau, by Ann Jackson.
Another Tableau winner, Rivers of Time by Samuel Parsons, visualizes the Marvel Universe’s timelines. (Thanks to Andy Kirk.) I have in no way absorbed or understood all of this, same way I am with the complicated movie/show timelines.
Philip Bump used Clive Thompson’s Townscaper data vis tool (post about it) to draw bar charts for an article.
Which reminded me that I had never posted about Nadieh Bremer’s cute castle-like data vis of schools with and without internet access, done for UNICEF.
A paper on data vis in games (“Toward Visualization for Games: Theory, Design Space, and Patterns”) by Bowman, Elmqvist, Jankun-Kelly (2012).
Star maps with d3, a lib by Olaf Frohn and others.
Misc Data Science Links
Collective Intelligence for Deep Learning, a Survey (arXiv paper).
OpenStreetMap find, a command line tool for searching for things in OpenStreetMap. This looks easier than some methods I’ve used.
Stickyland, make cells in Jupyter notebooks float on top, synced with their inline state. Super useful looking.
Gamey-Links
Amazing Interactive Fiction and Text Gen course syllabus, taught by @ccb, @LangTechLara and found by Prithviraj Ammanabrolu.
How The Sims Became the Internet’s Most Exciting Place to Eat - The New York Times - a cute piece on food in the Sims which made me really want to play and cook.
A long Ink game example: game, ink source. Always useful to see complex stuff handled.
Homer, a new story-flow web-based editing tool.
Meet the hundreds of horse girls running Red Dead Online's kindest posse | PC Gamer, thanks to Jacob Garbe for this wholesome link.
Books
⭐ When the Sparrow Falls, Neil Sharpson (SF). If you liked the Cold War atmo of The City and the City, or generally like Cold War paranoid political shenanigans in a run- down, oppressed totalitarian country, but with illegal AIs visiting to identify illegal AI remains, this is for you! I really liked it.
Stealing Worlds, Karl Schroeder (SF). I did not find this awesome, but it’s very timely (despite being published in 2019, as was pointed out to me when I complained about the buzzwordiness): smart contracts, decentralized autonomous corps, IoT sensors, cryptocurrency, AR/MR games, game economies run on the blockchain, lines like “technology is legislation”…. well. I felt tech-splained to. But it sure feels current, and might appeal to some of you? Short-recap is that our heroine is on the run and goes off the grid into a game economy, and then it gets very political.
Children of Ruin, Adrian Tchaikovsky (SF). A solid sequel to Children of Time, which I loved despite hating spiders. This one adds octopuses/pi, and cryptic aliens with the incredibly creepy line, “We’re going on an adventure.”
Book of All Skies, Greg Egan (SF). Difficult to conceptualize (I think the picture on the front cover helped the most?), but there are these linked worlds connected by arches/hoops… an archaeologist chases a stolen manuscript on their lost history and ends up traveling to the other side of the connected worlds through a gap in space between the end points. I love portal fic, even when I can’t quite picture it.
The Victoria Vanishes, Christopher Fowler (Mystery). I had not read any of the others in this series about eccentric, geriatric detectives in London’s Peculiar Crimes Unit, but I totally enjoyed this one. Charming tour of historic pubs, weird murders, and esoterica via grump.
TV
Archive 81 (horror/fantasy, Netflix). I mean, there’s a lot in here: haunted video tapes, mental health issues, witches, cults, weird moss, ghosts, missing mothers, crazy artists, maybe time travel, human sacrifice, a demon statue, an asteroid… I still liked it. But it’s a lot.
Anxious People (mild mystery, Netflix). This is surprisingly heart-warming in a non-cloying way, maybe because it’s Swedish? It does start with a suicide, but that’s not the tone of the rest of it. Basically, what happened with a group held hostage at a real-estate viewing, during and after, and why.
Silk s1-3. (legal drama, Amazon Prime). More-ish and un-put-down-able, it reminds me of the French Spiral/Engrenages, but purely legal. Maxine Peak as the fiery Martha Costello is luminous. It does however make me say “WTF” a lot about British barristers, solicitors, judges, clerks and clerking, London, and sometimes Martha’s decisions.
Raised By Wolves s2 (sf, HBO). I am not a fan of some of the tropes this season, including the Mad Max fight scene cliché, but nevertheless, it’s interesting. Something weird every episode! Mithraism is real, you know.
Young Wallander s2 (mystery, Netflix). Still fun, maybe less dark than s1. Also Swedish, but full of British actors for some reason. And evidently not filmed in Sweden?! I watch TV for the settings, come on!
Games
PC
Paradise Killer: This is the weird pocket-universe-with-tentacled-gods mystery game I wrote about last month. I enjoyed it, but the ending features your character shooting the guilty, if the judge agrees with your evidence. Eh. I wasn’t myself convinced by one I accused, and felt a bit trapped.
⭐ Strange Horticulture: Adorable game rec’d by friends and online reviews. You’ve inherited a mysterious plant shop with a cute black cat and have to figure out which magical plant is which for your very odd customers. Identifying plants gets you new pages for your herbal, and sometimes your customers point you to caves or marshes to collect new samples. Meanwhile, there are bodies being found at stone circles, invitations from creepy secret societies, people having disturbing visions, mysterious letters arriving, hidden messages… It’s just fab. And I loved the map.
Friends who liked this game might like to browse Nicholas Rougeux’s online gorgeous recreation of Elizabeth Twinning’s 1868 Illustrations of the Natural Orders of Plants.
VR Games
Skyrim VR: I’ve been playing Skyrim in VR and liking it, surprisingly. I wouldn’t say I’m totally comfortable yet with the controls and the massive inventory system, but I like tromping around in the world and looking at mountains and stone circles and ruined castles. I liked that I got told I was carrying too much to run, and had to dump 3 axes, 2 swords, 2 daggers, 4 pairs of boots, 2 animal pelts, etc. Lots of people you meet have news or jobs to offer you, which feels very welcoming. (No, I have not installed any of the mods to make it look superb in VR, despite being prone to do that sort of thing. It’s still pretty.)
Tetris Effect: I’m not a big Tetris fan, and never understood why this VR port has so many stellar reviews — it’s just Tetris, but with surrounding ambient animations and sound. OTOH, I had a personal trauma this month and played some, and it calmed me right down. Article of relevance on Tetris and PTSD symptoms, I wish I had played it sooner.
Poem
Where dips the rocky highland
Of Sleuth Wood in the lake,
There lies a leafy island
Where flapping herons wake
The drowsy water rats;
There we've hid our faery vats,
Full of berrys
And of reddest stolen cherries.
Come away, O human child!
To the waters and the wild
With a faery, hand in hand,
For the world's more full of weeping than you can understand.
from The Stolen Child, WB Yeats
I thought January was long and hard, but February has really beaten it into the ground. This month my cat died and a war started. The world is full of weeping. Stay safe, everyone.
Best, Lynn