TITAA 44: Reading Ancient Art with Birdmen

Hunting Accident or Magic? - Midjourney Zooms - Infinigen - Unity Ire

Lascaux cave painting, the mysterious bird man in the shaft (image from SmartHistory article)

Yes, more prehistoric art before the post-historic AI! I was traveling in the Dordogne River valley this month and snagged a ticket to a replica model of the Lascaux caves. The famous Lascaux prehistoric paintings are estimated at 20K years old, and the original cave had to be shut due to traffic damage. (Btw, there are now three replicas — Lascaux II, which we saw, Lascaux III which travels, and Lascaux IV, a newer more digital experience; you need tickets for all of them, in advance. Yes, the replicas sell out.) The picture above of the little stick-figurey man is actually hidden in a deep shaft, not in the main galleries with the famous bulls, horses, deer, and other animals.

There are a number of theories about this image, and here is one:

There, at the bottom of a sixteen-foot pit, next to fine drawings of a wounded bison and a rhinoceros, lies one of the rare depictions of a human being in prehistoric art. The man is lying flat on his back, palms up and arms extended. Next to him stands a bird perched on a stick. Nearby lies a broken spear that was probably used to disembowel the bison, whose intestines are hanging out. The person is clearly a man, for his penis is fully erect. And this, according to the sleep researcher Michel Jouvet, illuminates the drawing’s meaning: it depicts a dreamer and his dream. (source, Consciousness and the Brain)

So one theory (also, here) is that this is a shaman in a trance, dreaming and perhaps performing magic with his dream, and the caves above the shaft are full of magic art to summon animals to the hunt. Not, in fact, just art for its own sake. This would make the shaft, wide enough for one person, and in which ritual items were left, the core and key of the entire cave system.

But is this the picture of a dream, or a record of an event? Is this a man gored by an animal, dead or dying, while the bison also dies of another spear wound? (This is the “hunting accident” theory.) There is a rhinoceros to the left of this scene, but it seems to be in another style, and may have been added later. It has some of the geometric dots I wrote about in January. How is that to be read?

There are not many pictures of humans, and never detailed, amid all the wonderfully rendered prehistoric animal art. Note the bird man has 4 fingers, on each hand, not 5, making him even more birdy. The term for these human-animal mixes is “therianthropes,” and there are other prehistoric art examples: the “sorcerer” image in Les Trois Frères cave, in which a man appears as part-stag, and a bison-headed man in Gabillou (pics). Interestingly, in Indonesia there are some even older cave paintings of a hunting scene that also feature stick-figure therianthropes, including a bird headed figure. They’re twice as old!

TOC for the rest (will be links on the web page):

• AI Art Tools News (and then Procgen & Misc - Infinigen! Blender! Three.js!)

• Games News (a lot of post-Unity AI reaction)

• NLP & LLMs & Data Science

• Narrative Gen

• Game Recs, Book Recs, TV Recs

• A Poem.

AI Art Tools News

The biggest news of the past 2 weeks is the release of Midjourney 5.2, plus new zoom out features (“outpainting”) and “—weird”; Stable Diffusion SDXL .9, and arguably Zeroscope Text2Video, which is a big improvement on other recent video generation (although Runway’s Gen2 also looks a lot better now). Reminder, I covered Photoshop beta’s outpainting in my mid-month newsletter here.

Zooming in and out with Midjourney 5.2 and “custom zoom”, done in PS; not weird.

For the gif above, I used Midjourney 5.2 for a first image of “a penguin in a carnival mask” and then from the upscale, I chose “custom zoom” and reworded the prompt to get the scene context in stages (canal, bridge, fireworks). I used Photoshop to make the simple gif with this useful script to resize and stagger layers on the timeline. It took me only a few minutes.

Here’s a good video using zoom-out and MJ 5.2 (and After Effects! so cleaner) via Akhaliq on Twitter.

Trying out the new “—weird” parameter, it has a tendency to make things more cartoony.

“a spaceship hit by an asteroid —v 5.2 —weird <number>” in Midjourney (notice not many spaceships, sigh)

These are my SD XL 0.9 penguins in carnival masks made with Clipdrop, which are really cute but quite different style. I mean, I like them? I could do more prompt engineering, I expect.

Penguins in carnival masks via SDXL .9

Zeroscope: “zeroscope_v2_576w, a watermark-free Modelscope-based video model optimized for producing high-quality 16:9 compositions and a smooth video output. This model was trained using 9,923 clips and 29,769 tagged frames at 24 frames, 576x320 resolution.” There is an upscaler model that works in the Auto1111 web ui on the output of this one. (Source link info for weights on HF, and a colab tutorial here thanks to Dreamingtulpa). A couple good videos from it found by Akhaliq: text to video by dotsimulate with MusicGen music (misty moors), and a vid2vid example of “walking through the ages” by Fictiverse.

Note, the open source video tools still only do gif-y unconnected scenes, so it’s like making animated comics panels. (Hopefully Google will someday make available Phenaki, which I wrote about using here, which does actual multi-scene video.)

DragGAN code and demos are out. This is the one where you draw on the image to suggest how you want it changed, e.g., by dragging the nose around.

★ You should subscribe to Dreamingtulpa’s Friday AI art newsletter, it’s very good. And this week’s covers some 3D things I’m not covering, plus more tips on using Zeroscope.

ProcGen and Other Arty News

★ Infinigen: “Infinite Photorealistic Worlds using Procedural Generation” - holy crap, the code is out! Yesterday. “Infinigen is a procedural generator of 3D scenes, developed by Princeton Vision & Learning Lab. … Infinigen is based on Blender and is free and open-source (BSD 3-Clause License).” It seems to have a few issues but looks like it will be well-documented.

“Generating an infinite world with the Wave Function Collapse algorithm,” an update by the amazing 3D artist Marian42. They tell me, btw, that the settler’s buildings in No Man’s Sky are generated with WFC (they are a developer at Hello Games). I had just been looking at one and thinking, this is kind of a nice place, maybe I should settle down, too bad about the toxic atmo.

Threed.world - a project (with code) to create, edit, and animate in three.js. Seems quite nascent and I don’t see a way to import meshes yet? But hey, we need more tools.

A cool animated NFT procgen art project by Jess Hewitt, Boundary Issues, via Amy Goodchild:

Visit and click on them to activate! by Jess Hewitt

A review of some electronic literature works published by Counterpath Press in the Paris Review, “Beyond ChatGPT.” One by Rena Mosteirin does my favorite trick, “telephone” translation, running Shakespearean sonnets thru multiple languages before back to English. Allison Parrish’s Articulations work is there, as are other highlights.

Simulating sand dunes with cellular automata (code).

Egad - a procedural castle generator plugin for Blender that costs only $10. While looking, I found a lot of tutorial videos on how to make castles in Blender, e.g., this. What if, this August holiday…

Ok, the results don’t look like this but hey, still cool

Games News (AI and Not)

Unity’s announcement of AI tools hasn’t gone super smoothly (but here’s their AI-add-on tools market). They have already removed one vendor for violating terms, and Valve is reportedly screening out games from Steam made with AI if the authors can’t prove they own rights to the assets used to train their models. Given Midjourney’s terms say paid small company users own all created assets, this gives some backdoor leeway, perhaps? If this (irritating) Twitter thread of survey findings from a16z are accurate, lots of studios are already using AI tools, especially Midjourney. And obviously Adobe.

One of the responses to the Unity announcement was a blog post by Mike Cook, game design professor in the UK, arguing essentially that automating parts of design isn’t necessarily going to save anyone any time, it will just redefine scale of production. I’m simplifying a lot, since he also talks about procedural generation and games like Spelunky, which aren’t possible without it. I’m obviously pro-procgen and some AI design (given much of this newsletter); and would add that employers’ uses of automation and abuse of their workforces are issues quite independent of AI tooling and have been going on forever. We need better labor protection, income protection, re-training, but also frankly better jobs. To the degree I can use smarter tools and interfaces to make the grunt parts of my job better, I’m happier; to the degree a broke single dev or very small team can use tools to help them put out a game that’s fun, without needing a huge cash infusion, I’m also happier, as a game consumer.

If you missed my review of an LLM-assisted game last month, I recommend it as showing an early example of an AI augmented game, heavily on the rails, from a system that was still focused on human authoring. Its flaws were less about the AI LLM responses than the speech generation part, which will improve. Some misc links:

• “Creating a Dynamic Relationship System in Scarlet Hollow,” a game in dev, by Tony Howard-Arias. Not a ton of details, but a multi-faceted system.

• “LLM-Driven Game Characters,” slides from a talk by Marek Rosa of GoodAI. There are lots of video examples, and discussion of the needs for long and short term memory, as well as planning problem solving. And a better rate structure than currently offered by OpenAI. None super surprising if you are following the progress on agent modeling with LLMs, but good to see in a game application attempt. And combined with a planner.

• “Thematic Puzzle Design” - good slide deck by Aster Fialla at Narrascope 2023 (videos not ready yet!), via Chris Martens, on making puzzles that work with the narrative and themes. (Avoid “ludonarrative dissonance,” as they say 😀.)

• “Words, Friction, Syntax: Stuff I thought about when making Videotome,” an article from Nov 2022 by Freya Campbell about a tool for very linear interactive fiction. Interesting in thinking hard about uses of Twine, Ink, and other authoring tools for IF. RELATED: “The Best Free Tools for Narrative Games,” covering many of the same tools, in Game Developer.

• There’s an Excel plugin for EVE Online. I need to never ever start playing this game, not because of Excel, but because I might never stop and we’ve established I have game addiction issues.

• “Character Consistency in Stable Diffusion, Part 1”: a blog post on making character sprites (using ControlNet and LoRa). Useful detail.

• “Test Your Prompting Skills to Make Gandalf Reveal the Password for Each Level.” Via Ethan Mollick. I haven’t tried; I’m tired and have Zelda to play but it might be your cup.

Company news of interest (to me): Versed is a European startup trying to make it easy to create RPGs from text using AI tooling. “Versed empowers anyone to create their own role-playing video game. All you need to do is write. Versed generates immersive locations, diverse characters, turn-based combat, and branching narrative.” They have gotten funding from players including Google. They sound a bit like Hidden Door to me, which has also gotten more press recently but is more focused on fiction IP (TIA, I contracted there for a few months, but there’s still no open demo, I think).

NLP & LLMs & Data Science

New OSS LLM: MosaicML’s release of an Apache 2.0 model (30B) that has 8K context length and is supposedly competitive with GPT 3. The Chat versions aren’t for commercial use. (Code repo.) Lots of activity in open source models, I don’t want to fill this newsletter with it…

Training puzzles for LLMs - Sasha Rush is building cool little training tutorial code thingies for teaching LLM/AI stuff.

HuggingFace-Datasets-Text-Quality-Analysis: “Retrieves parquet files from Hugging Face, identifies and quantifies junky data, duplication, contamination, and biased content in a dataset using pandas.”

Weave, a new interactive data analysis toolkit (i.e., dataframes and on-demand charts, dashboardy) from Weights and Biases. (H/t Jeremy Howard.)

Embeddings of DBPedia entities on HuggingFace. This is potentially useful for entity searches and similarity matching.

Agenty things:

• A great overview article by Lilian Weng on LLMs and Autonomous Agents.

• SwiftSage from the Allen Institute is a framework for building agents that think “fast” and “slow” to reduce costs. Essentially, it does less calling to an expensive LLM model. Heuristics and planning ftw.

• “Vland” generative agents code — some folks built that Park agents paper behavior into the vland virtual event tools. (I think we’re about to see a ton of these.)

Narrative Generation

ORIBA: Transforming Artists' Original Characters into Chatbots, paper by Sun et al. (h/t Max Kreminski)

This space is growing… I am not advocating and haven’t tried these myself:

Gpt-Author (code) by Matt Shumer (contributions welcome):

This project utilizes a chain of GPT-4 and Stable Diffusion API calls to generate an original fantasy novel. Users can provide an initial prompt and enter how many chapters they'd like it to be, and the AI then generates an entire novel, outputting an EPUB file compatible with e-book readers. A 15-chapter novel can cost as little as $4 to produce, and is written in just a few minutes.

Two companies/businesses: Artemis.ai, AI-bedtime stories (I consulted on something that looks a lot like this), Storybird.ai, which aims at more than kids’ books, it sounds like.

Game Recs

Mask of the Rose. I am not more than half-way into this (due to travel), but it’s entertainingly weird and well-if-creepily-written. It’s a visual-novel-ish interactive fiction game, with endless opportunity for romance and a mystery to solve. If you want to constantly be asked if you want to flirt with someone or something, this is the game for you! It also has interesting story-template building mechanics for theories and information gathering, kind of like the templates in Storyteller (which I wrote about here).

Mask of the Rose idea builder template

Zelda: Breath of the Wild. I know, I know! I actually got a Switch to play this before playing Tears of the Kingdom, and it is BIG. Also awesome. I really love the world of Hyrule and exploring it. I love the options I have for sneaking or fighting, and I always detour for the magic mushrooms and plants even during fights. The bad guys are cute and dumb. It eases you into things in such smooth ways it feels like a lesson in great game design, honestly.

Some more links with good roundups of narrative and other games, and be aware of Steam’s summer sale: Guardian’s Best Games of 2023 So Far, Rock Paper Shotgun’s Favorites of 2023 PC games so far, and for the Steam summer sale (omg, I own so many unplayed games and yet!), this is a good thread of cheap and old-schooly recs from Colin Spacetwinks (if you are logged in on Twitter).

Book Recs

★ The Lords of Uncreation by Adrian Tchaikovsky (sf). The conclusion to the excellent Final Architecture series. Very good space opera, as the ints under Idris try to figure out how to deal with the Architects in Unspace.

Translation State by Ann Leckie (sf). A disinherited person is sent on a make-work mission to find a missing fugitive. Their path eventually crosses two youngsters’ who are similarly poor fits for their worlds, one a Presgr translator in training, the other an orphan who fantasizes about disemboweling people. An interesting cross between gothic, horror, politics, and non-binary romance — while I gobbled this down, I was bothered by the repetitive whininess of the characters. Ymmv.

TV Recs

★ Silo on Apple TV+. Excellent, really fantastic sf/fantasy TV. I hear it’s been renewed, which thank goodness! That last ep! Seriously, get yourself Apple TV+ for a month and binge this, Ted Lasso, and For All Mankind, just do yourself the favor.

I am writing this newsletter right now instead of bingeing The Witcher, just so you know. Otherwise I am nearing the end of my rewatch of Stargate SG1. It’s been particularly interesting in light of the UFO reporting now, since the series deals with questions like “is the world ready to learn about the aliens trying to kill us,” and “what happens diplomatically if one country has the gate and the others don’t,” and “how do we handle an industrialist calling a press conferences saying they have evidence the government is hiding this.”

Poem: “Summer Grass”

So much has happened.
Reality has eaten away so much of us.
But summer, at last.

A great airport—the control tower leads down
load after load with chilled
people from space.

Grass and flowers—we are landing.
The grass has a green foreman.
I go and check in.

--Tomas Tranströmer (1966), tr. Robert Bly (h/t Matthew Ogle)

Now I can go watch the Witcher, and play more Zelda! Please let me know you liked this if you did, and share it. It’s become much harder to get an audience for anything with the current state of social network collapse.

Best, Lynn (@arnicas on twitter —argh, mastodon, and now bluesky)