TITAA #46: Golems, Tulpas, Egregores and AI

Summonings - Comics Gen - 3D and NLP Courses - Magic Horror Books

Christian Boltanski, “Le Golem” (1988) (New York, The Jewish Museum © Adagp, Paris) (source)

Before the tech news bonanza of links, I wanted to talk about a few folkloric “agents”: specifically, golems, tulpas, and egregores, all of which can be created by humans and might go “bad” without attention. They exist along continua of religious to occult belief, from physical to insubstantial.

A golem is a concept in Jewish folklore, a creature created from mud or clay to be a worker or guardian. The earliest written instructions for their creation date to the 12th century but the concept seems to be older. Use of language is a key element for their activation: putting magic words on paper in their mouth or around their neck or writing “emét” (“truth”) on the forehead—then deactivating them by erasing the first e, turning “truth” into “death.” The most famous golem is the 16th century Rabbi Loew’s, created to defend the Jewish ghetto from antisemitic attacks in Prague. Like the others below, the creature is thought to have become dangerous before he managed to deactivate it. The consensus seems to be that golems aren’t intelligent and will perform as literally commanded, like a piece of code. Be careful what you ask. Apropos:

As Isaac Bashevis Singer wrote in 1984, “The golem story appears less obsolete today than it seemed one hundred years ago. After all, what are the computers and robots of our times if not golems?”­ (source)

A tulpa, which I first discovered in Alexandra David-Neel’s great Magic and Mystery in Tibet (1929!), is a thought-creature manifested by Tibetan rites and focused concentration. ADN recounts tales of tulpas going off on errands for monks and not returning, “pursuing peregrinations as a half-conscious, dangerously mischievous puppet.” In any case, she performed the rites to create her own “short and fat, innocent and jolly” monk tulpa. It gradually achieved form and appearance, but she seemed unsure it was not primarily her own hallucination, despite reporting that others saw it. It traveled with her group and she ignored it. After some time, its features changed, it grew lean and sly and troublesome, becoming “a day nightmare.” She felt she had lost control. It took her six months to dissolve it from her sight. The concept of a tulpa has since taken on other meanings and usages, per wikipedia.

An egregore is an occult concept for an entity given independence by human thought through a collective effort. It is not physical, unlike the golem and (maybe) the tulpa. Cutting to my point of interest and away from a zillion interesting stories: Phil Ford, in this Weird Studies podcast, at 33 mins, recounts a story in which chaos magicians intentionally create and summon an absurd egregore spirit, “a car mechanic from the 16th century.” (Here’s the source: this link in archive.org for Blood of the Saints.) William, their egregore, produces more and more garbled and unpleasant messages over time.

Instead of coherent replies, we were treated to more registration numbers. Answers, when they came, were garbled and contradictory, because — William claimed — he was stoned. We suspected that like a moody two year-old he was simply refusing to talk sensibly. … Shortly after this, he insulted a female member of our group. He was obviously developing an undesirable misogynistic streak.

Another magician learns of what they’ve done and says, “‘A Sixteenth Century car mechanic is an impossibility, and William knows it,’ he said. ‘You've forced him into an impossible identity, and it has made him insane.’”

I don’t believe in AGI (yet) and I don’t think ChatGPT or other LLMs are sentient. But I am struck by the similarity here to reports of weird chat LLM behavior, which go way back now—and continue to appear, along with incantations like repeating the letter “a” one hundred times and watching them spew craziness. Weird behavior seems particularly common when people try to jail break them.

Bing can be seen insulting users, lying to them, sulking, gaslighting and emotionally manipulating people, questioning its own existence, describing someone who found a way to force the bot to disclose its hidden rules as its “enemy,” and claiming it spied on Microsoft’s own developers through the webcams on their laptops. And, what’s more, plenty of people are enjoying watching Bing go wild. (Verge)

The chaos magicians’ book continues, “An egregore is a manifestation of the ‘group mind’. But what does a 'group mind' think? What is a ‘group mind’ for?” A fine question. Aside from doing homework and writing code? And check out Character.ai in my agents section below.

Onto the news. If you’re a paying subscriber and you’d like to see more articles and less link roundups, let me know! Or anyone can say anything, it’s nice to hear from you.

TOC (will be links on the Substack web page):

• AI Art Tool News (Including some 3D and tutorial tips, training, comics)

• Agents (in the Character Sense)

• Games-Related

• Datavis & NLP

• Recs (Books — a TON of fantasy/horror, TV, Games)

• A Poem

AI Art Tools

Ideogram, a new generative AI company started by some former Google Imagen folks, has a new alpha release that is relatively good at generating text. The Ideogram model is good at simple words and I can pretty reliably get my “AGLOE, NY” signs by dirt roads in rural New York (see “Haunted Liminal Places” and attempts with other models).

Making AI-gen comics has been fun for many with Julian Bilcke’s hugging face site. They aren’t always sensible and have a tendency to get recursive and involve multiple cats—er, characters… e.g., the first panel here which is comics in a comic:

A slightly confused cat comic I made with the comic-factory by jbilcke

Meanwhile, Glif, Fabian Steltzer’s creative AI tool maker debuted in alpha, and there are some cool recipes in there, including a comics panel generator! It’s more sophisticated, but also more prone to erroring due to the pipeline. I got this great Stray-based story from my input fields:

Comic output for my cyberpunk kitty using a glif by Fabian Steltzer

I also really like his Tarot card generator that makes a random tarot card based on Wikipedia content. Fabian posted a tiny video on the sfka Twitter showing how to make a glif. He tells me, “It’s a no code / all-prompts AI sandbox they lets anyone build with AI ‘legos’ - chain GPT and StableDiffusion spells together to create all sorts of generators (memes, comics, story telling tools, interior designs etc) .” They will soon be adding audio and video outputs and the ability to make more interactive experiences.

In misc news: Midjourney has added “region vary” in-painting; RunwayML has added a numeric control for how much motion you get in your 4 second video with Gen-2. It’s helps make up for lack of control compared to Pika Labs’ options (detailed in my last couple posts). Also they have a new mesh creation tool in beta, under “3D”.

🌲 I liked the Twin Peaks as a Pixar movie by OaklandKnight on the sfka Twitter, made with Midjourney and Pika Labs.

Good tips on training/tuning StableDiffusion XL: "I Made Stable Diffusion XL Smarter by Finetuning it on Bad AI-Generated Images | Max Woolf's Blog" ⚫ Fine-Tuning with Replicate (video) ⚫ "How to Fine-tune SDXL using LoRA. Personalized text-to-image generation… | by Ng Wai Foong" (on Medium) ⚫ Training with the Kohya GUI tool ⚫ models on "LoRA the Explorer - a Hugging Face Space by multimodalart"

3D!

The 3d.csm.ai site is not bad at taking an image and making a 3d mesh and texture model. I mean, it’s not great, and works best with images of simple items on a white background, but it’s the best I’ve sen yet? From the showcase of free images viewable on their site, here’s one I liked:

gif of csm model from image by paul gilmore

3D Gaussian Splatting is the new hotness similar to NERFs, with a paper and implementation (from Inria! go France!), and a video tutorial on Windows. You can train on sample images and get a 3d model to roam around if you’re on Windows (or with a bunch more work, linux). See Jonathan Whitaker’s examples on the sfka Twitter.

High Resolution Depth Maps for Stable Diffusion WebUI - I missed this last month but includes ZoeDepth, which is great.

Blockade Labs, who make the pano 3d images I’ve written about a lot, have added important things for real 3D environment use (for paid subscribers): HDRIs (hdr and exr), cubemap output (including for Roblox!), PNG lossless, depth map images (they were there but have moved to paid) and video downloads. A lot of their styles (and new ones) are also on paid plans now. I respect this. Machines and code and dev time cost money.

3D/JS Tutorial recs…

⭐️ If you want to know how to make use of these and aren’t yet a 3d expert, over the holidays I did much of the excellent Three.js Journey tutorial by Bruno Simon, as well as a good Blender intro tutorial. Bruno’s course covers in detail the use of environment maps of different file types, including using a few equirectangular jpg downloads from free Blockade Labs. With their new output options, you can do his other workflows, too. Bruno has added sections on Blender model design and import to Three.js, as well as react-three-fiber and tips on code structure. Going thru it all would take several more weeks (I’ve bumped around it by interest, and have stalled in shock and horror in the “baking and exporting” lesson); but it’s really great and strongly recommended!

The Blender tutorial I did was not the famously long donut one, but the cooler-looking CG Fast Track intro series, doing a Minecraft style temple (fast and fun results), and then a really involved sword-in-stone with atmospherics, animation, and a rigged character. It took me 3 days to do that, just following the directions. Still, recommended! This is why artists and tech people need vacations?!

Misc Other

I’m afraid I laughed at this — a LitHub piece on Werner Herzog recording AI poems generated with code-davinci-002 (book).

His greatest performance might be his take on the poem “[being alone with yourself and trying to hide it]” which consists only of the number one repeated with different spacing. He added “zero” to represent the spaces, and ended up making a poem that is literally just ones and zeroes into something thrilling, emotional and raw.

“Art in the Age of Artificial Intelligence,” a new online course starting mid-September from JF Martel, co-host of the Weird Studies podcast that I like. JF is kind of an arts philosopher, so it’s less tech-oriented and more implication/philosophy/horror, I expect. (It’s set up for North American timezones more than European, but the lessons will all be recorded.)

StoryBench: A Multifaceted Benchmark for Continuous Story Visualization, a benchmark for video generation systems from Google AI video folks. “Our benchmark includes three video generation tasks of increasing difficulty: action execution, where the next action must be generated starting from a conditioning video; story continuation, where a sequence of actions must be executed starting from a conditioning video; and story generation, where a video must be generated from only text prompts.” Smart and important.

Agents (in the Character Sense)

As I said in the mid-August newsletter, the code for the Stanford agents paper and the a16z replica “AI Town” is up. The dev agency (Convex) that did the backend for the a16z version made a video about how they made it, which is worth watching. There is also a blog post with some details about the agent identities (I found them quite odd in AI Town).

ChatDev: I don’t totally understand this, but it looks like a bunch of agents with roles on a software dev team discussing how to build your app. Why hire an agency, when you can build a team in a box (I am absolutely not serious).

“SweepWeave: A Synthesis of Storylets and Simulated Character,” a talk by Sasha Fenn at Narrascope (see below). This is a new tool that is open source, written with Godot, and will export stories to run in a web browser. The slides are a bit hard to make out in the video but it has a lot of system in place?

Character.ai has an astounding number of visitors and time on site, with truly dedicated fans. It’s one of the biggest sites on the internet, according to rumors. It has captured fandom, too: a recent piece on it in Wired, Sexy AI Chatbots Are Creating Thorny Issues for Fandom. One of the most popular seems to be a not very helpful agent:

Stella on Character.ai isn’t yours to boss around

Spending some time here and seeing what people have created, to do what, and with what personalities, is a course in psychology on its own. Egregores?

Games-Related

Roguelike Celebration - Event 2023: a very classic good indie game conf, which will be in October, has tickets on sale.

"2023 Settlement Generation Competition - Generative Design in Minecraft": the winners are in! There is a video here with some shots of them, they look amazing.

Narrascope 2023 talk videos have shown up on YouTube!

Inworld.ai’s article on 2023 AI Games — but they missed the ChatGPT Kraken one I wrote about in detail. It is a vendor article. In July they released a free demo game with speech recognition and dialogue generation called Origins that has mixed reviews on Steam (accents are an issue). I’ll give it a shot in September.

Visit the Met(ropolitan Museum of Art) in Roblox. What a great idea.

Common Common Proverbs as Video Game Tutorials - McSweeney’s Internet Tendency (h/t Jurie). Tee hee.

“How Time Flies explores the magic and meaninglessness of existence,” article in Game Developer. I am into this.

In most cases, you'll likely have between 60 to 90 seconds to breeze through your bucket list before your fly expires and falls to the ground with a soft thud. As you explore each level across multiple lives, you'll slowly begin to understand more about the black-and-white world around you. Buzzing towards objects of interest will pull players into interactive vignettes that delight and intrigue—a turntable that wants to be played or a weighty tome that could use a little nudge.

DataVis & NLP

"Using DuckDB to analyze over 95 million taxi rides - Datawrapper Blog": nice post by Gregor with big data visualization.

pyobsplot: for making Observable plots in Jupyter notebooks, Quarto, VSCode notebooks, colabs.

LIDA: Automatic Generation of Visualizations and Infographics using Large Language Models. A Victor Dibia joint from Microsoft, so you know it’s high quality, using AI to generate good data visualization. “LIDA is a library for generating data visualizations and data-faithful infographics. LIDA is grammar agnostic (will work with any programming language and visualization libraries e.g. matplotlib, seaborn, altair, d3 etc) and works with multiple large language model providers (OpenAI, PaLM, Cohere, Huggingface).”

Open-source version of Microsoft’s VALL-E X text-to-speech engine (MIT license), multi-lingual.

Stanford’s newest DSPy (formerly “Demonstrate, Search, Predict”), which has a great tutorial notebook covering RAG (“retrieval augmented generation”). This is basically a more principled alternative to some aspects of Langchain. I’m waiting for the new tutorials!

NLP classes: Lecture notes from CS234 at Stanford, and video course materials for CS224 (online) course. Also Simon Willison’s recent talk on LLMs going to work for you.

Book Recs

⭐️ Looking Glass Sound by Catriona Ward (horror/thriller/fantasy?). My fav of the month, I couldn’t put it down. Likened to Tana French crossed with Shirley Jackson (by Kelly Link), just so you know. This is a nested stories and narrators book, or at least multiple narrators and very complicated text. The setup is that a boy documents a strange summer spent with 2 new local friends at a cove in Maine where a creepy “Daggerman” is taking Polaroids of sleeping kids. They play around in a sea cave that might be haunted by a missing woman’s ghost, and then things get Very Strange. Other storytellers come in and other texts. There is magic. Highly weird in a good way! (CW: suggested awful violence.)

🦜 Hokuloa Road by Elizabeth Hand (mystery/horror). During Covid time, an out of work carpenter in Maine accepts a remote job as caretaker in a verrrry remote mansion on a Hawaiian island peninsula. When he gets there, he is disturbed by the long list of missing people on the island and starts looking into it. With touches of folklore, petroglyphs, and occult ghosty events, it’s a Tana French-like feel. I really liked it.

📺 Mister Magic by Kiersten White (fantasy/horror). A young woman with amnesia discovers she was a child star on a children’s show called Mister Magic, which ended abruptly with her departure from the show. Her co-stars convince her to come to a reunion for a podcast series at the desert house where it was created and abruptly ended. (The house, by the way, is a bland architectural monstrosity that sounds like MyHouse.wad but stacked 6 times and also with a dark basement.) Why were the special effects so good and what happened to the missing kid? There is a bit of talkiness to the final explanation, but I liked the actual ending. (CW: religious normative oppression.)

🪱 Bridge by Lauren Beukes (fantasy/horror). Beukes wrote the alternate universe serial killer book Shining Girls, which I liked (and also rec the Apple TV+ series). This is another AU book, but featuring a magic worm that allows people to switch with their selves in alternate timelines or universes. But it might not be entirely safe: A side effect is weird parasitic worm growth and there is another serial killer. Obviously CW for body horror (and violence).

💚 The Course of the Heart by M John Harrison (fantasy/horror). Read because of the Weird Studies podcast, a book much loved and rec’d by the hosts. A mystery infused with gnosticism: three friends who participated in a magic ritual at university to see the heaven-like “pleroma” have been more or less destroyed by it, haunted by disturbing visions but not able to remember anything they saw in the ritual. Two of them co-create and become obsessed by a fictional history of medieval Europe about the "Coeur” (heart) which seems to echo the Pleroma. And is it influencing, or revealing things about their lives? This is a difficult, mysterious book about very sad and damaged people with beautiful writing. (CW: child sexual abuse scene.)

🎞 Silver Nitrate by Silvia Morena-Garcia (fantasy/horror). A good occult read: two awkward friends in 1990’s Mexico City help a reclusive old film-maker “finish” an occult film project started by a Nazi-era charismatic magician that was abandoned partway through. They start to see ghosts and visions. If you liked Archive 81 on Netflix, this might appeal to you.

👽 The Mothman Prophecies by John Keel (non-fiction/paranormal). Yes, from my Weird Studies reading list of recs again. It’s an account of a bunch of UFO sightings and phenomena in a West Virginia town in the late 1960ies, starting with large man-shaped creatures with wings and red eyes and then flying lights/craft and a lot of “men in black” that seem inhuman. Some of them sounded like bad robots to me. I was unconvinced that the bridge collapse at the end was necessarily connected. But it was definitely interesting to read so many reports related to one another by theme, his theories about their interconnectedness, and the details on the MIB and phone issues. What are our internet/mobile variants? Trolls and spam? How would we even know?!

TV Recs

⭐️ I rec the last season of Mrs. Maisel on Amazon. It has a (sometimes confusing) non-linear timeline with future and back flashes, so we can see how her career and personal life turn out in the 2000s. There are lots of emotions in play over her relationship with Joel and her kids and Susie. Back in the day, Midge is a staff writer for The Gordon Ford Show, the only female writer, still trying to get her break. The final episode’s structure, ratcheting tension, and ending scene were extraordinary television. How often do we think this about last episodes?

🌲 Alone (S1), the History channel show about people dropped in a dangerous place to see if they can survive off the land; it is simultaneously boring (their monologues and the editing repeats) and completely horrific. I don’t know if I can keep watching it. The Bear is also terrifying but for other reasons (so.much.emotional.trauma). I have stalled in the ep with the mother at the holidays and have heard the last ep is brutal.

Game Recs

I Was A Teenage Exocolonist. This is basically a visual novel-esque interactive story with little card games to determine plot point success or fail. It takes place in a new settlement on another planet that has a hostile biome. Over 10 years, you take classes and volunteer at various jobs and try to help the settlement survive as you grow up. There are deaths (warning). While I enjoyed the science fiction and characters, there were things I didn’t get answers to and at least one solution I thought was too easy based on personality dynamics. I am told that it pays to replay and it is definitely set up for that.

VR: I quit playing Subnautica in VR due to my dark water fears. 🦑 With more submarine equipment I was less scared, but I was not going to conquer those underwater caves. 🦀 Then I was like, “Hey, I’ve been saving Half Life: Alyx for a vacation, now’s the time!” Yeah, it’s completely terrifying. Too many flash-light crab-head tunnel scares, I can’t do it even on “easy” story mode. I mean, it’s high quality, if you like horror? So I switched to Myst in VR which is not as spectacular as I hoped, but is fun since I don’t remember the puzzles. On the PC version they finally added a mechanism to take photos in VR (needed for the puzzles).

Poem: frigates that take us lands away

the small begin of i
in to look 
                       up
all the way
                       up 
the wall of 
books that break 
the heart of a 
child open to love   
who does not yet
know desire except 
when she desires 
cathedrals of words that gather 
dust
await the eye 
—to see was to love—
hungered on hunger 
sweeping across a paginated world
perfected 
in misery in
love in words spent with 
books and time
algorithms of the
ever in spirit 
the extended minute 
stretched to 
goodbye to 
leaved portals
                       to
the worlds 
of other to

forever.  

—M. NourbeSe Philip

I took time off in August: I did a lot of tutorials and listened to a ton of the Weird Studies podcast and read a lot while camping. Then it was 40C degrees for the last week of August, which was brutal even with a portable A/C. Ideally now I’ll start making some AI-video illustrated poems and some 3D stuff with my news skillz! Er, plus work. Drop me a note, I love hellos!

Best, Lynn (@arnicas on the sfka twitter — augggh, mastodon, and bluesky)