TITAA #55: Superpositions on Story

Quantum Pages - Gender Voice - Latens - Genre Templates - Dredge & Lorelei - Tons of Books & TV

Graham Francoise, “I Have So Much to Say”, at Gallery ERGO

If you’ve been a reader for a while, you know I’m nuts about alternate universes in art of all types, and this month had a bunch of relevant media content for me. I’m watching Dark Matter on Apple TV+, the Blake Crouch book as TV. It’s not my favorite fiction take on the multiverse, but it’s pretty explicit about what’s going on. For more poetic and weirder takes, I strongly suggest you watch Constellation which I rec’d a month or so ago. (But it was cancelled; not enough of a “sad Dad” show for Apple?)

In Dark Matter, the thesis is that you can enter this guy’s black box and come out in a different alternate universe, branched based on some decision made somewhere in time. You need a good drug to get into the right state of consciousness for this. The physicist main character refers to being in “superposition” until you open the door and the wave function collapses on a solid world version.

As a matter of fact, I have spent some enjoyable evenings querying AI search engine Perplexity.ai about topics related to quantum mechanics and superposition. With the new release of Perplexity’s “Pages” feature for Pro users (coming to others soon), we can now create shareable wikipedia-like pages with citations and topic sections customized by our own interests. I made one for Superposition and Quantum Mechanics, for your reading pleasure, covering: the basic science, philosophical theories, what it means for consciousness, artists and poets who’ve made work exploring quantum concepts, science fiction references (some good ones, including Greg Egan of course, and this fabulous recent article on the modernists), plus a little on quantum computing.

Sample:

In the von Neumann-Wigner interpretation, consciousness is seen as necessary for completing the quantum measurement process. The wavefunction is postulated to collapse only when the result is observed by a conscious mind. However, this view has been criticized for not clearly defining what constitutes a measurement or observer.

Physicist Roger Penrose has suggested a different perspective, where wavefunction collapse is an objective physical process that gives rise to consciousness, rather than consciousness causing collapse. In this "orchestrated objective reduction" theory, quantum superpositions in microtubules within neurons are seen as collapsing due to gravitational effects, producing moments of proto-conscious experience.

The process of creating the page had a few glitches I expect they will iron out (you get no chance to edit text including section titles, just reroll); but I really like the idea of a custom “take” with many citation links to follow up on. On the other hand, I occasionally find the citations are not linked accurately to the correct summaries (a thing I often see in the AI News summaries too, especially with links).

Related research competitor: A few months ago Stanford students published a paper about their wikipedia-page generator called “STORM,” which has an online demo up now. I compared the two. Perplexity’s Page allows section by section development and direction, for a more “high level” readability level, while the STORM one does not and ends up being a custom deep-dive. They have overlapping citations, though. Oddly, the STORM page mentioned Schrödinger’s Cat but the Perplexity one did not (despite citing Shrödinger). My STORM page as a pdf is posted here; it has a lot of high quality citations and is focused more on debates about quantum consciousness and reality and less about art.

Down below in the AI tech news, you should not miss steering and building a world in Cris Peñas’s Latens, or the framing around video generator Pandora, which suggests its world model takes you to alternate realities. Meanwhile, here is a poem from Tomas Tranströmer about opening (box) doors, one of his many Elegies:

I open the first door. 
It’s a large sunlit room.
A heavy car passes in the street
and makes the porcelain tremble. 

I open door number two. 
Friends! You drank the darkness
and became visible.

Door number three. A narrow hotel room.
Outlook on a back street.
A lamp sparking on the asphalt.
Beautiful slag of experiences.

There is more Tranströmer below and some links to articles about him in the paid supporters’ media recs post, which has more details on all the books, games, and tv. Door number 2, above, is obviously for Our Share of Night.

Table of Contents (links on the webpage):

• AI Creativity News (Simulation & Story Gen, Music & Video, Procgen & Web)

• Games News Links

• NLP & Data Science & Data Vis

• Recs (Books, TV, Games & VR; a lot of recs this month)

• A Poem

AI Creativity News

Simulation & Story Gen

The boundaries between my sections are all blurring as tool stacks are getting put together more and more. Some solid “simulation story telling” things are being grouped here first, despite overlap with games, as one of my favorite topics.

Fable, (in)famous for the AI South Park and their “show runner,” has made more press with their plan to be the Netflix of AI. Their URL now redirects to “showrunner.xyz.” It’s still almost all hype and PR, I think (I can’t stand the X RT’s), but they have a way with getting press for it, e.g. “This AI App Lets You Create Your Own TV Episodes That Look Just Like ‘South Park’” from IndieWire and VentureBeat’s games take from Dean Takahashi, reporting the “shows” available with vimeo footage as:

• Exit Valley — a vicious satire of Silicon Valley – the first two episodes of which are being released today.

• United Flavors of America – a cartoon political satire of US politics in 2024 [left this in to say “uh… gross.”]

• Ikiru Shinu — a dark horror anime focused on the survivors of a global calamity trying to rebuild society.

• Pixels — gentle family comedy of AI-enabled devices living in ‘Sim Francisco’.

• Thistle Gulch [see below] — a Western town with dark secrets.

• What We Leave Behind — a family drama about two orphans in Sim Francisco

All the shows are animated for now and are generated from a prompt of around 10 to 15 words. With that, users can make full AI TV episodes. Showrunner gives users scenes and episodes of two minutes to 16 minutes — all with AI dialogue, voice, editing, different shot types, consistent characters, story development.

“One prompt with 15 words can be enough for AI to generate a whole episode of 10 to 20 minutes,” Saatchi said. “But we’re finding that people prefer to like do a prompt per scene.”

The people who prefer to do a little more customization are the lucky applicants who get in via their alpha application process (the waitlist is 50K long). I imagine they generate a ton of press and use that to get relevant creatives involved in helping them actually make it useful—a bit like the OpenAI artist invites for DALLE-3 and Sora.

Thistle Gulch, of course, is available to play/download/try on Itch.

Thistle Gulch is Fable Studio's  first Multi-Agent Gym Environment or “MAGE”. This platform isn't a game; it's a groundbreaking tool that merges the art of storytelling with the latest advancements in artificial intelligence. It’s the same Simulation that powered our SAGA release, an open-source tool for agents to make decisions in such worlds. You can explore Thistle Gulch by interacting with the Runtime Simulation, but you can also customize Thistle Gulch Scenarios by editing/writing Python code within the Bridge.

InventionArts — “Create and Enjoy Richly Realized Characters in Stories, Guides, Roleplay, Commerce, Fun, and More.” I believe this is the project of a newsletter reader! It caught my eye because of the interesting “collections,” which offer multiple dialogue lenses on, say, a book like The Great Gatsby. Clever.

In other agenty sim news, there have been a few more articles about using character-driven agents to do work together, like this paper on doing long-form translation with multiple agents set up like an editorial staff. I wasn’t entirely kidding about sweatshops for AI agents mid-month.

StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative Planning — work from Autodesk that looks interesting, focused on emergent narrative moments.

We propose a novel plot creation workflow that mediates between a writer’s authorial intent and the emergent behaviors from LLM driven character simulation, through a novel authorial structure called “abstract acts”. The writers define high-level plot outlines that are later transformed into concrete character action sequences via an LLM-based narrative planning process, based on the game world state. The process creates “living stories” that dynamically adapt to various game world states, resulting in narratives co-created by the author, character simulation, and player.

⭐️ “Multigenre AI-powered Story Composition” - Ah, this was right up my street. They create a tool to extract dominant patterns found in genre examples as templates for types of genre stories to generate. For fans of tropes, narrative arcs, and genre. Mystery examples yielded:

1. Introduction of Characters and Setting. The protagonist, who is a detective or investigator, is introduced in their normal environment, along with other significant characters and setting details.

2. Presentation of Mystery. A mysterious event, often a crime, is presented that stumps the authorities, leading to the involvement of the main protagonist.

3. Initial Investigation. The protagonist begins their investigation, gathering information and initial clues about the mysterious event.

4. Uncovering Secrets and Clues. The protagonist, through their investigation, uncovers hidden secrets and new information that contribute to the complexity of the mystery.

5. Confrontation of Suspects. The protagonist confronts various suspects or witnesses, gathers their accounts, and tests their honesty.

6. Red Herrings and False Leads. The investigation hits a roadblock, with red herrings and false leads diverting the path of the protagonist.

7. Realization or Insight. The protagonist has a sudden realization or insight, uncovering the truth hidden in the complex circumstances.

8. Final Confrontation. The protagonist confronts the true perpetrator, revealing their identity and the motives behind their actions.

9. Resolution and Aftermath. The mystery is completely unraveled, with the protagonist explaining their deductions and the consequences following for the perpetrator. The protagonist then returns to their normal life.

For more on story, don’t miss the Games news section below, especially Emily Short’s talk at FDG.

AI Music/Videos

• TrippyTunesTV - on twitch a live stream of real-time AI music video generation, synced to the beat. I find it a bit mesmerizing.

• Mario Klingemann has posted some AI lipsynched music videos (with suno for audio) on his IG and twitter, e.g “Sidewalk Kings” (a grievance song; IG link, X).

• Safety Marc keeps posting Reforum ai synced videos using music generated with Suno (X, sorry).

• How to’s: Reforum is the branch of Deforum video generation code that allows timing indicators. Here is a thread of their process (on X), which uses After Effects and framesync. The Deforum Discord has a bunch of useful links including video tutorials on using this parseseq tool to generate music videos. Their Discord is really nicely organized.

Images That Sound — like that controlnet QR thing last year, this is a generated image that also is a waveform that plays audio. Weird. Which is good.

Coincidentally, also see Pet Sympawnies from Noam Oxman in Colossal:

Cartesia’s realistic fast speech generation is, wow, something. It can be used over API or try it out in their playground, including voice cloning (I didn’t try that). I gave it poetry and it did a fantastic job on pacing, using “classy British man.” Given the brouhaha over Sky/Her from OpenAI, I was smirky to see these unfiltered opinions about voices for the default menu:

The rest of the list of voices is fascinating too: there is a Nonfiction Man, but no Nonfiction woman, same with Friendly Reading Man; the Pilot Over Intercom is male; Indian Lady could be a fictional character, but not the Indian Man, the Calm Lady is “calm and nurturing,” etc. I mean, c’mon. But it’s fast and high quality, faster than darling Elevenlabs. (Although Eleven just added sound fx generation which is a thing I always want.)

“Jerky, 7-Fingered Scarlett Johansson Appears In Video To Express Full-Fledged Approval Of OpenAI,” the Onion.

Video Gen & Latents

⭐️ A pre-release alpha of Chris Peñas’s Latens.ai is up— at least temporarily, and while it’s slow and low res and takes a while to load, it’s amazing! This will become a whole class of experiences: 3d navigation in a dreamworld, with generation as you go. Here’s a much-reduced gif of some of my Bosch-inspired ruin wandering. It stops and starts because you have to press space to get it to generate a new view when you want one, and I’ve sped up the gif. (I posted the longer bigger one here.)

FIFO Diffusion — generating infinite videos from text prompt. (Huh, as of 4 days, there is code.)

The rhetoric and framing for Pandora, a “world simulator” for video control, fascinates me. “[Pandora] accepts free-text actions as inputs during video generation to steer the video on the fly. This differs crucially from previous text-to-video models which only allow text prompts at the beginning of the video.” They think of it as “alternate realities,” explicitly: “World models simulate alternative futures of the world. Pandora allows you to control the future. Here we show some counterfactual futures – different videos generated from the same initial state but different actions.” (There’s code, but the actual video output isn’t awesome. Apart from the bad ones, like the guy not dancing.)

Procgen & Web & Other

Ezra Klein interviewed Holly Herndon about AI Art and “collective intelligence.” There’s a transcript there. Klein says, “What I like about Herndon’s art is she uses A.I. to become weirder, stranger, more uncanny, more personal.” Exactly. She says, “I’m interested in exploring some of the weirdnesses in how we as a society define different things. That’s the kind of stuff that I’m interested in, not having a kind of like A.I. chat pet.” There’s a lot in there on her audio work and on collaborating with and permuting (poisoning) AI models; and her consideration that the model is the art, more than any single generation from it. I agree with her that meaning-making will become more important as infinite streams of generic content become more feasible.

Amy Goodchild is documenting her writing related generative art, both her handwriting generation and her sentence generation. She’s kind of recreating some sentence/text generation libs, imo, but it’s fun output. Meaningful Nonsense and Coding my Handwriting.

P5.js + ComfyUI tool on an M1 Mac, info from Golan Levin. Suggestion is to use generative art output as controlnet input to image generation models, for instance. ComfyUI is the node-based visual editor for controlling generative models, if you are new here. (I still haven’t succeeded installing it locally via Pinokio btw but running on a remote VM is fine.)

2D Rigid Bodies Collisions explanation and interactives page, so fun. H/t Chris Ried’s Gen Arts newsletter which you should sub to.

Voxelize: Build your own voxel games with this, full stack, MIT license.

City in a Bottle - explanation of the tiny shader code to make a moving city. 256 bytes of raycasting, explained.

Shader Park (h/t Nicolas Barradeau). “Shader Park is an open source web-based platform and community for creating real-time 3D graphics and animations using code. It integrates diverse computer graphics techniques into a unified programming interface designed for fast experimentation and live-coding.” Animated, editable, 3d.

The iCity city generating add-on to Blender which just uses procgen geometry (h/t Bilawal Sidhu).

A fractal game of life — zoom out. (h/t Hardmaru.)

Infinite Chef in the browser (you will be thinking of Delicious in Dungeon):

Games News

Aaron Reed’s great post about text-based games with wordplay: Genre Explorations: Wordplay in Text Adventures, which is an excerpt from the extension volume for 50 Years of Text Games, both available here. There were a bunch of games I didn’t know about in this post.

Outer Wilds: A Thrilling Graphical Text Adventure, from Mobius Digital (h/t Tom Granger). This is a text adventure prototype from 2014 of Outer Wilds, which: “We actually used this prototype to onboard new employees in the early days of making Outer Wilds at Mobius. In retrospect it was a pretty good way to get everyone on the same page about what we were trying to make.” The text game can be downloaded and played on Windows. People who sucked at navigating in OW could probably manage this, and maybe I would land the end game this way. (I got to the end game, read about it, and didn’t try. But the game is still outstanding and I dream about it.)

Emily Short’s keynote slides for FDG: How Many Storylets Are Enough? On scoping narrative design and how narrative works with your game mechanics, ideally. Like with clocks—how are you counting down, and what are the implications on the story? I particularly liked her points about narrative dead spots and avoiding those. A few slides:

"Building an AI game studio: what we’ve learned so far” from Braindump Incorporated, who claim “With Braindump, you build top-down/2.5D games or interactive worlds by simply typing prompts. For example, typing “Create a Starfighter that can shoot lasers and drop BB-8 bombs” will generate 3D models, game data, and scripts that make your prompt come to life.” They are obviously not alone in this effort, if you’ve been reading my newsletter. This is a small studio in Sweden, and their observations and methods of prompting for code are really sharp, actually. It’s a good read. There’s an alpha list to sign up for. 👉 In much less well-developed AI game tooling, there’s dreamlab.gg with no real content yet.

Best Cozy Games Of All Time in The Gamer, a fine list. On that note, the demo of Tiny Glade just came out. Fans of Townscaper (which was not in that list), take note! It has a ton of cute little UI features already, like you can pet the sheep, vines grow up the walls, the lights light up at night — everything is bubbly and reactive and cute, like Townscaper. I did this rundown cottagey ruin in 2 minutes (it has a photo tool):

Book Excerpt: Ultima and Worldbuilding in the Computer Role-Playing Game in Game Developer. This extract is about fan community extending and participating in the game narrative in various ways.

“Minecraft is 15 Years old and still changing lives.” Piece in the Guardian, about kids and autism and play.

Ryan Rigney talked to the Arctic Eggs game guy, who I’ve been following for a while now… their solo whacked game about flipping eggs in Antarctica seems like a must-buy that I kept forgetting about. The author says he learned from this $19.99 course on “How to Make a Retro Style 3D FPS in the Godot Game Engine” on Udemy by Narayana Walters, so I’ll be adding that to my list of unstarted courses.

Ryan’s article had this pic from the game which is here on Steam

"Here are your 2024 Games for Change Awards finalists"! Games For Change are games with a (message-y) point. But they aren’t all didactic.

Taylor Swift Loves Puzzles More Than You | ARGNet: Alternate Reality Gaming Network (h/t Jon Ingold) Mind blowing.

NLP, Data Science, Data Vis

What do I even say here… so much has been going on. Cherry picking a few—with reminder that I’m loving cursor.sh for Ai help programming, which I’ve rec’d before, and also Kagi has helped on a bunch of programming searches recently. Pay for things that help you.

Shade Map — show shadows by building/land profile for a time and place on a map. Relevant for people wanting shade in the summer and trying to get sun in the winter (me). Cool tech.

How to Think Like a Computer Scientist, Interactive edition, for teaching programming (problems and quizzes).

A free course on Agentic Design Patterns with Autogen.

Training and Finetuning Embedding Models with Sentence Transformers v3 from HuggingFace.

What We Learned from a Year of Building With LLMs, in O’Reilly Radar.

Anthropic’s look at interpreting the activations in the LLM Claude and its Golden Gate Claude temporary launch (a model obsessed with the Golden Gate Bridge) were one of my favorite things this past few weeks.

Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges — possibly useful reference by Becker et al.

Hyperdiv - Building reactive web UI’s in Python.

Book Recs

Reminder these are abbreviated here for length, and the paid supporter separate mailing has more detail, pics, and sample quotes.

🚀 The Mars House, by Natasha Pulley (sf romance). This is an arranged marriage queer/non-binary romance set on Mars. I still loved it (“still” because I’m a bit bored by most romances now). Pulley is funny and clever. Despite the obvious romance, the story was well told and even the footnotes entertained me. Class struggle, politics, and humans (strong, dense bones) vs. natural Mars citizens (light, can be killed by a shove).

🐜 The Bone Fire, by György Dragomán (fantasy). A translation from Hungarian, set in an Eastern European land in the throes of Communist revolutions. Young Emma goes to live with her grandmother and learns her ways, magical and otherwise. The community is reeling from the impact of various informers, and fingers are pointed at them. Grandmother tells her about her own history slowly and painfully. It’s not always clear if the magic is real or a child’s intense imagination.

🚪 Our Share of Night, by Mariana Enríquez (horror/fantasy). I wrote about this mid-month, in Lizards of Darkness. It’s a long supernatural horror read that slowly gathers steam… but warning for being very violent & gory. The medium who channels a demonic entity for a secret pagan cult in Argentina tries to hide his young son’s similar talent from them after they’ve killed his wife. It has some wonderful writing, too (kudos to translator).

🏞 The Other Valley, by Scott Alexander Howard (fantasy). A town sits in a valley between border walls to the east and west, which once crossed take you 20 years to their future and 20 years to their past. Teen Odile is in vetting for becoming a counselor who helps decide the merits of visitor requests to the east and west borders—usually to see their dead relations. A tense and emotional read, being made into a TV series.

🔎 The Evidence, by Christopher Priest (fantasy/mystery). A mystery set in the Dream Archipelago, in which a cranky mystery writer is handed an unsolved locked room mystery and desultorily looks into it… and of course things get weird, since it’s the DA (see my post here). On these islands is a reality permutation field, “mutability,” that tweaks probabilities of extreme events.

TV

🍀 Bodkin, Netflix (mystery). Podcast team composed of a driven, cranky Irish journalist and a golly-gee American dude, plus their researcher, descend on an Irish village to do a show on some people who vanished years ago during a Halloween party. Not everyone wants the team looking into it.

👍 Fallout, Amazon Prime (sf). Based on the Bethesda games, great press from many I know. Underground Vault dwellers have been living isolated since the 1950ies, thinking the world above is irradiated wasteland. It kind of is? A brutal attack sends one woman topside to find her father. CW: It’s incredibly gory and bloody, like in a way that wasn’t really necessary? But the dog lives.

🌉 Under the Bridge, Hulu (murder/trial). Traumatic, terrifying, based on a book about a true story: A Canadian-Indian teen girl is murdered by peer on Victoria Island. Lots of class and race going on, with the main suspects being bad girls in girls group home who style themselves after a gang.

🕵‍♂️ Sugar, Apple TV+ (detective). Woah, this was different than expected… Come for the hard-boiled detective noir with old film collage editing, stay for the very unexpected twist. I felt like a superposition had collapsed awkwardly. There’s a very good dog. CW: abused women, substance abuse.

🦌 Passenger, iTV (thriller/horror). Something weird is in the woods of Chadder Vale (Lancaster). Police woman Riya keeps saying something is wrong with this town and no one believes it, but people go missing, bins are being stolen in surprising numbers, a murderer was just released early, a stag was found mutilated, what is with this black liquid oozing, there’s a dead tree people come to pray at… Needs s2!

📦 Dark Matter, Apple TV+ (sf). Blake Crouch’s book as a show, about a dimension hopping physicist and his family. I am mildly enjoying it, but not loving it. I think for multi-dimensional plots you should watch Constellation, among other older things.

All the Walking Dead sequels: Daryl Dixon (C-grade), Dead City (B-grade), The Ones Who Live (A-grade). More in the separate mailing.

Games & VR Recs

⭐️🦑 Dredge. The very well loved tiny archipelago fishing game with occult and Lovecraftian story themes. I liked the fishing mechanic, the creep-factor, the UI and art, the island biomes and stories. I was glad I saved it till I had time to really dive in, after reading The Islanders.

🕶 Lorelei and the Laser Eyes. Accorded high praise in reviews (and from friends) for its Lynchian feel, occult storyline with magicians and mazes, and intense puzzle solving: it sure does have a lot of locks to figure out. It’s a game where you will spend some time making notes on paper and need a calculator. I am not that far in, but as soon as the weird storyline kicked in I got more interested in the puzzles.

VR

🎨 Vermillion VR Painting. I picked it up on sale because it’s highly praised… and yeah, it’s amazingly fun. They did a good job on the UI; you can simultaneously watch a Bob Ross how-to video while painting in VR. Oil painting with no mess! And fast cleanup. There are even paint-by-numbers options if you want.

A Poem

O field as grey as the buried bog-man’s cloak. 
And island floating darkly in the fog. 
It’s quiet, as when the radar turns and turns
its arc in hopelessness. 

There’s a crossroads in a moment.
Music of the distances converges.
All grown together in a leafy tree.
Vanished cities glitter in its branches.

From everywhere and nowhere a song
like crickets in the August dark. Embedded
like a wood beetle, he sleeps here in the night,
the peat bog’s murdered traveler. The sap compels

his thoughts up to the stars. And deep
in the mountain: here’s the cave of bats.
Here hang the years, the deeds, densely.
Here they sleep with folded wings.

One day they’ll flutter out. A throng! 
(From a distance, smoke from the cave mouth.) 
But still their summer-winter sleep prevails.
A murmuring of distant waters. In the dark tree

a leaf that turns.

—Tomas Tranströmer, from “Elegy” in 17 Poems.

Take care with your superpositioning,

Lynn (@arnicas on the sfka twitter, mastodon, bluesky and Threads)