TITAA #69: Braitenberg Vehicle Agents
Info Extraction Tools - Poetry Games - Video Gen - Stendhal's Tips - 3d Prison
“This is an exercise in fictional science, or science fiction, if you like that better. Not for amusement: science fiction in the service of science. Or just science, if you agree that fiction is part of it, always was, and always will be as long as our brains are only minuscule fragments of the universe, much too small to hold all the facts of the world but not too idle to speculate about them.”
That’s the first paragraph of a very unusual book, Vehicles: Experiments in Synthetic Psychology by Valentino Braitenberg (in 1986), which should appeal to any fan of agents or self-organizing systems and a-life. The vehicle designs start from simple systems — first, a motor and one sensor, and build on themselves; rather than a dry engineering account of functionality, we get a hypothetical exploration of possibilities and anthropomorphized agency. “Get used to a way of thinking in which the hardware of the realization of an idea is much less important than the idea itself.” What a great sentence, but they are all great!
For instance, the chapters:
It is all quotable, which means you should read it, not just my account.
Imagine, now, what you would think if you saw such a vehicle swimming around in a pond. It is restless, you would say, and does not like warm water. But it is quite stupid, since it is not able to turn back to the nice cold spot it overshot in its restlessness. Anyway, you would say, it is ALIVE, since you have never seen a particle of dead matter move around quite like that.
We move from sensors and motors to multi-sensor (introducing complex behaviors that appear like love or aggression), to complex sensor behavior (curves of activation and suppression—“These vehicles, you will say, are governed by INSTINCTS of various sorts and, alas, we just don't know how Nature manages to embody instincts into a piece of brain.”); from there we get to logic and memory, the start of a brain.
After vehicle 6, we have added in chance and random selection, a genetic engineering concept although he doesn’t say that. Also familiar to procedural generation fans: “Where there has been no conscious engineering at all, as in the case of our type 6 vehicles, analysis will necessarily produce the feeling of a mysterious supernatural hand guiding the creation.”
Vehicle 8 has object detectors. There are 4 dimensional cubes here. Vehicle 9 understands shapes. Then we confront the problem of thinking, or “having ideas.” Vehicle 11 has trains of thought.
As time goes on, we grow affectionate toward the diversified crowd of our vehicles, from the very simple ones to the more complex models displaying interesting social interactions and sometimes quite inscrutable behavior. We can play with them, we may get to know them personally (and they may get to know us), we can tease them, test them, teach them tricks, and let them love or fight each other. We do not feel, however, that they show any personality, not even the most complex ones of type 13. It is difficult to say what we mean by that.
It’s certainly hard not to think about LLM personalities in the context of reading this work (not to mention agentic programming systems)—here’s an emdash to say that very simple prompting can change the evident personality of LLMs, as reported in the latest Anthropic report on personality vectors. Braitenberg says then, “Perhaps we would accept them more readily as partners if they gave more convincing evidence of their own desires and projects.” But precisely what we don’t really want from LLMs that work for us, right?
Moving on: Vehicle 14 gets egotism, a pleasure seeking principle.
While dreaming and sleepwalking, the vehicle transforms the world (and its own position in the world) in such a way that ultimately the state of the world is a more favorable one.
We can only hope our machines do this with us, not against us. And omg, the pictures at the end (artist Maciek Albrecht):
And now the news. Table of contents (will be links on the webpage):
Narrative & Storytelling (incl. Research Links)
AI Creativity
Video
A ton of stuff this past couple weeks 😅.
Midjourney added looping and start/end frames to their video generation!
Also a fun Midjourney.tv to watch some of their sample output by their users. You can get to the author and prompt via this page. It’s a little weird without audio, tbh. Although I am not a “must have audio to be good video generator” team member.
Runway Aleph — much previewed by creative partners, rolling out (they say it’s out for paid subs, I am tired though). Has “the ability to perform a wide range of edits on an input video such as adding, removing, and transforming objects, generating any angle of a scene, and modifying style and lighting, among many other tasks.” Also their Gen 4 Turbo and Image can be run by API from Replicate now too.
An “emergent” ability in Veo 3 was discovered by Goog UX guy Nick Matarease: Veo’s multimodal backbone is “smart” enough to interpret annotated text directions on a start frame and make the additions asked for in the video generation. It seems to work best with photoreal imagery, I had poor results with artistic source. This one worked fine - the text says the flower opens, the fairy flies in and lands on it, the petals fall, in that order. (Reminder your best bet is to use Labs tool Flow for your generations — even with a paid AI Pro account, I keep getting told I can’t generate in AI Studio or use the right models in Vertex AI console, which I am confused by.)
Wan AI | Wan 2.2: Leading AI Video Generation Model — a good open source video model in a few sizes, which is being served on Replicate (e.g., this fast version) and other places (e.g., a HF demo). It can even be Lora-trained, as Ostris is doing in tutorials. According to @fofr on X, it works with artistic/stylized init images too, which is great news (and closes the gap a bit to Midjourney’s model).
Mirage from Decart — a live video stylizer much shared recently. You can try it briefly on your laptop or phone for free. It doesn’t stay super stable and has odd biases, as one might expect. It’s entertainingly weird. I like what it did with the picture over my left shoulder more than what it did to my appearance.
A research project for generating short movies from a structure, Captain Cinema, also of possible interest.
By the way, Wavespeed is supposedly a cheap model provider. And this might be useful: scottvr/wtffmpeg: a toy that has a local llm spit out ffmpeg commands from natural language prompts on the command-line.
Justine Moore on X is doing some threads of viral AI video content on various social media sites, and they’re really interesting.
3D - Not all AI.
DreamScene: [ECCV2024] DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling. Text to 3d scenes. There is code. I am not sure what to make of these scenes, maybe it should go in my mid-month weird issue. But hey, eye candy.
Hunyuan World 1 - a world generation tool with code and a colab now from camenduru; it seems to generate a mesh for a panorama scene. You can play with some prebuilt ones here on HF. You’ll notice you run into the limits anc clipping pretty quickly.
See also YUME world generation that has code (but no demo I can see).
VR-Doh — hands on modeling 3d in VR. Well, these tools kind of exist, but are usually more parametric? This uses splats though. It does look fun and cool.
Get Up Close to Frank Lloyd Wright's Unrealized Buildings with David Romero's Digital Models. Article with pics only, via Clive Thompson. But for 3d models….
Funes open 3d architecture dataset with web viewer — this is fun to explore and a little weird. There are prisons, abandoned towers, an “An Overly Ancient Coal Preparation Plant in Qitaihe,” a Chinese cement factory… get your non-Frank Lloyd Wright but world traveling weird here. And a bunch from eastern Europe — actually, I recommend reading the text, too, which may be oddly translated? but intriguing.
One EASY TOMORROW — this is an ad that plays like a 3d game in the browser. I’m impressed. I honestly wasn’t even sure the products are real, or is this a concept? Because:
Radical Software terrain generator. (Not AI!)
Threepipe — a toolkit for doing three.js coding (wrappers to make some stuff faster/easier).
3D gaussian splat fans should probably subscribe to Radiance Fields’ newsletter.
Images and Other
Ideogram.ai released a character creation and consistency model. Upload an image of a person (usually face) and it will try to place them in a new clip (some pre-made settings prompt templates already exist). For instance, I used the character I created above in Midjourney in one of their prompt templates (the UFO one), and it relit her and made her real:
OpenAI code for generating images with high input fidelity with their API.
Web Misc, Arty, Fun Etc
Wordsearch drum machine, via Clive Thompson. This is a “sonification” of words in drum sound types, as it scrolls through the text of a few books from Project Gutenberg. It’s a sad fact that to get to a very interesting beat rhythm, you have to use common stop words (pronouns, common verbs, etc) rather than content words. It doesn’t really go fast enough otherwise. But still cool!
Veodyssey — I liked this little demo of Google generative tools by Alexander Chen and dcmotz. It’s a bunch of animal-illustrated clips based on the text of the Odyssey, and if you type in something not suggested, it does a similarity search to get you the closest relevant clip. The audio is snippets from the text. It’s cute and with more content/length would be almost educational and would definitely retain kids in play, even big ones.
I also like the Infinite Wiki demo, a clickable generated fake wikipedia with some clever ascii art creations, on Google AI Studio tools.
Draw a Fish Dot Com. I don’t want to spoil the result, which will definitely make you smile. Go draw your fish. And enjoy the others.
The ADHD experience. I felt very seen, once I found it open in my zillion tabs. (Alert, it’s an arty project with discordant and flashy things.) H/t Gorilla Sun.
I Built the World's Largest Translated Cuneiform Corpus using AI. AMA, haha. Actually, this is interesting, in that you can also compare AI translations to human. Don’t trust the machine (on the left):
Generating an infinite sea shanty in the browser: “For a project I’m working on, I wanted to generate an endless looping soundtrack that felt like a sea shanty continuing indefinitely without a fixed melody. This proved to be a fun exercise for both musical and technical reasons.” (Via Laura Olin.)
Narrative & Storytelling
The Stendhal Novel That Doubles as a Playbook for Surviving Autocracy - The New York Times (gift link). Okay, I am going to be working on my French-reading help app over the holidays, this sounds so great. (I feel like I should be linking to more general interest story & writing things, but on the other hand, so much stuff…) Stendhal’s Charterhouse of Parma playbook (wow, this looks pretty familiar):
1. Never criticize the prince in word or deed, or voice any disloyal thought. It will get back to him.
2. Make the prince see you as indispensable, without making him resent your skills and influence.
3. Radiate prestige and prosperity at court to bolster your status and enhance the prince’s self-regard.
4. Adopt the fashions of the court, even if they are ridiculous, to show social and political orthodoxy.
5. Never appear independent-minded, virtuous or enthusiastic; it fosters suspicion.
6. Track court gossip at all times; be alert to calumnies against you and to changing princely priorities.
7. Cultivate allies among the prince’s family and intimate circle.
8. Cultivate powerful allies outside the prince’s circle.
9. Recognize that your allies may be unable to help you, or may betray you. Have backup plans.
10. Study the motives and aims of allies, rivals and foes alike, so you know how best to manipulate them.
11. Grant favors to allies, rivals and foes alike, to make them beholden to you (though it may not work).
12. Never regard your own status as secure. Work at all times to buttress it.
13. If you excite envy, expect retaliation — work to pre-empt it.
14. Anticipate attacks against you from rivals for the prince’s favor. Thwart them, and strike back in kind.
15. Know that even if the prince seems to like you, he may seek to destroy you, for sport or out of malice.
16. Act as if you are under surveillance at all times, because you are.
17. Know whom you can trust, but be careful what you let them know, and anticipate their likely slip-ups.
18. Make yourself popular with the common people as a hedge against smears.
19. Avoid becoming so popular with the common people that the prince feels threatened.
20. Be prepared to defend yourself effectively but respectfully when rivals denounce you to the prince.
21. Do not expect the law to protect you; the judiciary is surrendered to the will of the prince.
22. If your influence wanes, invent a conspiracy against the prince; punish its members to impress him.
23. If you are threatened with incarceration in domestic or foreign prisons, leave the country at once.
24. When you travel, carry a passport that will pass muster with border officials and court spies.
25. In life or death situations, enlist the help of powerful enemies through blackmail or bribery.
26. If, for self-preservation, you must resort to illegal actions, plan well and cover your tracks.
The new “stealth” model “Horizon-Alpha,” widely believed to be an OpenAI model (we notice GPT 5 didn’t happen?), placed first on longform writing in Sam Paech’s eqbench tests, and in the top 3 on creative writing (after O3 and Kimi 2). Today there is a “Horizon-Beta.” Evidently minor prompt engineering gets it to admit it’s from Open AI. Some believe it’s a “nano” version nevertheless.
Showrunner, an AI-powered streaming service, launches in alpha this week. — The newest version of The Simulation (Fable)’s effort at tv made by you… being funded by Amazon now among others. (I was underwhelmed by their first South Park effort, which was admittedly very early.) Coming from a fanfic and fan video background, I actually have a lot of opinions about this space, and about what my people might want to make. Some of it would be NSFW, frankly. Some of it might even be gay. But I bet that wouldn’t be the main focus of this tooling.
There is an AI assisted writing contest in August.
Research stuff
Distant Writing: Literary Production in the Age of Artificial Intelligence - Minds and Machines. As a contrast to close reading, the author (behind a paywall) argues, “The distinction between writing (close) and “wrAIting” (distant) [me: oouuuuch] reveals how LLM-assisted creativity can generate narrative possibilities previously inaccessible, transforming literature’s modal space while challenging traditional notions of authorship, creativity, and literary production.”
Orchid: A Creative Approach for Authoring LLM-Driven Interactive Narratives in Proceedings of the 2025 Conference on Creativity and Cognition. (pdf with cute pictures) “Orchid allows users to structure the hierarchy of narrative stages and define the rules governing LLM narrative generation and transitions between stages.”
AI-generated stories favour stability over change: homogeneity and cultural stereotyping in narratives generated by gpt-4o-mini. “Although the stories do include surface-level national symbols and themes, they overwhelmingly conform to a single narrative plot structure across countries: a protagonist lives in or returns home to a small town and resolves a minor conflict by reconnecting with tradition and organising community events. Real-world conflicts are sanitised, romance is almost absent, and narrative tension is downplayed in favour of nostalgia and reconciliation.”
Info extraction from narrative:
LLM Meets Scene Graph: Can Large Language Models Understand and Generate Scene Graphs? A Benchmark and Empirical Study | Semantic Scholar. They propose a benchmark after finding “Our analysis indicates that these models fail to effectively decompose discrete scenes from a complex narrative, leading to a bottleneck when generating scene graphs.” Scene graphs are useful notation when turning text (especially narratives) into entities and relations that change over time.
Characters: CREFT: Sequential Multi-Agent LLM for Character Relation Extraction: Understanding complex character relations is crucial for narrative analysis and efficient script evaluation, yet existing extraction methods often fail to handle long-form narratives. “First, CREFT builds a base character graph through knowledge distillation, then iteratively refines character composition, relation extraction, role identification, and group assignments.”
Games
Disappointingly, in the surveys I did, the games links were the least interesting to people. So trying to keep it short:
Poetry-inspired game jam submissions on Itch, via Bathysphere newsletter. I have only scratched the surface, but love the energy and inspo here.
In turn, Martin Pichlmair rec’d Oku, a wandering haiku-writing game that looks lovely.
Prologue: Go Wayback! (wtf), a map-based game in dev that looks very interesting if you’re an open world survival game fan; I was bemused by the early review on Aftermath (“I’m Not Sure What PUBG’s Creator Is Trying To Do Here, But I’m Sort Of Into It,” lol), and it seems you can edit your own maps too, for the procgen world gen:
Quaternius • Free Game Assets — 3d assets fyi. Like Kenney’s 2d assets!
Data Science, NLP, AI Tools, Infovis
There were a bunch of great open model releases in the Qwen family, including coding models. They are being served from Cerebras, which I would try next (e.g. via Cline); trying Kimi K2 via Groq led me to sad “overloaded, try again later” responses. But onto some good refs:
Linguistics proper: A research survey on Pragmatics in the LLM era, what a great thing. (With a former boss in the author list.) Examining a writer’s grammar (repo) with a tutorial here, using R and the idiolect package.
Infovis & data: the Pudding’s look at all the OCR’able text in NYC streetview, on maps, searchable: NYC’s Urban Textscape.
Maps: There’s a satellite image embedding model to go with this AlphaEarth project from Google Deep Mind.
Information extraction—wow, this is exciting, and I haven’t had a chance to try them yet:
Pandore: automating text-processing workflows for humanities researchers, a text processing toolkit for digital humantities developed at the Sorbonne. (H/t Mia Ridge.)
Introducing LangExtract: A Gemini powered information extraction library- Google Developers Blog.
Also, GLiNER2: An Efficient Multi-Task Information Extraction System with Schema-Driven Interface, with release!
CiteEval: Principle-Driven Citation Evaluation for Source Attribution (code).
A Poem: Stories
When the light goes out, and the book is set down by the bedside, it all comes flooding in: the story you are reading; the story of the day; the understanding that it is a story, the day now past, those ahead, the clock-hand sweep of time; that you are the hero of your own story; that it will end in death but along the way come triumphs, misadventures, nuptials, tears; that the story contains several plots and connects to countless others; that you will never read all the books collected on your shelves but as long as you breathe the hero lives, pages will be turned; that stories keep us alive; that stories end—the tale of the drunken shoemaker, the tale of humankind—all stories, however beautiful, ingenious or corrupt; that fables are forgotten, myths corrode, gods vanish with the languages that named them; that darkness swallows the world, as in legend, but night in turn is vanquished by dawn; that even the sun, whose radiance authored life’s unpaginated complexity, will someday dwindle to extinction. Or so the story goes.
Safe travels and breaks, everyone! I am broke after moving and shitty ex-landlord behavior with my deposit, so I am just driving aimlessly and camping. I’m looking for a cheap ebike to go with, can’t wait!!