TITAA #58: Valley of Petroglyph Marvels
TITAA #58: Valley of Petroglyph Marvels
Ideogram - Narrative & Creativity - LitRPG - Trisha - Checkbox Pointilism - Prompts
This issue has a ton of fun stuff and updates, starting with some rock art archaeology from my summer vacation.
TOC:
AI Art Tools (Image gen, Video updates)
Narrative & Creativity-Related (research and articles)
Recs (Books mainly, see separate supporters mail for all)
Petroglyphs!
For my summer vacation, I decided to visit the the Valley of Marvels (English wikipedia not as good as the french), or “Vallée des Merveilles” in French (french wikipedia). The valley is part of a giant collection of rock art (petroglyphs) in the Alps near the Italian border, north of Nice, surrounding Mount Bego. They are hard to get to — if there is interest (comment or email), I’ll write a separate post about the logistics and adventures I had getting there. Bref, I decided to approach it from Italy instead. As a result of the travel difficulty, I made it to Fontanalba, rather than the Valley itself, but it’s part of the same site collection.
The whole site has something like 100K engravings, originally dated to the Bronze Age (specifically 1800BC-1200BC), based on the presence of axes and spears in the imagery. But evidently more recent archaeologists think they are even older, Copper Age (3000 BC), with some of the weaponry added later on top of pastoral scenes. The engravings are very very high up, and were made painstakingly in a kind of pointilism engraving method (a dot at a time). This site was a collective art project, one million checkboxes in rock, but lasting millenia.
Some of the more famous pieces of human figures include these, collaged for you by me, the sorceror (a shaman figure? holding knives, so cute though), the “Christ” (certainly not actually Christ), the Chief of the Tribe (figure on left in white), and the figure with zig zag arms:
It’s hard to see the zig zag, but the foremost writer about these, Henry de Lumley (that links to Int Archive copy), suggests the zig zagger has the head of a sun, and arms that depict rain or storms falling on fields. So, a goddess, perhaps. Most of the figures are male, even the “danseuse”
The largest number of the icons are “corniforms,” or horned heads, thought to be bulls. I mean, maybe they are demons? or angels? Or people with their arms up? Sure, bulls. Here’s a Lumley page on their many forms:
I took some pics of them, myself — here from Fontanalba, accessible only via the guide, from a complex panel of unknown meaning, that had links connecting scenes…. (I took 3D pano pics of it but haven’t constructed the video yet.)
The guide’s theory was that the lower left corner had a “tree of life” type symbol in the pen that also held a horned figure.
Lumley’s theory is that this was a bull-worshipping people, and the guide suggested that engraving in the mountains was a religious act. It would require enormous time and dedication to come to this space, under the elements, and engrave anything point by point. Perhaps the process itself was a meditative religious act? (Hints of the Ted Chiang article linked below…)
In any case, if you want more details about the trip itself, from navigation to the mountain drive and hike bits (not to mention the Google directions), drop me a note or a comment and I’ll write up My Crazy Summer Vacation. Onto the news!
AI Art
"LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation" - this was obviously going to be my favorite of the past 2 weeks, if you’ve been following me. The code isn’t up yet but is promised soon (maybe by mid-month newsletter?). It offers artistic style environment output as well.
Somewhat relatedly, some dude just posted a tutorial on using Blender + image gen to make 3d environments, clever but a lot of work. And while on 3d tooling, you can use splats inside After Effects, per Bilawal Sidhu on X. (There’s a lot of research momentum on intelligent 3d splat generation from only a few images as input right now. Feel free to ping me if you want links.)
Also see CarlosBannon’s AI x Architecture experiments on X if you want- realtime generation of buildings from Comfyui using Processing generated shapes (h/t DreamingTulpa).
Image Gen Tools
Ideogram 2.0 — the new release is very good, especially at text in image. AND they offer an API. Here are some low res examples of a hardish prompt I gave it — “A road that turns into pixelated art in the distance. A sign beside it says "The Future".” The top one is what I was after, but the bottom one is interestingly deviant, it suggests we’re already living half pixelated. And the distance is to the side? It’s maybe not wrong. Anyway, the service is popular and they’re lowering free perks to deal with demand.
In contrast, Midjourney, not free —
Midjourney’s web UI is finally out of beta and open to all users. It also has a proto web editor tool that improves on the command line and Discord junk, but isn’t a full-fledged editor (like, it allows cropping with the UI but then errors and says cropping isn’t implemented). They also say they are getting into hardware—via X comments—but this seems like a really risky move given the many terrible AI hardware objects out there. Interestingly, they are now doing a ratings effort on prompt adherence; I found it very difficult. Labeling data yourself is the best way to understand how hard something is.
The free Gemini App from Google will do a bunch more things now, including pics. I got these shown below — it didn’t like the “the” article on “The Future” but honestly it wasn’t terrible, although I don’t know if I think fog is a pixelated future. (I’m just stepping away from the Pixel phone AI articles here; I wrote about this phenomenon in more detail here.)
In other Gemini news, if you have a Gemini Advanced subscription, you can now make Gems, which are Google’s version of Claude Artifacts or “GPTs,” I guess. Evidently they can read your docs and YouTube videos, which is pretty useful, frankly. (X demo link, sorry.)
Meanwhile, "Hobbyists discover how to insert custom fonts into AI-generated images (in Ars Technica)". Well, that’s not a good title, it’s a Flux model trained on the Cyberpunk 2077 logo using a Lora. You can now do this very fast and cheaply on Fal. You just need your set of images. Yep, the data collection part is always the hard part.
Gradio Spaces for Background Removal - a ysharma Collection - useful.
Video Updates
Luma Labs updated their Dream Machine animation model to 1.5. It’s also less easy to use for free now, due to demand/cost. I’ve been using Deforum’s Studio app with their music beat (and now “event”) syncing, and it’s really interesting. The papercut style is fab. You still need to do a lot of editing in a video tool, though. (Not to mention the wait times.) My magnum opus is not done yet.
There are 2 more video models open sourced now, neither of which are up to par with the closed ones, but it’s a start: CogVideoX-5B and Streaming SVD from PicsArt (paper was from March, but they finally released code I guess?). Evidently there is a new Chinese model competitor to Kling, as well.
💓 I’ve become a giant fan of Trisha Code. She’s a pink-haired AI video creator with the most fanTAStic northern English accent. She’s hilarious. I (still) consume mostly via X but you can see them in a more uh wholesome place on YT: “Trisha Code - YouTube".
Misc Web & Procgen
Your Name in Landsat Images — so great. (H/t Mia Ridge on mastodon) Here’s arnicas (more interesting than Lynn was):
unit.software — A bit Yahoo pipesy? Visual programming tool in beta. Also see Holograph.so, a visual coding tool built on top of tl;draw.
Stipple-Effect, pixel art editor and scripting/animator via Chris Ried.
The One Million Checkboxes guy posted some interesting things about the project. The cool posts: How He Scaled It, and a new one on hacks by smart teenagers (or, young folks) which is pretty amazing and is now shared everywhere: "The secret inside One Million Checkboxes | eieio.games". (Yes, he has a newsletter.)
Amy Goodchild is making nonsense charts (using her generative handwriting) and I love them.
Video-to-ASCII-Art — lol, you can make the side-pixelated road yourself using this tool, as a video effect.
Arroost: Unblocking creation with friends - a cool article about a visual tool for sound making, via Lu Wilson of tldraw fame. Lots of fun references in it too.
MipMap is a newsletter with a lot of Blender for games stuff in it.
Everything Procedural 2024: Conference playlist is up! (H/t Gorilla Sun.) Lots are game procgen related.
Open Call: Microgrants 2024 - Rhizome’s open call till Sept 15 for micro-grants for web art projects. Like, for hosting payment. I’m tempted to apply.
Games
Some fun articles:
"Book Excerpt: Animal Crossing tracks the rise of the original cozy life sim"
Strange Scaffold’s Xalavier Nelson in the NYT! (I unsubbed but maybe you didn’t.)
Crafting a 13KB Game — Space Huggers blog post.
History is written by the modders on Historical Games Network — Interesting look at game mods and platforms. I didn’t realize some big AAA games like Counter Strike originated as mods. I knew about smaller ones, but… Especially interesting wrt historical setting games.
Conjuring a gestural spellcasting system for VR on Game Developer.
Games coming that look especially cool:
"Decipher ancient languages in upcoming hand-painted adventure The Land Forgotten, over a decade in the making" — via Thinky Games. “All the languages you encounter in my game are real.”
Hand-drawn pen&ink The Collage Atlas.
Claymation Scottish-hero themed Judero coming soon.
Tiny Glade comes out as a full release on Sept 24! It is lovely.
This AR game Wall Town Wonders looks adorable — build a little town in your room. (h/t Luokai on Threads.) “Help colorful characters build their world through fun quests and mini-games, unlocking unique buildings and surprises along the way.”
Hidden Door has opened up a Crow-based game world (partnership for the new film), but I had some state issues with the storyline that I reported on their Discord. Still, worth a look at what they’ve been doing with their int fic AI platform.
Game gen papers:
Moonshine: Distilling Game Content Generators into Steerable Generative Models. More dungeon map generation, a Togelius joint.
“Learning to Move Like Professional Counter-Strike Players” — an AI agents in games paper.
Also Google’s AI-generated real-time DOOM got a lot of interest. Fits in with the citation from Lai at A16Z below.
Conferences coming up:
Roguelike Celebration - Event 2024 - schedule and tickets. Always a good event and online too.
AI & Games Conference in London this fall. Schedule and speakers underway.
Narrative & Creativity
The big share now is Ted Chiang in New Yorker on “Why AI Isn’t Going to Make Art.” His focus is more on the process of creation as a task of fractal attention to detail, which he doesn’t believe a machine can help with?, but he includes an example of an artist making “art” [scare quoting because an unexamined term here] with DALLE-2 that shows similar curation and attention to detail in the process. I was frustrated by this piece because it doesn’t deal deeply with “what art is” when it is perceived by others, independent of the process or author that created it; and it doesn’t seem to address well the hinted at curation and detail from the DALLE-2 case. I believe good writing tools that work at many levels of editing and creation—in collaboration with human writers—via a lot of editing and curation, are going to enable interesting works. And maybe already are.
The Next Generation Pixar: How AI will Merge Film & Games | Andreessen Horowitz" - Article by Jon Lai from A16Z, a firm whose politics I am not a fan of… on AI & games. Unfortunately, they are paying the most visible attention to and funding the things I’m interested in right now. “In each of these technology waves, the early prototypes that started off as novelties became new formats for deep storytelling, led by a fresh generation of creators.” As a response to Chiang (it wasn’t), I’d reiterate that the talent of the human creator plays a huge role in the successful use of AI tools—you know this if you are watching closely the good animated visual work that requires giant numbers of tools and complex asset pipelines. But we knew this in the early days of Midjourney, too—artists were making better AI images.
⭐️ “Imagining from Images with an AI Storytelling Tool” — Very nice work on using images to drive a story, using narrative tooling. Really good read, actually (well grounded in narrative and visual arts respect).
The method is supported by a fully implemented tool, called ImageTeller, which accepts images from diverse sources as input. Users can guide the narrative's development according to the conventions of fundamental genres - such as Comedy, Romance, Tragedy, Satire or Mystery -, opt to generate data-driven stories, or to leave the prototype free to decide how to handle the narrative structure. User interaction is provided along the generation process, allowing the user to request alternative chapters or illustrations, and even reject and restart the story generation based on the same input.
The tool is online and you can try it… I did get a coherent story despite very disparate image inputs (images that were much better than the ones generated, tbh, and I didn’t save a screencap of the input step). Pity about the ongoing awful schlock text style we get from GPT4.
⭐️ Drama Engine: A Framework for Narrative Agents - and the project page, from Martin Pichlmair and Char Putney of AI writing tool WriteWithLaika fame. This open sourced framework is based on Autogen, the agent toolkit from Microsoft (and Victor Dibia). Their examples are the first that for me blend the tool agents with character agents with personalities, as one would in games (and in their case, helping with writing). It’s all in typescript, and saves state in IndexDB in the browser, which is interesting. (See a useful Autogen tool link in NLP & Datascience below.)
Assessing Language Models' Worldview for Fiction Generation — “This study investigates the ability of LLMs to maintain a state of world essential to generate fiction. Through a series of questions to nine LLMs, we find that only two models exhibit consistent worldview, while the rest are self-conflicting. Subsequent analysis of stories generated by four models revealed a strikingly uniform narrative pattern. This uniformity across models further suggests a lack of ‘state’ necessary for fiction.”
Controllable Text Generation for Large Language Models: A Survey. Useful to save.
What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluation — nice resource! Covers image-based stories as well. And datasets for evaluation. Plus collaborative systems! They point out that evaluation of long form generated content remains a big challenge with current methods.
Fine-Grained Modeling of Narrative Context: A Coherence Perspective via Retrospective Questions — this was June, but I missed it in the giant slew of narrative papers I linked to in June’s issues. Super interesting for knowledge graph folks, and they are looking at RAG applications:
Complementary to the common end-to-end paradigm, we propose a fine-grained modeling of narrative context, by formulating a graph dubbed NARCO, which explicitly depicts task-agnostic coherence dependencies that are ready to be consumed by various downstream tasks. In particular, edges in NARCO encompass free-form retrospective questions between context snippets, inspired by human cognitive perception that constantly reinstates relevant events from prior context. Importantly, our graph formalism is practically instantiated by LLMs without human annotations, through our designed two-stage prompting scheme.
Creativity in generation: "Min P Sampling: Balancing Creativity and Coherence at High Temperature" - and a video overview/explanation of this from Johno Whitaker. Also see “exclude top choices” code here.
NLP & Data Science
Prompt engineering interactive tutorial for Claude, from Anthropic — many people have been enjoying this. Also Anthropic’s Cookbook.
Autogen Studio docs looking good, if you want to try orchestrating agents in a framework.
Scikit-learn’s skrub: "Prepping tables for machine learning.” Various cleaning ops.
Ongoing SAE (sparse autoencoder) fun:
Showing SAE Latents Are Not Atomic Using Meta-SAEs — AI Alignment Forum.
Ian Johnson (enjalot) has been hard at work on training SAEs for Nomic’s good sentence embeddings, and is rolling out content for his latent-scope embedding clustering tool (small X thread). Some useful process notes here. You can try the current SAE features via a nicely interactive umap explorer:
Lots of search stuff still surfacing, just a couple libs—I have a lot saved, especially for RAG, if you want more drop a note.
Baguetter “is a flexible, efficient, and hackable search engine library implemented in Python. It's designed for quickly benchmarking, implementing, and testing new search methods.”
d0rc/deepdive: “Conduct in-depth research with AI-driven insights : DeepDive is a command-line tool that leverages web searches and AI models to generate comprehensive knowledge graphs on any topic or question."
Maria Antoniak is collecting DH course syllabi 👩💻.
Media Recs
Reminder that the paid supporters are getting a separate, more detailed mailing of these and can stop here if they want :)
Books
Ra, by qntm (sf/fantasy). Magic is a industrial research topic, but there is a parallel dream world that is encroaching. What does the shocking disappearance of Laura’s mother during a shuttle launch disaster have to do with it all?
A Talent for War, Jack McDevitt (sf). Long-time readers know I am a big fan of his readable, archaeology-infused sf. Alex Benedict goes on a historical records search—spanning planets—after his uncle disappears while investigating the career of a war hero who perished in a space battle.
Dungeon Crawler Carl, by Matt Dinniman (fantasy, LitRPG). An alien species has flattened the earth and turned it into a giant televised role playing dungeon game. Carl, who has just broken up with his out-of-town cheating gf, gets teamed up with her show cat Princess Donut. Donut sold me first, but I am really enjoying these!
Carl’s Doomsday Scenario, by Matt Dinniman (fantasy, LitRPG). Things get a bit more political as the remaining players level up. Some Hunger Games stuff here. Solid cozy reading.
A Sea of Troubles, by Donna Leon (mystery). A Venice-set Brunetti mystery. I really liked the detail and her style.
Don’t Look Now and Other Stories, by Daphne DuMaurier (lit/fantasy). She’s a really easy and evocative writer, modern simple style, great at characterization and mood without being over-written. Solid weird fare.
Games & TV
I have moved on to Fallout 4, which I am loving. Maybe a little too much… I should play some non-shooters after this. I spent my road trip time rewatching 12 Monkeys from the syfy channel (on my new Steam Deck). A few more bits and bobs in the separate mailing!
A Poem
... And it is curious how often in steep places You meet someone short who frowns, A type you catch beheading daisies with a stick: Small crooks flourish in big towns, But perfect monsters - remember Dracula - Are bred on crags in castles; those unsmiling parties, Clumping off at dawn in the gear of their mystery For points up, are a bit alarming; They have the balance, nerve, And habit of the Spiritual, but what God Does their Order serve? ... To be sitting in privacy, like a cat On the warm roof of a loft, Where the high-spirited son of some gloomy tarn Comes sprinting down through a green croft, Bright with flowers laid out in exquisite splodges Like a Chinese poem, while, near enough, a real darling Is cooking a delicious lunch, would keep me happy for What? Five minutes? For an uncatlike Creature who has gone wrong, Five minutes on even the nicest mountain Is awfully long.
—WH Auden, excerpt from The Bucolics: 3
I did not meet monsters, but it did rain on me. I’m not sure I made it home, tbh. I can’t stand that the summer is over! Drop me a note if you want travel adventures. Now, to play more Fallout.
Great list as usual, only starting to read thru the recommendations. I've read Ted Chiangs article and I come to a different take away, which is that Mr Chiang is a writer who takes himself and his craft too seriously. I like his point about "making choices" and "putting in effort" - no doubt that's true and comes from someone who does it for their craft. But the rest? AI doomerism. It's not really creative because you haven't worked hard. It's not creative because you didn't make choices. You kids with your fancy computers are not as creative as me!
Well. Do I think that typing a few words into an LLM or image processor is the same as writing a novel or painting an image? No, and I don't like to label this as "(my) art" in any case. And yet, I can produce images that are visually interesting and even tell a story with them, or I can get ideas on how my characters could look, or what environments I haven't thought of. Anyway, I respect Chiang for writing iconic science fiction stories. I think he is elitist about AI tools and creativity and I don't like that.
More comments on the other links tbd 👀
Thank you for the shoutout Lynn! I wish Substack had a better system for seeing where your publication gets mentioned since I would've missed this if I hadn't been snooping through some of my analytics.
I've always felt that AI tools would need to end up being the assistant rather than the artist. There's just something hollow about seeing AI "make" art, games, and music, but if a person can take what an AI spits out and make something from it, it can unlock a whole new layer of creativity. Thanks for sharing all of these great links!