TITAA #59: Citizen Satellite Glyphs
TITAA #59: Citizen Satellite Glyphs
Zelda Worlds - Creative/Multimodal Writing - Frankenstein Vis - Skydrop - Tutorials - Weather Landscape
Table of Contents (links on Substack page):
Intro article: AI Used for Finding More Nazca Geoglyphs in Peru
Games & Narrative News (non-AI briefly, then Game Narrative AI, and Creative Writing and Multimodal Creative Writing)
Recs (Books, TV) — short form because of paid subs newsletter option
Satellite Archaeology and Nazca Geoglyphs
Evidently use of AI with drone imagery has uncovered more of the mysterious geoglyphs in Peru! (Wikipedia background page if you want it.) An article covered in various forums including the Guardian, announces “AI-accelerated Nazca survey nearly doubles the number of known figurative geoglyphs and sheds light on their purpose”. Using AI with low-flying drones allowed an astonishing find of over 300 more glyphs. Many of the new ones are also small—up to 7 meters across—unlike the large 90+ meter ones known previously.
The new ones also represent earlier cultural work, back to 200 BC (source Guardian):
“We can say that these geoglyphs were made by humans for humans, they often show scenes from everyday life, he said. “Whereas the geoglyphs of the Nazca period are gigantic figures made on mostly flat surfaces to be seen by their gods.” … “The new figures included large linear geoglyphs, mainly representing wild animals, but they also included gory figures showing humans holding decapitated heads, abstract humanoids and domesticated camelids, such as llamas and alpacas.”
These new ones (“relief type”) seem to be on hills, by trails, possible for locals to view, unlike the giant “line-type” which may have had a different purpose (for the gods and ceremonies?).
There is a bunch of detail on the AI algorithm and usage in the PNAS paper, but briefly, it was used to identify possible images of interest, and classification filtering and human review narrowed down to the real “finds.” The AI help allowed them to identify over 300 new geoglyphs in 6 months. Evidently the news ones occurred in groups, and if the model found one possible glyph, the human reviewers could then find more in that area by human inspection.
Using aerial imagery in archaeology is not new, of course. In 2023, satellite imagery was used to find a lot of previously unknown structures in Central Europe (Heritage Daily). These were hill forts dating to 1600-1400 BC. Other archaeological discoveries made with satellite imagery have used “citizen science” volunteers to review the data, rather than AI. In May, the Guardian reported, “Hobbyist archaeologists identify thousands of ancient sites in England,” in a project using trained human eyes:
Truck drivers and doctors are among more than 1,000 people who participated in Deep Time, a “citizen science project” which has harnessed the power of hobbyists to scour 512 sq km (200 sq miles) of Earth Observation data, including high-resolution satellite and lidar – laser technology – imagery.
Currently on the site for Whiteadder in the Scottish borders, there is a big Lidar map you can browse and label yourself. It’s a fascinating thing:
From the astounding real world to the mundane simulation: With the upcoming release of the new MS Flight Simulator 2024, with refreshed satellite maps and detailed land modeling, I wonder if we can do citizen science in there? If you’ve been with me a while, you know I went looking for some famous UK geoglyphs in a Forza driving game and in the last version of MS Flight Sim.
AI Tool News
I’m feeling a bit “meh” about the latest AI tool breakthroughs, because most of it now is just “a better Foozit” based on other work or an open-source lib to do a closed-source thing (not as well yet). Some is company efforts that are hard to evaluate without time/money. I’ll pick a couple that people might care about, but let’s get past it to the rest of the news ASAP.
DrawingSpinUp: 3D Animation from Single Character Drawings — you too can make this guy:
A 3d competitor to PartCraft which I mentioned before — Dreambeast, for fantastical 3D animal generation. With code. (H/t DreamingTulpa.)
Latent Trees for infinite 3d scene generation. They do it in patches, coarse to fine, and the resulting apartment complex looks like a game nightmare. (This is good.)
Building a Gen-AI Story Illustrator — code and walkthrough on Weight & Biases. No longer free to use. (I mean who hasn’t thought about this? Or tried it? But good to see recent code.)
Misc Links:
Video tool: Kling AI’s new motion brush and improved resolution on video gen. Various other models proposing good video have been released recently (like JV-CV and Emu), but nothing you can try super easily.
For fun video, you should google “reddit gordon ramsey ai videos”. Mostly made with a chinese site called Hailuo AI.
Soundscape stylization — change the audio background to make it seem like you were at the beach
Audio: Release of the Moshi model code for speech gen
The Diffusers-Image-Outpainting demo by fffiloni on HF now lets you do infinite zoom out, by reusing your output as your input.
3d Parts: 3DTopia-XL: supposed to be super 3d generator from an image but the demo always gives me an error on the glb part.
A new Blender add-on to show 3d gaussian splats, allows rendering, lighting change, etc.
Flux-360 panorama generation on Replicate (i.e. can be run with an API).
Various people have sent me Wonderworld for 3d scene generation, which is from June and was in a newsletter then, but the paper was revised recently (still no code).
Art: Frank Lanz (a famous game designer and writer) argues with ChatGPT about making original art in “Art Project.”
Misc Web & Procgen
⭐️ Project Skydrop — some guys hid a gold chalice thing somewhere in this circle, and people are searching for it. If you pay a fee, you get aerial hint photos sent to you daily. And the value of the prize grows, too, by the folks paying in.
StoryTribe, a wireframe storyboard creator tool I haven’t had time to try beyond one panel — it’s not AI, it’s a comics creator, it seems? You get limited options on the free trial.
Linear WebGl — webgl demos/code in a single file, “as god intended” :).
3D Differential Growth in Blender tutorial video(s) on YT.
Web Games latest newsletter roundup is out. I’ve rec’d them before. I think it’s mandatory reading if you’re into three.js, webgl, and javascript for creativity in general. Also see Excaliber.js - a recent 2d typescript web game engine.
Glitch’s drawing tools jam collection.
The JS13K tinygame jam results are out, games that are <13Kb. (H/t Gorilla Sun.) You can play Frank Force’s driving game here.
AIC-Bash, a demo from the Art Institute of Chicago showing one way to use their API: turn their artworks into Ascii art at the command line.
Games & Narrative News
Planetary Alchemy, or Learning to Read the Earth With Zelda. A lovely article about fictional earths.
[Zelda: Tears of the Kingdom] is a playable version of the “planetary turn”: a recent shift in the humanities and social sciences toward addressing human culture and politics on a geological scale. It’s a project driven by the sense that the climate crisis isn’t merely global but planetary, that it requires us to confront our imbrication in the systems that make the Earth run.
Only very loosely related in that it’s Zelda worlds, but I love this detailed project (Koholint), commenting on all the tiles in Zelda: Link’s Awakening. It’s almost satellite archaeology! “We talk about the physical layout of the tile. We talk about the monsters and NPCs on that tile. We talk about any plot events that happen on that tile. If there's a cave or a building entrance on that tile, we talk about the interiors of that tile, too. We talk until there's nothing left to say, and the next day, we talk about a different tile.”
Game Developer posts some good Deep Dives, and also topical posts; in the past couple weeks they have done a series on photography gimmicks in games, or how photos are used as a mechanic or tool in some way. There’s a nice deep dive on Highland Song’s custom tool for 2d images (made by Inkle).
Our secret with A Highland Song is that it’s not fully 2D. It’s built in 3D in Unity, using our own custom tools to stack tens of thousands of flat layers over miles on the Z axis. When you start the game, the mountains in the background are actually all the future levels in the game, extending away from you, leading to your goal: the lighthouse by the sea.
Where Will Games be in 25 Years? In Eurogamer (h/t webcurios). Game makers were asked and said… well, some of them told stories, others just expressed depression over the current world of game writing. Jon Ingold says, “In 25 years time, Disney's lawyers will have found a way to copyright the concept of storytelling itself, so it'll be illegal to make any kind of narrative content at all.”
Ink Jam is on this year, November 1-4. Write a story game with ink. Speaking of Ingold.
Game Narrative AI
Cybever - another tool in progress to make 3d worlds easier for game developers using 2d “sketches” to do generation. (h/t Jurie Horneman). It’s not easy to see how the world config differs from Scenario.gg’s growing tooling for game assets and Blockade Labs is also focused on 3d environment creation, aiming at games too.
AI People alpha from Marek Rosa is out, but with a waiting list. These are AI NPCs. “This is the future of gaming: intelligent characters that remember your actions, form opinions, react dynamically to the environment, and generate unique emergent storylines.”
Narrative co-creation with SPORE: a spatial hypertext based narrative recommender system. “SPORE is a client software running on the Component-Based Open Hypermedia System (CB-OHS) Mother. It is a hybrid platform, bringing together spatial hypertext with a combined narrative recommender system and knowledge graph.” So, a canvas-based thing that suggests related concepts and entities. Evidently the knowledge base’s limits frustrated some users.
Creative Writing
⭐️ EQ-Bench, a benchark leaderboard for progress in Creative Writing. Hmm, some Gemma models are right up there. (H/t Rohan Paul.) In particular, this Gemma finetune (fable) is second only to top GPT4o. You can see samples. Bringing me to…
Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs. A fine-tuned BART Large vs. humans and GPT4o. Judged on “grammaticality, relevance, creativity, and attractiveness. BART Large outperformed human writers in most aspects, except creativity.” GPT4o was coherent but not creative compared to BART. Hmm.
"Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing?" Tl;dr: No.
we have carried out a contest between Patricio Pron (an awarded novelist, considered one of the best of his generation) and GPT-4 (one of the top performing LLMs), in the spirit of AI-human duels such as DeepBlue vs Kasparov and AlphaGo vs Lee Sidol. We asked Pron and GPT-4 to provide thirty titles each, and then to write short stories for both their titles and their opponent's. Then, we prepared an evaluation rubric inspired by Boden's definition of creativity, and we collected 5,400 manual assessments provided by literature critics and scholars. The results of our experimentation indicate that LLMs are still far from challenging a top human creative writer
A dataset of generated stories, MirrorStories:
“We present MirrorStories, a corpus of 1,500 personalized short stories generated by integrating elements such as name, gender, age, ethnicity, reader interest, and story moral. We demonstrate that LLMs can effectively incorporate diverse identity elements into narratives, with human evaluators identifying personalized elements in the stories with high accuracy.”
Multimodal Creative Writing
StoryMaker - consistent characters in visual stories. Code. They say they can do two reference images in a panel as well.
The Art of Storytelling: Multi-Agent Generative AI for Dynamic Multimodal Narratives. Co-creation of multimodal narratives with children. “The system combines GenAI-driven narrative co-creation, text-to-speech conversion, and text-to-video generation to produce an engaging experience for learners.”
A Character-Centric Creative Story Generation via Imagination: using images and their descriptions to help make story generation richer and more creative.
Generating Visual Stories with Grounded and Coreferent Characters. “In this work, we introduce the task of charactercentric visual story generation, which aims to create stories with character mentions grounded to the input image sequence. As explained earlier, the task requires understanding coreference relationships between visual characters and aligning textual character mentions to their visual counterparts.” They create a new dataset for this task, and train a model for it.
Related, a recent survey paper on comics understanding.
NLP & Data Science & Vis
There’s a new release of Stanford NLP’s Stanza library for NLP, with a very spaCy-esque API.
Code for Hands-On Large Language Models from O’Reilly.
Data: Structured Wikipedia content on HuggingFace.
Tutorial on dataset creation: Generating a ColPali document-questions dataset (about UFO sightings) to train a ColPali model, a useful article by Daniel van Strien at HuggingFace. From it I learned about the PDF to Page Images tool on HF, which I have immediate need for.
Simon Willison wrote up some fun stuff about the NotebookLM podcasts. I was planning a deeper dive in the weird issue in 2 weeks (eg, see link here); but I note that Simon hasn’t covering the hallucinations you can get (and that the podcasters pull content or knowledge beyond the provided texts); and there’s been a claim that it’s often a male voice mansplaining to an astonished woman (I haven’t verified this myself).
Data Vis
Frankenstein versions compared in the Frankenstein Variorum. For people interesting how to visually show and navigate text differences. (H/t Scott Weingart.)
Data Vis Teaching and Learning materials site.
⭐️ Weather Landscape - visualizing weather forecasts thru landscape imagery. This is adorable. (Code.)
OpenFreeMap — also see Simon’s post about using the fast, free, OpenFreeMap tiles. They even have 3d buildings 😍.
Recs
This section onwards is covered in more detail in the separate paid subs Recs mailing/post. Become a paid subscriber and fuel my enthusiam!
Books
Alien Clay, by Adrian Tchaikovsky (sf). A standalone, imagine a Stalinist gulag on another planet, with scientists who aren’t “orthodox” enough sent as prison labor. Weird plant/animal life and ruins that defy explanation. I really liked.
Crypt of the Moon Spider, by Nathan Ballingrud (sf horror). This is a novella full of body horror… it’s a 1920ies with a sanitorium on the moon, where depressed patients are treated with brain implants from a dead monster spider worshipped by an underground cult. Not my thing in the end.
⭐️ Glass Houses, by Madeline Ashby (sf thriller). A private plane crashes on a deserted island, full of the team of an acquired (Canadian) startup doing some kind of emotional bitcoin product. They find an empty automated house, and people start dying. Kristen, the head of HR (“Chief Emotional Officer”) is the pov character, and she is very, very angry. There are narrative reasons for her anger at the many compromises made by women in tech, but it won’t be to everyone’s taste (I loved it).
Beyond the Reach of Earth, by Ken MacLeod (sf). Lightspeed Trilogy book 2, I reviewed the first one here. The rock intelligences are making more moves, as the group from the Scottish stealth FTL ship builders discovers they might be traveling in time as well as space — sometimes.
The next 4 of the Dungeon Crawler Carl series.
TV
Lady in the Lake on Apple TV+ (mystery/thriller). Hrm — Set in the 1960s in Baltimore, a white Jewish woman leaves family and moves into an apartment in a poor Black part of town. She gets involved in the mystery of a missing Jewish girl. Meanwhile, a Black woman struggles to make a living for her family under a crime boss. (CW: rape, drugs, dead girls/women.)
Kaos on Netflix (fantasy?). The Gods in Crete, in modern day, with a host of British actors and Jeff Goldblum as a monstrous Zeus. A plot to overthrow him goes mostly unnoticed as he hunts for a missing watch and shoots the golf boys on his estate. I mostly enjoyed it and the rewritten legends. (CW: animal harm, cut out tongues.)
Star Trek: Strange New Worlds s1. Enjoying this series at my brother’s suggestion.
I rewatched Jonathan Strange and Mr. Norrell (BBC version) and wrote more about the imagery in the paid subs Rec’s newsletter.
Games
I tried to play Shadows of Doubt and has some issues. Detailed more in the paid subs recs post.
A Poem: By Etel Adnan
In a parallel sleep I came back to a body and love, there where time never dared show itself And so I can say that madness sings in the morning in a setting of water and dream its red coat worshipping the sun Beings and their shadow have left the garden the chairs look at each other, asking if they should talk among themselves or be quiet There is: a split sky, branches cutting the air an octagonal pond there's no more night and day has not yet been created. the angels don't know that earth exists. the end of love pitches us into purgatories or hell there's no need to worry about paradise. the sky will never stretch far enough to contain the shadow of this unhappiness.
—Excerpts from Etel Adnan’s “No Sky” in her book Time, XVII-XX
Oof, I hope that’s enough linkiness and something here was of interest!
Lynn (@arnicas on xitter, mastodon, bluesky and Threads, but mostly posting on Bluesky, I guess)