TITAA #19: More CLIPping, Entities and Emily Dickinson, Drone Archaeology

Drone photo by David R Abram of neolithic tomb on Anglesey, Instagram.

David R Abram has been crowdfunding travels for his forthcoming book of aerial drone photos of neolithic sites in the UK and (possibly) Brittany. He was just fully funded - unfortunately, he won't be offering book sales outside of the UK, but maybe we can get our friends there to help out.  His posts are beautiful and educational.  I'm really hoping he does Brittany, because seeing Carnac thru his lens would be outstanding.
 

Barclodiad y Gawres, southwest Anglesey. Its interior chamber housed a collection of large slabs adorned with geometric shapes - lozenges, bands and spirals - close in style to the rock art of Newgrange on the Boyne River across the Irish Sea. This should come as no surprise. The coast of Ireland, 65 miles west, would have been far more easily accessible to Neolithic mariners than the chalk uplands of Wessex, whose chambered tombs were a completely different, more elongated shape. This one in Anglesey also dated from a period long after such structures had ceased being built in southern Britain but were still commonplace in Ireland. ...

AI & Creativity Links

Yet more on CLIP text-to-image generation, which seems to be something Really Important in the AI art tools moment!  Janelle Shane reviews how prompt engineering changes the output in her latest newsletter, and has some good links. This is a also a good article about CLIP + GAN art, which includes a bunch of good output: Alien Dreams, by Charlie Snell.  I liked the list of implementations and examples from Softology's blog.  (But good grief, colabs with no author info make me crazy.)  The EleutherAI Discord's Art channel is where a ton of link sharing is happening, FYI.

This paper (Visual Conceptual Blending... by Ge and Parikh) by extends the CLIP-GAN generation methods using complex concept creation from large language models and then generating them in several CLIP-GAN models. The best-rated visual performer was Big Sleep from @advadnoun/Ryan Murdock.  But I was very entertained by their language generation stage, using both knowledge bases and masked models. 

From their paper:

To identify a relevant object, we use a simile-inducing input: “the moon is like a [MASK]” and ask the language model to predict the masked word. The language model produces ghost, i.e. “the moon is like a ghost”. Next, we utilize the prompt “the ghost has the property of [MASK]”, where the language model predicts the word dead. We plug the predictions into a template and produce the description of the blend “a moon that is dead like a ghost”.

Then they ask a CLIP-GAN tool to visualize it. So cool.

Story2Hallucination: "This is a Colab notebook used to render paragraphs and whole stories of text as animations using Big Sleep."

CLIP-Draw is also awesome.  The Colab notebook is here.  (You have to unhide the code to get to the prompts.)  I enjoy the content label attempt almost as much as the artwork it produces.  This is "a watercolor of a ruined castle in the forest" with negative prompts approximating "a really bad messy terrible drawing." (Hah.)

CindyJS and the Droste Effect:  This particular infinite zooming demo of a guy looking at art that contains him is very cool. You need to increase the "zoom" slider to activate the effect.  Thanks to the vis folks on Twitter who shared it (JanWillem, Nico).

Low-Fi Games: This piece on Bitsy low-fi games is lovely: Bitsy is Beautiful! by Nathalie Lawhead.

Text Generation:  I wrote up part 1 of an article series on using tabular data to generate text, based on my NaNoGenMo attempt to tell stories from Dwarf Fortress Legends data files.  Part 2 will cover the tools I discovered that might have helped.

 

Some Data Science/Viz Links

VR Data Vis Toolkit: I'm not convinced by data vis in VR (despite being sold on games in it), but this toolkit is under active development.
  Vis of Big Data in Python: Updated Datashader will allow big-data interaction in matplotlib. W00t!

Good NLP course: The short Huggingface Transformers course is a good intro to using their libraries and also to the concepts in transformer language models.

New NLP book: The code for the new book Applied Natural Language Processing in the Enterprise, which is a very up-to-date book including SpaCy v3.
 
A Few Tools for Handling Named Entities (Which I've Been Working on Recently):

• Probablepeople (parse names and corporations into parts)

• Textpack (good clustering of similar entity strings although a tough install, use Docker)

• RapidFuzz, fuzzy string matching even faster than FuzzyWuzzy

Books

It's been a weird month of reading things I never got around to before, especially books about "magic schools" after some chatter with friends about them.  Not all have been worth recommending, so it's a light month's recs.

Fantasy: I ended up enjoying Trudi Canavan's Black Magician series, despite a rough start in book 1 which meanders too much before we get to the magic school.  The rest of the series involves some good magic archaeology in foreign lands, horribly over-the-top peer hazing of our young main character, an ambiguously mysterious head magician who seems evil but might not be?, a thieves guild, a romance I did not hate despite the many ways it was a bad idea.  There are lots of nice supporting characters, and they're not kids.  Very little is really about the magic school curriculum itself, though.

SF: The Last Watch, by JS Dewes, is some excellent adventure SF.  A bunch of disgraced military prisoners, plus one punished rebel royal heir, are tasked with watching the edge of the universe for alien invasion. Then the edge of the universe does something weird.
 

TV

I ended up liking The Nevers on HBO.  Yes, it has messy parts and messy lineage, but a solidly surprising science fictional last episode and the general theme of empowered women sold me.

Lupin Part 2 was excellent. We even got the catacombs. If you like Ocean's 11 types of heists, you should watch this.  Plus it has Paris.

I'm struggling with content-warning advice over Dickinson, on Apple TV, the modernized tale of Emily Dickinson, which I mostly like? But the men who scream at Emily that she can't be a poet and lock her in her room have been a real issue for me.  I like the magical realist universe she lives in, and the modernity of the music and language.  But it's painful.  I've just finished season 1 and am starting on 2.

Home Before Dark, also on Apple TV, is delightful.  I'm not a fan of family stories in general, but the 9-year old precocious reporter Hilde and her father (a professional journalist on a break) are charming and well-acted.  Hilde is fearless and annoying (adults wonder how to "turn her off") to the people she badgers for information about the cold case in the coastal town they have moved to.  We get flashbacks of the case as it happened: her father's best friend disappeared when they were teens and he has never gotten over it. Apparently Hilde is based on a real person, so we have to shelve our worry that "no 9 year old is this smart and determined." And our chagrin that she's a better writer than we are.

Games

I liked Overboard, Inkle's compact 2D narrative game about getting away with murder on a cruise ship. I sometimes wonder at the wisdom of hugely branching narrative, I mean, how many people really play multiple times and try to tweak the ending?  But a game like this is ABOUT exploring that space.  You have to restart a few times, making slightly different choices, to see how to save Veronica with her freedom and fortune intact after she knocks off her husband.  They built in some tools to make replaying sections "the same way" even easier. Clever.

Kentucky Route Zero:  I'm finally playing this on Saturday afternoons, after hearing it called "magical realist" enough.  Yes, it sure is. This is not a game to race through, it's a meditative weave of the stories of different characters intersecting (who then take control of the point of view), with odd musical numbers, giant flying birds, embedded text adventure games, creepy corporate skeletons, caves and mines, strange museums and art exhibits and plays, sad old men and office workers, ghosts, and a whole lot more.  Also, this screen and a few others made me laugh out loud, although you probably need to have played it for a while to laugh too.

VR: I finished Red Matter, and it was great. Thumbs up.  Good atmosphere, graphics, and fairly integrated puzzles.  I should have rec'd A Fisherman's Tale as well, which is a somewhat dark nested perspective puzzle game of French design.  It's one of the ostensibly-for-kids games I've really enjoyed (like Down the Rabbit Hole), although this one is a bit darker? 
 

Poetry

A whole section this time!  I really liked this long-read NYT scrolly story about Elizabeth Bishop and her famous poem, One Art.  I learned a lot about one of my favorite poems.  (It should be loved by all who move too often.)

I learned about Phantom Kangaroo, a supernatural and eerie poetry magazine created by Claudia Dawson, from her terrific Recomendo newsletter. The blurb says, "Phantom Kangaroo is a portal. The Mysteries of the universe congregate here. Some of these poems are incantations meant to be read aloud. Others are testimonies of supernatural accounts or glimpses of a dark reality here on Earth. Dark realities are also unnatural. There is art here too. If you stare long enough you might see your reflection."
 

A Poem

“Hope” is the thing with feathers -
That perches in the soul -
And sings the tune without the words -
And never stops - at all -

And sweetest - in the Gale - is heard -
And sore must be the storm -
That could abash the little Bird
That kept so many warm -

I’ve heard it in the chillest land -
And on the strangest Sea -
Yet - never - in Extremity,
It asked a crumb - of me.

--Emily Dickinson

I've been busy with a couple extra gigs in June, but could always use more pointers, especially to NLP contracts.  I'm getting my second shot on Monday.  I'm trying to renew a visa and apply for a new one just in case.  Things feel busy?  Yet no holiday plans!

I hope you are well and getting or have gotten vaccinated.  If you like this newsletter, pass it on, drop me a note, say hi on Twitter, or buy me a coffee. :)

-Lynn / @arnicas