Michael:

Do you ever feel like an “analogue-era” Moses, stuck on this side of the River Jordan, able to glimpse but unable to enter the promised digital land? I often do looking at interesting new developments in computing. I particularly felt that way looking at an article headlined “Darwin in a Box” by Steven Johnson from the August issue of Discover Magazine (okay, so I’m a little behind in my reading—blame it on blogging.) Since I'm unequal to the task of executing a notion that came to me while reading the story, I'll share it with you and our readers--maybe somebody can bring it off.

The article discusses genetic algorithms, originally invented by John Holland in the 1960s at the University of Michigan. (Which of course makes me feel even worse, as that was pretty much the equivalent of my own back yard at the time.) According to the article:

[The technique] creates a random population of potential solutions, then tests each one for success, selecting the best of the batch to pass on their “genes” to the next generation, including slight mutations to introduce variations….

To give the clueless (like me) an example of how the process works, the author shows how Torsten Reil, an Oxford researcher turned animation entrepreneur, used these algorithms to solve the problem of making a digitized character walk. He started with a simple stick figure, added muscles and a neural network to control them. The control network was the actual focus of the evolution.

Reil and his team created a genetic algorithm to explore the potential ways that the figure’s control system could be refined. The ingredients of a genetic algortithm are actually relatively simple: a population of “organisms,” each with a distinct set of “genes”; rules for the mutation and recombination of those genes; and a “fitness function” to evaluate which organisms are the most promising in each generation. In this case, the fitness function was “distance traveled from the origin without falling over.” The algorithm generated 100 animated characters, each with a [random variation in its neural network.] Then the algorithm let them all try walking. Predictably enough, the first generation was almost completely inept. But a few figures were slightly better than the rest—they took one hesitant step before crumbling to the ground. By the standards of the fitness function, they became the winners of round one. The software made 20 copies of their neural networks, introduced subtle mutations in each of them, added 80 new participants with [randomly varied control] networks, and started the next generation walking.

Eventually, the computer program created a successful striding figure, which Mr. Reil incorporated into an animation software package called Endorphin.

I don’t know about you, but genetic algorithms—which are getting a fair amount of real-world use in the engineering realm to do things like optimize refrigerator design for cost and efficiency—would seem to offer a lot of possibilities for the cultural realm. To take one example out of many, imagine applying this idea to writing screenplays.

We could create “organisms” (i.e., screenplays) with “genes” (characters, set-ups and rules for the interaction of same), rules for the recombination of those genes, and a “fitness function” to be determined by an analysis of financially successful films (points awarded for happy ending, surprise twists, romance, whatever.) The “genes” could be created either out of whole cloth or—more likely—taken from successful movies of the past (say the character played by Bogart of “Casablanca”). Then the algorithm could cook up increasingly fit “organisms” which could be handed off to teams of writers to flesh out with dialogue. To keep our “organisms” up to date, hip “genes” (like characters or situations from successful sitcoms or newspaper stories) could be introduced every so often, the “fitness function” would be recalibrated based on recent movie releases, and the algorithm re-run.

If this strikes you as all too pop-culture-y of course, one could utilize the same approach by analyzing work shown in particularly hip art galleries and museum retrospectives and use the algorithm to get to today’s artistic “sweet spot,” defined as the commercially successful part of the cutting edge. Then we could just pass off the idea to some artist to actually fabricate (assuming that any of today’s cutting-edge artists actually deign to do such things.)

As Mr. Johnson remarks in this article:

The genetic algorithm doesn't make the computer self-aware in a HAL9000 kind of way, but it does make the computer genuinely creative, capable of imaginative leaps and subtle connections that might elude the minds of human engineers.

Or who knows, maybe even the minds of human artists?

Heck, if we deliberately played up the cybernetic angle of this, I suspect we could win the Turner Award within three years. Who wants to bet?

Cheers,

Friedrich

You've got my brain buzzing, but I've got no energy. Even so ...

Thanks for the posting, which is very tantalizing. I find myself wondering: Isn't the genetic-algorithm model for art ... just great? And also a pretty down-to-earth model of how the arts in fact have always worked?

To illustrate: traditional forms, genres and styles equal genetic code. Artists out there doing their best (for whatever reason, whether lust for fame or idealism -- who cares?) provide the motor energy. Audiences (and a few other factors) do the pruning and selecting. A few wildcards (new technology, perhaps a weirdo or genius or two) toss in some randomizing factors, helping the whole thing adapt and evolve.

It seems like what can happen nowadays that's dfiferent is that it can all be electronically modeled and powered. Let 'er rip, says I.

It does leave me reflecting a bit about modernism-etc. In the first place: what a bad mistake for "art" to identify itself only with the innovative/genius/weirdo side of things. In the second: If we can agree that what the avant-garde is up to is either throwing out the inherited-from-tradition genetic code or replacing it with something rather arbitrary, then the art that they've grown has basically veered away from having any useful relationship with its (if you will) host organism. "Art" under the sway of the avant-garde has become an excrescence, a wart or a tumor, out there multiplying and growing in its own terms, but in terms that the larger organism (society) experiences as a bother or worse. And I say this as someone who's fond of a fair amount of avant-garde art.

To my mind, what's exciting about Christopher Alexander, or even Robert McKee in screenwriting, or the New Urbanists, or the New Formalist poets that Mike Snider has talked about here, or people like Wynton Marsalis in jazz, or any of these New Traditionalist types is that they're tring to synch art back up with genetic codes ("patterns" in Alexander's language, classic forms in all the other arts) that aren't arbitrary, that don't lead to absurd and cancerous growth, that in fact derive from and ultimately serve the larger host organism. They're trying to get art back on track, in other words.

It certainly seems to be true that some randomizing or mutations (ie., innovations) are necessary to keep things flexible and adaptable. On the other hand, it seems insane for an entire field to identify itself (as the avant-garde establishment has done) with innovation, subversion, defiance, etc. It seems inevitable that such a field will make itself irrelevant in a short time, and will be disliked and found a pest by the larger society too -- not a bad description of the fate of the avant-garde, no?

Posted by: Michael Blowhard on September 28, 2003 11:20 PM

I guess one of my points was, though, that at this point the "avant garde" can probably be modeled just as accurately as the super-commercial movie industry. One deliberately panders, the other surrepitously panders, while describing its activities as "cutting edge." In some respects, I would assume that contemporary advanced art is probably more easily modeled...it being less affected by independent variables like economics and demographics.

Posted by: Friedrich von Blowhard on September 28, 2003 11:38 PM

In terms of time complexity genetic algorithms aren't exactly efficient. The random process ("mutation") is more efficient than simply trying every possibility, but if you know enough about the parameters to narrow the problem, you can usually cut the GA out and use something more traditional. See hill climbing as an example.

re "imaginative leaps", my opinion is that they are contained in the "evaluation function", something the programmer writes. We like to think of it as part of an autonomous computer program, but if we're going to be fanciful, I'd prefer to see it as the beginning of an extended mind.

Reflecting back on the evaluation function, there are all sorts of human concerns that would confound the process. "fitness" doesn't have fixed values, it's relative to the moviegoing public (here I'll assume an average, but there are good reasons to assume otherwise). a simple example: the twist ending in the movies the sixth sense, identity, and matchstick men are neat the first few times, but if I see another movie with this sort of ending I'm going to pull my hair out.

Thinking about a GA for screenplays is interesting, but well beyond our means. Reducing our current movies to forms of narrative a la Aristotle's poetics and other postmodern forms is fun, but how would you cash that in to measure fitness? (dollars spent isn't "granular" enough -- how do we know it isn't the physical appearance of the star/hero?) And would the random process simply select from the forms of narrative we already have? Is that novelty? "fitness" is a relation between a thing and an outcome, how would one project past success (here I'm thinking of some mathematical function) to accurately predict future success? There's a big gap between thinking there's an optimal narrative and numerifying it.

Well, great post anyway. To be clear, I don't mean to say that genetic/evolutionary algorithms aren't interesting; for example, see some links related to Jordan Pollack's work, here, particularly, this

Posted by: Shai on September 29, 2003 12:03 AM

regarding art, you'll find quite a lot of material if you search for "genetic art", "evolutionary art", "generative art". The most famous example of A.I. art is AARON

Posted by: Shai on September 29, 2003 12:11 AM

FvB -- I'd bet you're right, that it'd be a lot easier to model avant-garde art processes, which have the advantage (from an easiness-to-model standpoint, anyway) of having been dreamed up and imposed rather than grown and evolved over long eras -- they're "rationalist" in the purest sense.

Shai -- Interesting thinking and info, thanks. I'm not sure FvB or I mean to be taken terribly literally on the topic -- we don't know enough, for one thing. But also (speaking for myself anyway) for many of the reasons you raise -- there are so many elements that go into the growth and development of an art form. As you say, was a given movie successful because of its narrative, or because of its stars? What role does "magnetism" play, and how can that ever be defined or quantified? I'd throw in something that occurs to me as I type: artworks that were never popular but have been influential -- "Kane," for instance, was a flop, relatively speaking, yet it's hard to imagine a more influential movie. Amost none of Altman's movies since "MASH" have been hits, yet he's certainly been one of the most influential of all recent moviemakers. It'd be fun to come up with other elements that computers would probably have a hard time modeling. Or that, at the very least, should be included in any kind of attempt to model art processes. Still, an interesting moment, what with evo-bio and neuroscience's impacts being felt in the arts at the same time that purer art-world phenom like Alexander, McKee and the various New Traditionalisms are happening, no?

Posted by: Michael Blowhard on September 29, 2003 12:31 AM

Why does this sound like the old "monkeys at typewriters" gambit?

Posted by: Tim Hulsey on September 29, 2003 1:41 AM

"I'm not sure FvB or I mean to be taken terribly literally on the topic"

Right. I didn't think otherwise. Imagination is part of what makes computer science fun. On the other hand, the recent history of AI suggests it's important to make a distinction between impractical (science fiction) and practical technologies.

"New Traditionalisms are happening, no?"

I wouldn't be studying compsci if I thought otherwise about science and technology. But it can't just be about the technology. It's part of the reason why books like "Gödel, Escher, Bach" are popular (I haven't read it myself). I can't say what is or is not important to the art world, but if I ever study AI in graduate school, I have a feeling I'll have to audit neuroscience and psychology courses to have any hope of doing anything useful and original. (whereas in the art world you sometimes get a sense that popular science is hooked into art as a variation on a theme, some sort of new kitsch -- but I'm just getting into art, and aesthetics in general, so maybe malicious stereotypes are at work)

Posted by: Shai on September 29, 2003 1:46 AM

err.. I accidentally cut off part of that second quote

Posted by: Shai on September 29, 2003 1:56 AM

Tim -- I think 2Blowhards, and maybe the blogosphere generally, exemplifies monkeys at typewriters pretty well.

Shai -- If only artsies would spend a little time on real science! You're absolutely right that what little they expose themselves to they tend to misuse in the most amazing ways. If you do a search on this blog for "Salingaros," you'll run into a multipart q&a we did with a math-and-computer prof who's been wrestling with art and archictecture in very interesting ways. I'd be interested in hearing your reaction to it.

Posted by: Michael Blowhard on September 29, 2003 2:09 AM

Michael,

Interesting, I'll have to check it out tommorow. My vision is blurring, i.e. I'm about to have a migraine.

The ACM50 symposium I linked to has a lecture titled "the future of storytelling", although it only broaches this subject tangentially. Most of the talks are hilarious as 50 year predictions and they're only 6 years old.

Posted by: Shai on September 29, 2003 3:34 AM

Shai: GA evolved neural nets can be very useful if you have many degrees of freedom in your parameter space!

Posted by: David Mercer on September 29, 2003 7:16 AM

As a first time visitor, this post is scary long and arcane. No doubt interesting (well, some doubt)

I'll come back when you're feeling less loquacious

Posted by: Andrew | BYTE BACK on September 29, 2003 8:41 AM

Hey, we never feel less loquacious.

Although sometimes we try to make an effort to be less loquacious ...

Posted by: Michael Blowhard on September 29, 2003 9:08 AM

When does Endorphin learn to minimize and exaggerate for effect?

Imagine a Bugs Bunny who obeyed all the same cumbersome laws flesh and blood creatures are stuck with. Or go see some darn Pixar movie...

Posted by: j.c. on September 29, 2003 9:37 AM

J. C.

I think Endorphin actually dreams up a new set of rules for each character. Whether that means it can "modify" physics for comic or dramatic effect I have no idea. Of course,even cartoon characters have certain fixed qualities--I seem to remember Chuck Jones talking about animating a cat with the logic that it was a light[ly built] animal that walked heavy.

Posted by: Friedrich von Blowhard on September 29, 2003 10:36 AM

Shai:

I spent some more time thinking about this since I posted it; I'm guessing (what do I know?) that genetic algorithms work better the more that the quality of "fitness" can be objectively defined and simulated. Coming up with a workable definition of "fitness" for movies (or other art forms) would be difficult--and essentially constitutes a completely new intellectual problem--but I'm not sure it is unsolvable, merely difficult. I would suggest as a beginning interviewing some people in studio marketing departments to see what they can tell you about movies that are easy to market. I bet they have some theories--and who knows, even some data on this subject.

But, of course, this process doesn't have to be driven by commercial profits (although it would be easier to get it funded if it were.) As a writer, you could simply come up with your own notion of what you'd like to see in a movie and turn the program loose to match your personal fitness paradigm. And if you didn't trust your ability to design a fitness algorithm at all, you could just read all the output the thing generated and pick up some interesting stories! (This amounts to the "I know it when I see it" fitness algorithm!)

Posted by: Friedrich von Blowhard on September 29, 2003 11:35 AM

"I know it when I see it" algorithm--

It sounds like dog breeding, or goldfish, or any human-controlled artificial selection. You could have a "Filmmakers Kennel Club" developing breeds of movies for people with different interests. "I'd like to see some films from the terrier group, and some of the more aggressive hounds."

Of course a dog-breeding analogy would introduce sexual reproduction to the movie mutation algorithm--now that would be hard to model. But I'd love to see someone try!

Posted by: Nate on September 29, 2003 6:01 PM

"Whether that means it can "modify" physics for comic or dramatic effect I have no idea." That's really the crux of it, right?

Posted by: j.c. on September 29, 2003 6:03 PM

dave mercer:

clearly. I linked to work by Jordan Pollack evolving simple machines, their fitness being (in part) average velocity in a physics model. But a large "parameter space" doesn't imply one would necessarily want to use one. robot soccer, a predator and prey game, and video game AI are examples where simply coding dynamic behaviors, pathfinding, etc will on average outperform almost any GA NN. And typically when they do get used, they arent doing most of the work. it makes some sense for the optimization problem Friedrich cites (3d model ambulation - but judging from the small number of iterations he'd already narrowed the problem down; still, a useful heuristic - my original point along with the minsky link i provided in my original post is that there are other, better algorithms (in terms of success and time complexity) for many problems)

Friedrich:

of course we can think of the cliches like "think like the average 15 year old", therefore the movie should have some sex, gross out humor, awkward action scenes, if a buddy movie a tension between the characters, a love triangle (or the other variant, brother pissed off at buddy going out with his sister), some conflict with comic evil character, or lots and lots of action. we could get a lot more precise than this, but I'm not sure how that would help because it's possible you'll find plenty of movies that meet the formula that either aren't good or arent successful. and it sounds a lot easier to me to encode movies that already exist, then recognize according to some formula (like an extended genre selection tool), than to generate anything novel. if anything, it would be a safety net formula for movies whose financiers are too risk averse to go beyond the current paradigm.
But I don't know what the GA would contribute.. I assume scriptwriters will have a keen understanding of "neat things about being human", "what makes people laugh or cry", but to me there's a pretty serious encoding problem here. it's like assuming because you can get a robot car to stay on the road, stagger itself, and change lanes, that you could program it to do everything else a human does while driving. (there are things about being human and driving that arent contained in the formulation of the problem)

as far as "i calls it as I sees it"

a GA will typically be enumerating through an exponential number of possibilities (thousands, millions, whatever), so you'll need some sort of metric. how this will generate the "idea" for a novel script e.g. the movie Rashomon, I don't know.

Posted by: Shai on September 30, 2003 1:51 AM

I'm obviously not qualified to discuss the finer points of all this, but, upon reflection, what I find more interesting is not the usefulness of genetic algorithms per se to develop new story lines, but rather the analysis of story in terms of what genetic algorithms terms "genes." By which I seem to mean how one far can one model a fictional character as a chemical element, tending to interact according to a rule-based scheme, at least plot-wise. Is Hamlet, viewed as a story-gene, a character who will come up with one brilliant, dazzling delaying tactic after another until all hope of getting what he really wants (e.g., Mom) disappears? Is MacBeth a sort of machine for attacking men who are either father-figures to him or rivals who are themselves fathers, and who is easily manipulated by women questioning his manhood?

Posted by: Friedrich von Blowhard on September 30, 2003 10:53 AM

Shai: True about there being many simple algorithms that will often perform better than a GA NN in many cases. I've seen a GA NN proposed where a bounding box algorithm ended up being more than adequate.

But there are some crazy things where you don't know enough about stuff to get good results otherwise (such as bankruptcy prediction and some related things). I've been exposed to some proprietary intellectual property in use in the finance industry, and let's just say that there are folks who sell hundreds of millions of dollars a year of software that has GA NN's at their core. They don't trumpet that, but GA NN techniques are big big business in risk prediction.

Posted by: David Mercer on September 30, 2003 5:34 PM






Post a comment

Name:


Email Address:


URL:


Comments:


Remember your info?