Videogames: a Living Work of Science Fiction

For years now we in the video-game community have had to look up to the film industry as far as art is concerned. Obviously, since there are no real limitations other than the animation and modelling tools themselves, the sky's the limit as far as film is concerned when it comes to poly counts, texture resolutions, mutli-passes, etc.

We've always been picking up their scraps, as it were. Trying to squeeze in this technique here, possibly adapting that technique there. Occassionally lusting after the next generation system so we can push the envelope one more inch closer to what we see on screen, graphically.

But in a twist of fate, it would seem, the film industry has something to learn from videogames.

In the newest LotR movie: Return of the King, Peter Jackson turned to programmers to have the hordes of minions and soldiers react in real time to events happening on the battlefield.

"So each of these computerized soldiers is assessing the environment around them, drawing on a repertoire of military moves that have been taught them through motion capture - determining how they will combat the enemy, step over the terrain, deal with obstacles in front of them through their own intelligence - and there's 200,000 of them doing that."

This sounds familiar... Oh wait! It's like what I take part in everyday at my job! Motion capture animation sequences, and put them in the game engine while programmers script them to be reactions to in-game events. The only difference here is scale; we can't possibly have 200,000 characters onscreen behaving independantly. Hell, we'd be lucky to have 200,000 characters onscreen; it's a limitation of the hardware.

The only thing I found upsetting about the article explaining all of this is that there was no reference to how similar - if not downright borrowed - all of this is to video-game AI. All they've done here is re-create the wheel on a much larger scale (granted, a very impressively large scale). The only thing that's new about this is that it's in a movie, and not a video-game.

"So to create these individual agents, there was a code that was especially written and developed," Taylor says, adding that it was like being involved in a living work of science fiction.

We know, Mr. Taylor. We've been living and playing in virtual worlds of science fiction and fantasy for decades now. Perhaps you might like to join us.

Comments

[TitusByronicus]

AI is not stricly the domain of video-games, though. Robotics and simulation programs (outside the scope of games) are also heavy on AI. Does WETA really have to cite video games on the inspiration for their technology? Especially since it doesn't appear they are claiming to have invented the idea, just implementing it (for the first time?) on a huge scale for a movie.

PS. I was amused at their anectode about having to dumb-down the AI because the armies kept running away from each other :-)

[Philip]

Many academics working in ecology try very hard to develop models which simulate the behavior of populations and communities. The standard method for this is set up a system of differential equations (DE's) and find a computational approximation for the solution. Very recently, a few ecologists have started to explore individual based models (IBM's) as a tool. Some interesting work has been done, but few (if any) major breakthroughs have come about.

Perhaps they need to stop thinking so academically, and instead get some video game AI designers working on this. If we can answer the question "How 'dumb' can individual soldiers be and still have the army retreat from battle?", perhaps we can answer the question "How sensitive to food scarcity can an individual organism be and still have its population persist?"

[Kurayamino]

you lot might want to watch the appendix DVDs on the extended edition LotR The Two Towers, theres a lot about the MASSIVE program they used.

and they didn't dumb down the AI because they ran away from eachother, they AI was too dumb, it didn't see an enemy, so they kept running. it was thier default state. "run forward"

thats said, watch the WETA digital stuff on those DVDs and get the story from thier own mouths =P
what that program can do is nothing short of damn cool.

[Kurayamino]

and in case your wondering,
they used this

[Bowler]

Titus: "Robotics and simulation programs (outside the scope of games) are also heavy on AI. Does WETA really have to cite video games on the inspiration for their technology?"

I would say yes, because no one in robotics (or really simulation for that matter) has attempted to re-create large volumes of human characterization to replace actual humans in a simulation. Video-games did it first, and until now, did it better.

Kurayamino: I'm going to check that out. Thanks!

[Bill C]

Bowler,

I think you are slightly off target here. I agree that video game AI pathfinds and then attacks a target with a preset animation or two.

What MASSIVE does is more impressive in terms of the level of realism of those moves, and the fact that when two *AIs* are fighting they thrust, parry, riposte, etc. and look like virtual stuntmen doing so. (Ok look like virtual stuntmen shot in a sweeping helicopter shot over a field of battle that is highly chaotic.) :)

bc

[Bowler]

Bill, I say to you "don't believe the hype."

Remember, this is what I do for a living. I work with this every single day. Video-game AI has moved so far beyond what WETA would like to call "binary AI". Go watch any fighting game running in Demo mode (where the computer takes the roll of both sides). Street Fighter. Tekken. Soul Calibur. If you didn't know it was demo mode, could you discern the difference between a human playing or the computer?

Massive likes to think that they've invented this "fuzzy logic," where their AI uses multiple "grey states" to allow blending between animation states. This sounds a lot like "bubble logic" that was first seen in the Reaper bots as far back as Quake 1 (the same AI logic that went into Unreal's bots and I think Half-Life's). Hell, in the game I'm working on now, our defensive AI is constantly blending between four animations at any given point in time just for his movement alone. This isn't anything new that they're using.

Also keep in mind that Massive looks good when it works because it's using 200,000 characters onscreen at any one point in time, the shots aren't left onscreen for more than a few seconds each, and the view is almost always from such an extremely long range shot that it's difficult to even see detail. With parameters like that, even a trained eye can't pick out individual flaws and errors.

I'd like to see a Massive AI test on just two soldiers, close up, interacting and fighting. I'd bet real money that what you'd see would be no different than what you'd see with video-game AI.

[William]

What Massive does that videogame AI doesn't - because it doesn't need to - is scale, erm, massively, and also be finely tweakable. After all, this is about direction, not modelling. Having a suitable interface for shifting weights to generate desired behaviours en masse, with the result being a large number of agents with widely differing inputs, whose behaviour is also an input for nearby units. Is your neighbor panicking? You're more likely to panic. Wedding this sort of straightforward network to a more action-oriented wireframe "NPC" is what's happening here. In Virtua Fighter, you may have some good fighting AI's, but I don't say any state that maps onto beliefs or perceptions.

The closest I've seen to all these elements assembled together in a videogame has been in the Total War games.

[Bowler]

William: "After all, this is about direction, not modelling. Having a suitable interface for shifting weights to generate desired behaviours en masse, with the result being a large number of agents with widely differing inputs, whose behaviour is also an input for nearby units. Is your neighbor panicking? You're more likely to panic. Wedding this sort of straightforward network to a more action-oriented wireframe "NPC" is what's happening here. In Virtua Fighter, you may have some good fighting AI's, but I don't say any state that maps onto beliefs or perceptions."

But again I have to disagree. Computers cannot emote. They don't have beliefs or perceptions. They can fool us into thinking they can. This is displayed in an animation. They only have default "states" that they can exist in that are dependant on an animation assigned to that state. To a computer, [paraphrasing] "being more likely to panic if your neighbor is panicking" is no different to "being more likely to block if your neighbor is throwing a punch." It looks at what it's neighbor is doing, and assignes an animation that is appropriate. It doesn't matter if that animation is block or panick, attack, or run away. That is assigned in a table of states which are displayed when the situation demands them. Yes, Massive did this on a "massive" scale. I've admitted as such. My issue is that videogames do exactly what Massive does, before Massive did them, on a smaller scale.

Take a look at the AI in Winning Eleven soccer. It's managing a delecate balance of keeping players in their positions (respective states) while reacting to what the players are doing around them, while attempting not to break any of the rules of soccer. On defense alone they have to decide to either attack the ball, encroach on the ball carrier, give more ground, cut off the pass, play more passively, get aggressive, etc, at any given moment. These states are also layered with the player's commands of playing in a different formation on command, and having the pace of their game dictated by the player (be it in protecting the lead or attacking the goal mouth). It's exactly the same as what's going on in Massive's AI engine, only on a scaled-down level due to the limitations of hardware constraints and the game itself.

I'm not saying what Massive did wasn't impressive; just that videogames did it first.

[William]

While I'll dodge the question about whether AI's have emotions as too closely related to questions of qualia (if there was a network of values which made the behaviours we associate with anger more likely, and were triggered by the stimuli we associate with anger, would we say it's not anger just because we don't know whether it "feels" like anger to the agent?), I disagree about the beliefs/perceptions issue, starkly. A belief is a proposition about a state of affairs in the world: any simple OCR program has perceptions (an input array) and beliefs (that the input array maps onto the letter "Q", for example, with some percentile of confidence.)

[Mosher]

This is much more informative, and actually gives a nod to the gaming industry:

"Game people are the ones who are driving it. They have to do this to be competitive. But the live-action industry will embrace this approach like crazy. When they can feed their ideas verbally into a system and don't have to wait two weeks for animation to come out of a 'black box,' they'll love it."

[Bowler]

Thanks for the link, Mosher. While I loved what they had to say about Massive's roots in video-games, I could write volumes about what they don't know about animation and motion capture, but that's for another day, and for another forum.

[Johny Zuper!]

I don't think the technology is the largest influence from games in The Return of the King. In my opinion, it's the way human bodies are used as abstract rigid physics objects that can be thrown around in utter disregard of the living creatures that they represent. It reminded me of the level of abstraction in The Matrix: Revolutions. Only in that movie, at least it was just machines hurling around, making pretty patterns. New in cinema and stolen from games: disrespect for human bodies. Hooray?

[False Prophet]

Thanks for the interesting revelations. I was rather impressed with how realistically the sprites (can we still use that term?) reacted in the face of combat. Unfortunately, my enjoyment of the mass battle scenes was diminished because both armies had no sense of tactics.

This article goes a long way in explaining that, at least. The soldiers on each side were not acting as a disciplined unit that trained together, but as a horde of individuals with little concern for his fellows:

"I want battles like nothing anyone has ever seen on screen," Jackson said. "I want every soldier fighting for himself - you have to come up with something."

"So each of these computerized soldiers is assessing the environment around them, drawing on a repertoire of military moves that have been taught them through motion capture - determining how they will combat the enemy, step over the terrain, deal with obstacles in front of them through their own intelligence - and there's 200,000 of them doing that."

That might be what you wanted, Peter, but that's not how armies fight. Not successful ones, anyway. This trilogy was a fantastic cinematic event, a great showcase of what can be done with CGI these days, and the films rank among my favourites of all time. But I still cringe when I see the battle scenes. I'll take the mass battles of Spartacus, Braveheart and Gladiator over Jackson's video games any day. Not because they use human soldiers but because the directors actually researched how wars are waged.

[Nick]

FlaseProphet: But he does get very close to Tolkeins descriptions of the battles which if we remember were written in present day as mythology. The latest in tactics was pretty much Beowulf shield wall.

Not to disagree that most of the massive rendered scenes aren't 'helicopter' style shots but there are a few shots in the Two Towers of the Orc army marching on Helms Deep from about 2 feet away that are 100% Massive pathed (I suspect all of it was rendered in Pixars Renderman) or at least so they claim on the commentry in the Two Towers extended edition.

[outsider]

jpb ppyt psycholog zdrowa ywno nieruchomoci projektowanie stron agencja reklamowa soczewki kontaktowe nauka angielskiego agroturystyka opony klimatyzacja domy opieki akupunktura hydraulik projektowanie wntrz soha jpk paa ki wypadki tfrd jh sw jft pp fdr

[pppp]

strony www wrocaw, tworzenie stron, lub, hale namiotowe, rozdzielnice, flash lite games development, adidas nike puma, buty nike