Why 3D Doesn't work for TV,But is great for Gaming?

Jimi Hendrix explodes onto the screen in the Central Hall of the Las Vegas Convention Center. It's an iconic, well-worn sequence of archival footage from Woodstock, the stuff of retrospective montages and '60s music anthology infomercials. Only now, it's in 3D. Hendrix seems to pop out from the screen, while the stage and band members are mashed flat behind him. A crowd shot reveals not so much a massed sea of fans, but sliding layers of them. Just before the Sony logo appears, one of those fans flashes the peace sign. It doesn't look like a three-dimensional hand. It's more like a cardboard cutout, jutting out into space like a half-finished work of hippie origami.

This breaking up of old footage into somewhat arbitrary layers is the opposite of remastering. And it's one of many problems with 3D entertainment. Sometimes 3D is eye-popping. But not everyone fell in love with Avatar's 3D wizardry—with so much groundbreaking digital performance capture and panoramic CGI on display, why suffer through the migraine-inducing view from behind a pair of RealD passive polarized glasses just to experience the occasional moment of vertigo? Even for those who were unabashedly amazed by Avatar, did the magic of its 3D elements really last past the two-hour mark? And sometimes, as in the case of Jimi Hendrix, 3D is just a bewildering parlor trick. In either case, it never manages to be more realistic than 2D, since it can't copy the way the eye flickers from foreground to background, or mimic the more complex cognitive function of processing depth. In movies and on TV (and with all due respect to Avatar's $1.3 billion box office) 3D will never be much more than a gimmick. But everything that makes it so wrong for television and movies makes it perfect for video games.

At CES last week, the atmosphere at Sony's 3D gaming area and at Nvidia's panoramic 3D Vision Surround gaming demo stations was totally different from that at the many booths with 3D TVs on display. Middle-aged regional distributors slip on the same glasses that they had just used to watch animated characters over-emote in Monsters vs. Aliens on Blu-Ray, and looked at the same LED-backlit displays, but now they were grinning like children, belting out things like "whoa" and "that's cool!" Some games looked better than others in 3D—Avatar: The Game, ironically, featured strangely flat, 2D water effects, while the downloadable PlayStation Network title Super Stardust HD, with its tiny arcade spaceships and ball-like revolving planetscape was awesome. One of the most impressive glimpses of 3D gaming, though, wasn't playable. A demo reel at Sony's booth showed 3D footage from various games, including the off-road racer MotorStorm Pacific Rift, and a spine-quivering jump from a cliff. It's not the first video-game stunt to induce vertigo. But it might be the first to inspire a complete stranger to turn to me, bug-eyed, and say, "Holy s**t."

The point here isn't that all games are universally great in 3D. There are bound to be specific 3D effects that work better in some games, and some games that are naturally more or less of a fit for 3D. Gran Turismo 5 is exciting in 3D, but is it exciting enough to justify buying a $50 pair of glasses and high-end LED 3D-ready TV, not to mention a family-size bottle of ibuprofin for the resulting headache? On the other hand, playing Avatar: The Game on Nvidia's three-monitor setup is completely astonishing: Its use of 3D is more consistent and universally effective than in the movie. Somewhere in the middle is the 3D-ification of Little Big Planet, which was already composed of layers of 2D imagery, now just a little more visually striking with those layers placed more distinctly forward or backward. The effectiveness and overall impact varies, but nowhere in these games does the 3D look bad. There's no equivalent of the shoddy, cardboard cutout effect seen in that Hendrix clip, or in other hit-and-miss footage from 3D movies or television.

There's a simple technical reason for the high quality of 3D visuals in games—virtual cameras are easier to control than physical ones. 3D's central optical illusion is pulled off by either filming with two cameras, which recreates the stereoscopic effect of perceiving depth with a pair of eyes, or else massaging existing footage until it kind of looks as though it has depth. The latter nearly always looks bad—the image is still mostly flat, but now it's flat in layers. The former, which is referred to as native 3D content, requires incredibly precise double-camera rigs, and generally some amount of post-production cleanup to keep pixels captured by both cameras from straying too close or too far.

On the other hand, maintaining image quality in 3D games is as easy as inserting an additional virtual camera into the game, whether a game is developed in native 3D or retroactively patched for 3D, according to Atsushi Honda, an electrical engineer who worked on Sony's new 3D LED displays. Developers routinely add in-game independently moving cameras to a given gaming environment, such as in split-screen multiplayer modes. For 3D, the only difference is that both cameras are "grafted" onto each other, with a tightly controlled amount of space between them (enough to accomplish that depth-perception trick). More processor-heavy is the fact that, unless developers are willing to live with a plunge in image quality, the frame rate essentially doubles, to some 120 frames per second. The PS3 should be able to handle this graphical bump to 3D with a firmware upgrade, available later this year. And although everyone else at the Sony booth dodged the question, Honda confirmed that the PS3 will run in 3D on any 3D-ready TV, so long as it's compatible with HDMI 1.4, an updated transfer protocol.

As important as the PS3's imminent 3D upgrade might be for the technology to go mainstream, PC gaming has been compatible with 3D for years. "Every PC game, starting from later versions of DirectX 8 [a suite of development applications], had the 3D information in it," says David Chechelashvili, head of the gaming division at 3D hardware-maker XpanD Cinema. "There was no media, however, to display it." The dual virtual cameras have been running since 2002, but until recently there was no way to check the quality of the combined footage, or to tweak a given scene or 3D effect. The result is a library of some 400 PC titles that will run in 3D right now, provided you have the right setup: a monitor that's 3D-ready, a graphics card that can tap into Direct X's 3D capabilities and active shutter glasses like the ones available from Nvidia. (By this summer, XpanD will sell its own set as well.)

Game developers' experience with native 3D, plus the inherent precision of virtual stereoscopic cameras, which don't require NASA-grade stabilization motors to keep out-of-sync frames from melting into an eye-crossing haze, might explain why some 3D games look good, some look great, but none look truly bad. "In terms of image quality and impressiveness, gaming is far, far ahead," says Chechelashvili. "With movies and entertainment on TV, you see a more varied opinion of 3D. With games, everyone loves it, except for those die-hard skeptics, who aren't talking about the way the games look. They're just saying we're never going to wear those stupid glasses in front of the TV."

There's one other possible explanation, though, for why 3D seems to be such a perfect match for gaming. It's harder to quantify, and even harder to defend. Here it is: No matter how incredible the graphics are, and how immersive the gameplay is, games are silly. They are unapologetically gimmicky, and only nominally realistic. Ammunition counts and targeting reticles are stamped into your virtual mind's eye, and peripheral vision is replaced by red semicircles and dinky little grenade icons pointing to incoming threats. You're a half-blind, half-deaf ghost in every game world, able to hit 120 mph on a dirt road without feeling the engine screaming in your chest and the steering wheel bucking in your hands. Time, space, even physics are all approximations, and you learn how to interact with all three. So when the alien jungle you're crashing through in 3D looks like one magnificently painted strip of cardboard stacked behind another, or the view from the driver's seat ignores the windshield entirely, that's okay. The sense of interactive depth when your pistol rounds streak downrange and thud into an alien creature, or the visceral panic that hits the moment before you drift into a wall—that's what 3D is for. Think of it as a consolation prize—even if the biggest gamble in consumer electronics crashes and burns, 3D is still well positioned to become the new state of the art for video games.

And if you think gamers can't bear to put on a bunch of dumb-looking glasses and embarrass themselves in front of each other, consider Dance Dance Revolution, Guitar Hero, Rock Band and every single game on the Nintendo Wii. Gaming is a safe haven for silly. 3D will fit in just fine.