This summer, Guillermo Del Toro’s Pacific Rim continued the illustrious Hollywood tradition of movie computer interfaces that look totally awesome and that no sane person would want to use. I don’t know about you, but if I was in the middle of helping building-sized robots fight monsters—and who knows, with climate change continuing apace, anything’s possible—I’d probably want to do something a little more precise than make what may or may not be circles in the air.
For all the silliness of Minority Report-style interfaces, though, their ubiquity in film makes sense: They are, after all, more visually arresting than, say, someone banging away at a keyboard. Yet, now that similar interfaces have started to infiltrate the real world—first with Microsoft’s Kinect, and most recently with the recently released Leap Motion—it’s also becoming clear there’s more to our affinity for these new modes of interaction than our appreciation for the whims of Hollywood’s VFX artists. Instead, the excitement over motion control seems to be about getting to “touch” the things behind the screen—as if what we really want is to break the barrier between the digital and the physical.
And, to be clear, Leap Motion is a disappointment, but mostly because the buildup to its release promised so much. A small, matchbox-sized trinket, Leap Motion sits in front of your computer and tracks the motion of your hand or hands, turning those gestures into commands a computer recognizes. Though the intro videos suggested a revolutionary experience of controlling your computer, like Kinect before it, the sheer wonder that you can wave or point to perform actions quickly gives way to a sense of frustration as the limitations of the technology set in. It’s not awful; in Windows 8 especially, it’s occasionally quite useful. But it has a habit of being finicky and unpredictable, and, at least in its current state, it just isn’t the future.
For all its failures, though, you can see what drove the excitement. In the moments when it works, you not only feel like you’re in a sci-fi movie, but, more importantly, as if you are pushing past that physical-digital border and interacting directly with the space on your monitor. There is a quasi-tactility to moving your hand left and having your screen move left, and in that familiar one-to-one quality, something like a feeling of immediacy and control. Unlike the abstract gestures of the mouse or pressing a black square to conjure an “A,” swiping through the air to produce digital results somehow feels “more real.”
That, however, may be irrational. When using Leap Motion, it’s easy to be reminded that interfaces are akin to language: They are the mechanisms through which we manipulate objects of meaning. The arbitrariness of a mouse-click to activate something onscreen is like the randomness of, for instance, the word “chair” meaning “a seat with four legs and a back”: It is only convention that dictates the relationship between the two. In that sense, motion control is like the onomatopoeia of computer interfaces—it tries to get at the object it represents by replicating it as best it can.
I’m ambivalent as to whether that’s a good thing. The recreation of the physical is both the best and worst of motion control—best because it makes things feel familiar and intuitive, but also worst because, well, isn’t the point of digital to be better than physical? If interfaces are like languages, it’s hard not to feel motion control isn’t a bit like wishing that the meanings of words would stay fixed or clear: it’s practical, but hardly artistic or innovative.
Maybe, then, computer interfaces shouldn’t aim to be “interesting.” One wonders, though, what will happen as the tech improves, and developers can start to maybe get a little more sophisticated than the computerized equivalent of “achoo!” or “brrrr!” After all, there is nothing “natural” or “tactile” about the pinch-to-zoom feature with which we’ve all become so familiar thanks to Apple (and others in its wake). Instead, it sets out a simple abstract grammar—move your fingers in this fashion to zoom in and out—that makes interacting with photos both efficient and satisfying. It is an action that doesn’t seek to replicate what we already know, and is better for it.
Pacific Rim, as well as Avatar, Iron Man and a host of other films, seem obsessed with how the bodily and technological interact, as if in their techno-futurist fantasizing, they dream of a world where the barrier between the two realms blurs and disappears. It seems that the desire to “reach through the screen” and make digital objects just like physical ones reflects a similar wish. I just wonder if this perfectly okay film fantasy isn’t somehow limiting in real life—if maybe what we want isn’t to place our hands into the digital and discover what we already know, but, instead, to reach into the virtual and find that which has no counterpart in the physical realm.
