This article was initially published on August 20th, 2015 by Mark Ristich
What would you do with another dimension?
Recently, I attended a talk by Dr. Ken Perlin of NYU to see his "vacation research" on Virtual Worlds and human interaction.
I was amazed by the cases that he described as "real experiences" in virtual space, because, from what I could tell, they were poor approximations of the experiences we ought to be having in virtual 3D spaces.
To his credit, Dr.Perlin is doing amazing work, looking far into the future of what life will be like post-VR. Guided by the clever observation that "technology is only things invented since we were born", he imagines children who grow up in a world that has already created virtual and augmented reality. These children will have vastly different perspectives on what this space ought to be used for, and will drive their own creative revolution in the way that they produce and consume inside virtual space.
It is worth mentioning that the trials coordinated between Dr.Perlin and Emily Carr University students have demonstrated some very interesting (to me) interactions that lead to a surprising conclusion:
We're being sold something that no one knows how to use. It's as if someone went to a restaurant and asked for a burger and fries, and the charming cashier was able to convince them to upsize to a bucket of caviar. We may have thought it was a good deal, but we're currently entirely unprepared to use it. People are gathering around the technology, whether because of FOMO or raw curiosity, but the real implications of using immersive 3D spaces likely won't be seen until we become accustomed to living (or at least playing) in them. We're getting an entirely new plane to interact with, and it is almost embarrassing how little we use it.
Let me explain by recounting the image presented - two 16-year-old girls we hooked up into Samsung Gears and networked into a virtual space using motion tracking sensors and OptiTrack cameras. This allowed them to move wirelessly through a virtual world. In addition, the subjects were given wands, each with a button that, when pressed, allowed them to draw pictures only visible in this virtual space. Together, in their virtual playground, able to see one another and their creative constructions, they decided to play a game. The game was Tic-Tac-Toe.
Tic. Tac. Toe. The most 2-dimensional of 2-dimensional games. The researchers were ecstatic that these participants felt so comfortable in a virtual world that they wanted to have fun. I couldn't share in their excitement, because I was absolutely shocked. The failure of VR, and people are cheering? I stood, eyes downcast, questioning this future that I want to create, until I asked myself "What game I would play instead?" It dawned on me that most of the games that we learned as adolescents actually discouraged playing in three dimensions. As we settled into desks and sought social interaction in quiet spaces, writing became the medium through which we communicated. Passing messages, scribbling doodles, and, of course, playing paper games such as Tic-Tac-Toe. These interactions were subtle and surreptitious, enjoyable, and most importantly, they permitted privacy. The space that we had could be folded inside a textbook and hidden from prying eyes, or crumpled up and thrown in the garbage to be forgotten. The scraps that we kept made their way into notebooks and journals which acted as the most sacred of the adolescent spaces - the personal diary or journal. Before you point out that this was a singular case, I should point out that this was a trend across the entire presentation. They put a research grad in the system to test it. He drew a two-dimensional horse, then tried to ride it. Others drew shapes on the ground as though they were using sidewalk chalk, or wrote messages for others to see. Nearly everyone perceived themselves in a 3D space, yet created uniquely 2D constructions.
The byproduct of technological limitation is that we are inadvertently teaching ourselves that much of the "real world" is 2D, and that the third dimension that we played in as children (sandboxes, alphabet cubes, Lego, MineCraft) is too simple for expressive communication. We grew into adults who type on flat screens, read flat books, order from flat menus. Nearly all the art we interact with in our lifetimes will be flat, and any sculptures often come with a caveat that they ought not to be touched. It's impossible for any of us to live in three dimensions when so much of our communication and interaction with one another takes place in only two. The third dimension is superfluous, used only for crazy adventures, amusement park trips, and the occasional mind-bending puzzle.
If we interact so rarely in three dimensions, it is even less likely that the average user will be able to create in three dimensions. For Virtual Reality, this is a real problem. If we want our users to be actors (which we do) instead of spectators, they need to know how to create in 3D. With this in mind, I suggest that with the arrival of Room-Scale VR/AR (such as SteamVR and CastAR) we incorporate tutorials. These tutorials could take many forms, but consider this simple example:
The user logs in to see a wireframe tetrahedon in front of them.
Why a tetrahedron? Two reasons.
- It has faces that express across 2D very poorly.
- The use of sharp corners and edges allows the user to "feel" the edge of the shape.
As the hand moves across the faces, they change colour and texture. I believe this simple experience is enough to reorient the mind to thinking in 3D, providing a primer for both the motor functions in the arm that draw from the shoulder (as a good artist ought to do) instead of from the wrist. By forcing a larger range of motion, you also introduce inaccuracy, which I expect would augment the sensation of presence. When doing fine-motor actions, our brains are fantastic at predicting what the final motion will look like. This is demanding on the hardware in these early stages, as micro-movements used in writing are currently very challenging to track without tremendously expensive equipment. Large, sweeping motions of the hands acting as units not only teach interaction with the medium itself, but also lead to accidental depth changes due to acceleration from the shoulder joint.
In short, I believe that our lack of working, reading, and generally living in three dimensions is not universal, but is a present concern. Future generations may not have these problems, growing up with access to virtual spaces from an early age may not lead to the same two-dimensionality of private spaces, and will encourage creative and constructive activities in 3D over longer periods of development. I'm eagerly anticipating these cases, alongside waiting to see more studies on the first experiences of VR on children and teens, which may only serve to disprove my argument here. If I am proven wrong, and adults adapt quickly to living in 3D, this only bolsters the possibility of immersive, virtual, social spaces as a new form of interaction.