3D vs 2D for Legacy Applications

When Alan Kay’s team at Xerox created the overlapping window user interface, they were working for a document company. Everything was organized around documents mimicking paper, sitting in folders except when being operated on by one application or another.

We don’t need the paper metaphor anymore, but we sure have a lot of 2D paper-oriented legacy stuff laying around. While 3D is pretty clearly winning for new applications in which people work together(*), it hasn’t yet demonstrated something so much better with which to replace all the existing 2D docs and their applications.

The state of user-interface design has provided two ways to deal with this: virtual computer displays embedded into the 3D world, and floating 2D windows. Both are pretty good and have their place.

One avatar is operating an in-world bio-signs display, which is also being shown in the lower left as a 2D panel floating over the view of the virtual room. The text chat is a 2D floating panel in the upper left, while the procedure timer and other applications are in-world.

(*)Without people, there is no realtime collaboration. 2D includes people only as second class abstractions rather than reifing them as first class objects in the communications model. For teaching, training, and working meetings (as opposed to mostly unidirectional sales presentation meeting and lectures), 3D has emerged as the most natural way to show individual presence, and the simplest way to provide enough immersion to give a sense of shared presence.

Even without people, 3D gives you context. The spatial relation between items gives you a sense of their relationship, as it does in office planning, home decorating, or museum design. Sometimes this matters more than other times. For example, if you’re simulating a physical operations room or command center, you really need to be able to see what will be visible and useful from various positions, so the live interactive computer displays really have to be properly positioned and oriented at their in-world locations.

Then, too, if you’re in a situation where you’re using 3D because of what it gives you for working together (a sense of immersion and of who is doing what), you need the applications to be in-world.

The whiteboard and computer monitor are directly operable in-world.

Sometimes, though, you don’t need the same shared context as everyone else. Sometimes you would like to keeps a general idea of who is doing what while you develop your own personal perspective on the documents. You might even want to bring together two running applications side by side that are not normally even visible at the same time in the 3D world (e.g., because they are on opposite walls). In such cases, it is nice to be able to click on the in-world displays and bring up a floating 2D window on just your computer that gives you a nice flat view of the live document. You can re-arrange such windows to suit without disturbing the shared 3d setup.

Some worlds provide one or the other of these, but I think it’s important to have both available. Virtual worlds are potentially isolating enough at their best, without giving it all up to have those present really not “there” but instead each working on their own 2D windows. But with all legacy experience we have, there’s no point making a 3D world be a worse experience than what can be done with messy-desk windows.

Several kinds of media and applications are operating simultaneously. Text chat(3) and video chat(4) are in 2D floating panels. Readouts on people(5) and places(9) are in fixed 2D displays. Various other applications are operating in-world.

While the 3D and 2D usages both make sense to me, I feel they do so only when there are multiple applications operating simultaneously. If a virtual world can only show you one document, picture, slide show, spread sheet, or video at a time, then it sort of doesn’t matter what you use because it’s not really then a collaboration and you might as well use some 2d single-media tool.