3D vs 2D for Legacy Applications

When Alan Kay’s team at Xerox created the overlapping window user interface, they were working for a document company. Everything was organized around documents mimicking paper, sitting in folders except when being operated on by one application or another.

We don’t need the paper metaphor anymore, but we sure have a lot of 2D paper-oriented legacy stuff laying around. While 3D is pretty clearly winning for new applications in which people work together(*), it hasn’t yet demonstrated something so much better with which to replace all the existing 2D docs and their applications.

The state of user-interface design has provided two ways to deal with this: virtual computer displays embedded into the 3D world, and floating 2D windows. Both are pretty good and have their place.

Intensive Care Unit
One avatar is operating an in-world bio-signs display, which is also being shown in the lower left as a 2D panel floating over the view of the virtual room. The text chat is a 2D floating panel in the upper left, while the procedure timer and other applications are in-world.

(*)Without people, there is no realtime collaboration. 2D includes people only as second class abstractions rather than reifing them as first class objects in the communications model. For teaching, training, and working meetings (as opposed to mostly unidirectional sales presentation meeting and lectures), 3D has emerged as the most natural way to show individual presence, and the simplest way to provide enough immersion to give a sense of shared presence.

Even without people, 3D gives you context. The spatial relation between items gives you a sense of their relationship, as it does in office planning, home decorating, or museum design. Sometimes this matters more than other times. For example, if you’re simulating a physical operations room or command center, you really need to be able to see what will be visible and useful from various positions, so the live interactive computer displays really have to be properly positioned and oriented at their in-world locations.

Then, too, if you’re in a situation where you’re using 3D because of what it gives you for working together (a sense of immersion and of who is doing what), you need the applications to be in-world.

whiteboard and monitor
The whiteboard and computer monitor are directly operable in-world.

Sometimes, though, you don’t need the same shared context as everyone else. Sometimes you would like to keeps a general idea of who is doing what while you develop your own personal perspective on the documents. You might even want to bring together two running applications side by side that are not normally even visible at the same time in the 3D world (e.g., because they are on opposite walls). In such cases, it is nice to be able to click on the in-world displays and bring up a floating 2D window on just your computer that gives you a nice flat view of the live document. You can re-arrange such windows to suit without disturbing the shared 3d setup.

Some worlds provide one or the other of these, but I think it’s important to have both available. Virtual worlds are potentially isolating enough at their best, without giving it all up to have those present really not “there” but instead each working on their own 2D windows. But with all legacy experience we have, there’s no point making a 3D world be a worse experience than what can be done with messy-desk windows.

multiple media, multiple modes
Several kinds of media and applications are operating simultaneously. Text chat(3) and video chat(4) are in 2D floating panels. Readouts on people(5) and places(9) are in fixed 2D displays. Various other applications are operating in-world.

While the 3D and 2D usages both make sense to me, I feel they do so only when there are multiple applications operating simultaneously. If a virtual world can only show you one document, picture, slide show, spread sheet, or video at a time, then it sort of doesn’t matter what you use because it’s not really then a collaboration and you might as well use some 2d single-media tool.

About Stearns

Howard Stearns works at High Fidelity, Inc., creating the metaverse. Mr. Stearns has a quarter century experience in systems engineering, applications consulting, and management of advanced software technologies. He was the technical lead of University of Wisconsin's Croquet project, an ambitious project convened by computing pioneer Alan Kay to transform collaboration through 3D graphics and real-time, persistent shared spaces. The CAD integration products Mr. Stearns created for expert system pioneer ICAD set the market standard through IPO and acquisition by Oracle. The embedded systems he wrote helped transform the industrial diamond market. In the early 2000s, Mr. Stearns was named Technology Strategist for Curl, the only startup founded by WWW pioneer Tim Berners-Lee. An expert on programming languages and operating systems, Mr. Stearns created the Eclipse commercial Common Lisp programming implementation. Mr. Stearns has two degrees from M.I.T., and has directed family businesses in early childhood education and publishing.


  1. Is this Croquet?

  2. Yes, Qwaq Forums is one of at least two commercial applications built on Croquet. (The other is http://www.planet-plopp.com/ from Impara. In addition, several companies offer services based on Croquet, such as http://greenbush.us .)

    Alan and David Reed are our advisors, who with David Smith and Andreas Raab created Tea Time. Smith and Raab are our architectural leads. Josh Gargus and I work for Qwaq, but we previously launched the Croquet Collaborative with the first public release, public router, and extended end-user public uses of Croquet, which was work that began with Julian Lombardi at the University of Wisconsin. (Lombardi is now at Duke.) The first embedded apps were through that work, and Gargus continues to architect embedded media at Qwaq.

    Some comments on milestones and influences for the project are at http://wetmachine.com/i

Comments are closed