3D Interfaces

[handslive]

I've often (or hoped would maybe be better) that the 2D 'desktop' meme would eventually be replaced a more 3D interface. So far, what one usually sees is simply additional decoration on the original flat concept. Little polygons of content stacked limitlessly on top of each other.

One of the basic problems I've seen spouted when moving to 3D interfaces is with navigation. How does one go about arranging content, applications, and basic tools in a 3D environment. There are a number of straightforward solutions to the problem of applications and tools:

• 2D menus that can be brought up with a gesture or appropriate command
• a "spherical" style interface (which suggests the idea of infinite scrolling in any direction) where a sphere appears in front of the user that can be rotated appropriately to move to new menu items (this idea comes straight from Tad William's Otherland series)
• "open architecture" rooms designed around navigation elements (3DNA)
• single rooms with, again, application icons or shortcuts arranged in the space (3DTop) although I think this is a little flat unless there's more control over the shape of the room

But I honestly think that access to applications is usually least of our problems. What we have much bigger problems with is organizing the data associated with these applications. Newer desktop designs that associate applications with content, and this actually goes much further back to Unix CLI interfaces with the 'file' command and magic numbers at the start of files to identify different types of content. What we need help with is organizing our data space.

The normal 2D interface presents problems because of the general uniformity of appearance. Even back with Windows 98, the use of a more web-based navigational approach) allowed the user to alter backgrounds for different folders or the appearance of the file listings in different folders, like by showing thumbnails (KDE and XP use this more extensively). But providing bookmarks to common locations and showing different backgrounds is still not completely intuitive to my mind. And how many people actively consider using such memory aids when organizing their data?

I keep thinking of something Neal Stephenson hints at in Snow Crash and more explicitly John Crowley in Little Big, both of which refer to memory aids used by orators in ancient Greece. People's memory space is much more related to location, landmarks and the paths for getting places.

So, why have only one room? What we need is a way of adding new 3D spaces to the existing navigation space and building paths to these new spaces. Something that more resembles neighborhoods or large houses. I don't think Gibson's notion of cyberspace organized this way, or Stephenson's Snow Crash either, really makes much sense except in the "mall" notion of browsing and shopping. These are fine for very specific on-line experiences but not as dataspace. We need something personal for our own data, easily customizable.

What we want to avoid is the uniformity of data space, the flat uninteresting view of organizational hierarchy. That kind of thing works well for the anal retentive geek (hey, I've got 3 and 4 deep menu hierarchies on my home desktop's Start button). My mother-in-law has a single folder with everything in it. Usually with names like "New Document" and "New Document (1)" and "New Document (2)". I know she's a little more organized with her physical world. The problem is all documents look alike on the digital surface and they're all in flat, white boxes that look identical. People will probably still put stuff down and forget what they did with it, but hey, we've got search engines now on our desktops for finding things.

The ironic thing is that there are a lot of 3D games with editing tools for creating architectures and rooms that would probably do with some clean-up for the limited needs of the ordinary user. Especially if you make it possible to drop in pre-fab rooms (like themes). OEMs are distributing systems with limited 3D acceleration now so the computing needs of fairly static 3D spaces displayed in 1024x768 isn't out of reach for even a simple workstation. But most of the sample 3D desktops feel amateurish or clumsy. Why, when Doom and Quake are in the public domain now?

Comments

[handslive.livejournal.com]

Just as a joystick doesn't cut it as a pointer controller for the traditional 2D desktop metaphor, likewise a mouse probably doesn't cut it for a 3D environment. We probably need something else ... a data glove? Something else?

I actually think that a combination of keyboard and mouse presents a reasonably good 3D interface. Certainly, a variety of pretty good approaches have been tested in games and found useful. Obviously, my personal favourite is the first-person shooter paradigm: the mouse controls POV and the keyboard controls vector. Most of the freebie desktop replacements use this model. A second favourite is the one employed by Homeworld. In this one, a specific navigational point is selected as the origin and POV rotates around that, always looking "in". This interface had many useful features, but is based pretty closely around the idea that the navigational space is enclosed and limited. I think the first-person point of view is better because it automatically suggests paths and separated spaces, which is more natural to our notion of "location" and "purpose".

You bring up the idea of "room" and other similar concepts of location. But does the interface have to imply storage at location?

I'll admit that what I'd be aiming to do is move the current informational model directly from a 2D to a 3D environment. My thoughts are related to what I think of as the benefits of doing so with very little additional modification. The advantage of this, which I'm too lazy or too cowardly to examine personally, is that I believe an existing 3D engine (Doom or Quake by example) could be used as a base for implementing it over the already present file system arrangement. All that would really change is the UI, not the underlying infrastructure.

More importantly, I think most discussion about 3D interfaces is completely theoretical. The existing user interfaces are, in my opinion, based entirely around the applications, which these days means our menu systems. They're stilted and hard (or impossible in some cases) to modify to any extent. And they resemble closed boxes. We don't need a better interface for our menus; we need one for our file systems and data.

Improved "stickyness" of data, so that related things can be found easily, would be important in any interface. But how do we, as users, find the data unless we either employ ubiquitous search engines (so the availability of data is based on meta-data or keywords) or resort back to location? In either case, how do we show the connectedness of the data? The problem is broader than just 2D vs 3D.

Another question -- what widgets are available in 3D? What do they look like? How do I interact with them?

I should show you Worldcraft. This is the dominant editor for Half-Life (and quite a few Quake 2 based) games. The two other editors I've looked at for 3D games have many similarities with Worldcraft, but the problem spaces are identical so that's no surprise. One interesting element of most new editors is that they go to some length to present the editing environment similarly to the game POV where they can. Worldcraft feels a little clunky in this regard, but was certainly pointing the way in the mid-90s.

But it's pretty easy to show what important elements could be taken out of these editors and made part of a highly user-friendly tool for constructing spaces in which to store data. And obviously, there'd be a niche for pre-fab spaces and objects so the user only has to make minor (or no) tweaks.