SOFT Software Framework for Teleimmersion

What is SOFT and why we believe in it

The virtual environments/3D visualization software (not hardware) community has been plagued by a long-term, recurrent problem: It always seems to be coming out of the starting gate again, as if each research site or company is making up the whole concept of dynamic VR software for the first time. There have been numerous good ideas tested and published about VR user interface widgets and data visualization techniques, and yet these are rarely seen or tried again. Admired ideas from different labs are even more rarely combined together in a compound system and tested.

For instance, is anyone using ideas from Sketch to set up starting conditions for simulations of wooden structures under stress? There are some great simulation tools for this problem (which matters a great deal to those of us who live in wooden houses in earthquake-prone areas) but they aren't used as much as they should be because of difficulty in data entry. In the current research environment, there isn't enough personal time for tool originators to work with each other and/or end users to make all such good ideas come about.

Another example: There have now been clinical trials to demonstrate the advantages of surgical simulation, and yet there is no instance of a surgical simulation tool being used except by its originators. These tools are so idiosyncratic and indivisible that they can only survive in the womb.

The only examples we have of sites sharing work are where there's a lead site that manages clones. That's what's happened with EVL and CAVE users, for instance. We can't all be clones.

In other computer science research communities, researchers have been able to take advantage of each others' work because there were established constants in the implementation environment that created an anchor for variation. For the historical canonical instance, in the UNIX community, one could use modular tools like GREP in unanticipated ways, without requiring an affiliation with the originators. Everyone's work connected through the idea of text command lines and files for i/o.

If the VR software community could share pieces of the big puzzle like this; if we could count on the fruits of our labors being reusable, compoundable, scalable then we would start working as a community instead of as lone labs. That is how you make improvement happen exponentially instead of linearly.

So, let's suppose you believe that great user interface and visualization techniques are vital to the well being of humanity, which is facing a complexity explosion in everything (I believe that to be true), and let's say you are frustrated that the researchers at the cutting edge of that work seem to each be nurturing their own local development bubbles (I have that frustration). Then you are compelled to search for the anchors that might create exponential cooperative progress, a la UNIX.

As I look at the state of VR software, I can see lots of potential candidates for this anchor for sharing, but most of them don't hold up to scrutiny. One examples: You could create a standard for static mesh geometry and hope it does for VR what text did for UNIX. Well, we have VRML1 and it doesn't do all that much for us- the static geometry isn't the hard part.

Another approach: You could try to find a standard way of describing "behavior" and get everyone to use it. The problem here is that you either end up with a narrow set of behaviors (VRML 2) that cannot implement non-trivial ideas (like, say, Randy Pauch's "pinching" for object picking), OR you end up demanding that everyone buy into a single development environment, often packaged with a concept of how control is organized (as in JAVA 3d), and you face a very very tough sell.

One hard thing about virtual environments, as opposed to other UI work, is that in order to experiment with varying ideas about continuous control, real time interaction, and lots of stuff going on at once, you simply have to be able to change the outermost loops as well as the innermost ones. All the previous CS research communities were able to tie down the outermost loops and share inner loop modules. We can't do that.

Even if you could convince yourself that you had found the perfect framework for sharing tools related to behavior, I believe it would be politically unrealistic to expect to get a critical mass of tough-minded VR researchers to buy into one golden solution at this time.

I have thus far been able to see one strategy, and only one, for making VR researchers into an "exponential research community", and that is what SOFT is supposed to do.

If you look around at the many varied users of VR and VR-like tools, you see extraordinary variety. What do battle scene simulators, DOOM/QUAKE players, surgical simulators, and scientific visualizers have in common? They use different development tools, hardware platforms, concepts of outer loop control, and input/display devices. They share less than any other community of CS researchers. But there is one thing that just about all of them have bought into- scenegraphs.

They use different scenegraphs, true, but the differences are not in fundamental concept, only in implementation. THAT is why SOFT is designed to be a vampire of scenegraph calls.

If we treat a generic concept of scenegraph as the "real-time buss" that runs between sites, we can make any local tool function as a networked, shared tool, and we can make our various pieces of the big puzzle scalable, and compoundable.

The world I want to live in looks like this: If one lab makes a virtual hand tool in JAVA3D that sifts through a point cloud and changes the colors of certain visible points to warn that they represent overlapping data points- and another lab develops a hand tool in Body Electric that renders and adjusts nested translucent bubbles within point clouds to indicate statistical deviance in spatial distribution, these two tools can be used at the same time on the same point cloud without modification, and they can be used by a scientific visualization person who has very strange ideas about outer control loop design, and who never talks to the tool builders. (These tools might not run very fast when they're combined at first, but if the compounding seems worthwhile, the developers can work together to improve performance.)

SOFT will set the stage for exponential progress.