Inventing the Future

We do a lot of live demos. Our VR software is in alpha and we sometimes run into bugs, but we pride ourselves on showing the true state of things instead of slides and videos. It’s often a build with radical changes created just a few moments before. And yet the most common trouble is the unpredictable network connectivity one finds at demo venues. Our folks have done quite a few tethered to a cell phone.

But this demo from Sony is in a whole other class. Ouch. I’m sure we’ll all have great hand controllers within a year or so, but right now, hand controller hardware really sucks.

By the way, I think their IK looks great, and I especially like the jump. But notice that the avatars are stuck on a pedestal instead of moving around. I haven’t seen anyone combine IK with artist-designed animations the way we have.

We had our weekly company meeting in HighFidelity today, just like we always do. For a while now, we’ve all been using Oculus VR headsets and trackers for our hands, and we now have avatars that look like like us. We have good multi-user physics and whole-body tracking-animation for our avatars.

Today one of the guys showed off his recurve bow. As long as you hold it, it re-arms with arrows. Pull the string back with your physical hands, aim and let go, and the arrow flies. It’s basic fun with weaponry. People took turns playing with it.

But quickly people started saying, “Oh, shoot me between the eyes!” (including our CEO). It’s pretty cool to see the arrow coming right for you. Can’t do that IRL! (In Real Life.)

But I never said this. Instead, I stood among the circle of avatars in front of the shooter, and kind of stared him down. Without thinking, I conveyed, “Come at me, bro”, but with nothing so obvious as arms wide or a Neo-like “come on” finger-wave. My avatar could do that with just a stretch of my physical arm or squeeze of my hand, but I didn’t gesture or say a word. Our team is pretty competitive, but we’re not the kind of folks who would shoot the person in front of us without being invited. We need to be invited. But multiple shooters did shoot me. Non-verbal communication works!

Some terrific initial results of our Avatar IK. This is not from post-processing of someone wearing a motion-capture suit with a bunch of ping-pong balls stuck to it.  All the movement is instead driven by a (developer’s edition of a soon-to-be-retailed) gaming VR headset (capturing head position and orientation in 3D), and two (somewhat dodgy) gaming hand-controllers (capturing each hand’s position and orientation, plus a trigger). As described here, we interpret these 3 measurements to trigger an appropriate underlying base animation, which then provides more measurements to drive a realtime inverse-kinematics skeleton.

I went to http://www.paintingvirtualreality.com last weekend.

Billed as the World’s First Virtual Reality Painting Exhibition, it featured:

• artwork one could view individually, using a Head Mounted Display (HMD) with single-camera tracking;
• artists at work wearing HMD with dual lighthouse tracking (and the results displayed live on a big monoscopic screen).

The work was done with http://www.tiltbrush.com, which appears to be a couple of guys who got bought by Google. The project – I’m not sure that its actually available for sale – appears to be evolving along several dimensions:

1. 3d Model Definition: covering stroke capture (including, symmetric stroke duplication), “dry” and “wet” (oil paint-mixing/brushwork effects), texture patterns, volumetric patterns, emmision, particles.
2. Interactive Model Creation: tool pallette in one hand, and brush in the other.
   1.   Videos at tiltbrush.com suggest an additional moveable flat virtual canvas (a “tilt canvas”?) that one can hold and move and paint against. The art on display was clearly made this way, as they all felt like they were a sort of a rotational 2.5D — the brush strokes were thin layers of (sometimes curved) surfaces.
   2. The artists last night appeared to be working directly in 3D, without the tilt canvas.
   3. The site mentions an android app for creation. I don’t know if it is one of these techniques or a third.
3. Viewing: HMD, static snapshots, animated .gifs that oscillate between several rotated viewpoints (like the range of an old-fashioned lenticular display).

  I haven’t seen any “drive around within a 3D scene on your desktop” displays (like standard-desktop/non-HMD versions of High Fidelity).

  The displays were all designed so that you observed from a pretty limited spot. Really more “sitting”/”video game” HMD rather than “standing”/”cave” exploration.

My reactions to the art:

• Emmission and particle effects are fun in the hands of an artist.
• “Fire… Light… It’s so Promethean!”
• With the limited movement during display, mostly the art was “around you” like a sky box, rather than something you wandered around in. In this context, the effect of layering (e.g., a star field) – as if a Russian doll-set of sky boxes (though presumably not implemented that way) – was very appealling.

Tip for using caves: Put down a sculpted rug and go barefoot!

Philip gave a terrific quick demo and future roadmap at MIT Technology Review’s conference last week. See the video at their EmTech Digital site.

Today we put up a progress report (with even more short videos!) about the accomplishments of the past half year. Check it out.

I wonder if we’ll find it useful to have such material in-world. Of course, the material referenced above is intended for people who are not yet participating in our open Alpha, so requiring a download to view is a non-starter.  But what about discussion and milestone-artifacts as we proceed? At High Fidelity we all run a feed that shows us some of what is discussed on thar interwebs, and there are various old-school IRC and other discussions. It’s great to jump on, but it kind of sucks to have an engaging media-rich discussion with someone in realtime via Twitter. Or Facebook. OR ANYTHING ELSE in popular use today.

William Gibson said that Cyberspace is the place where a phone call takes place. I have always viewed virtual worlds as a meta-medium in which people could come together, introduce any other media they want, arrange it, alter it, and discuss it. Like any good museum on a subject, each virtual forum would be a dynamic place not only for individuals to view what others had collected there, but to discuss and share in the viewing. The WWW allows for all of this, but it doesn’t combine it in a way that lets you do it all at once. Years ago I made this video about how we were then using Qwaq/Croquet forums in this way. It worked well for the big enterprises we were selling to, but they weren’t distractible consumers. High Fidelity could be developed to do this, but should we? When virtual worlds are ubiquitous, I’m certain that they’ll be used for this purpose as well as other uses. But I’m not sure whether this capability is powerful enough to be the thing that makes them ubiquitous. Thoughts?

Our Philip Rosedale gave a talk this week at the Silicon Valley Virtual Reality Conference. The talk was on what the Metaverse will be, comprising roughly of the following points. Each point was illustrated with what we have so far in the Alpha version of High Fidelity today. There are couple of bugs, but it’s pretty cool to be able to illustrate the future with your laptop and an ad-hoc network on your cell phone. It’ll blow you away.

The Metaverse subsumes the Web — includes it, but with personal presence and a shared experience.

The Metaverse has user generated content, like the web. Moroever, it’s editable while you’re in it, and persistent. This is a consequence of being a shared experience, unlike the Web.

A successful metaverse is likely to be all open source, and use open content standards.

Different spaces link to each other, like hyperlinks on the Web.

Everyone runs their own servers, with typable names.

The internet now supports low latency, and the Metaverse has low latency audio and matching capture of lip sync, facial expressions, and body movement.

The Metaverse will be huge: huge spaces, with lots of simultaneous, rich, interactive content. The apps are written and shared by the participants in standard, approachable languages like Javascript.

The Metaverse will change education. Online videos have great content, but the Metaverse has the content AND the student AND the teacher, and the students and teachers can actually look at each other. The teacher/student gaze is a crucial aspect of learning.

The Metaverse scales by using participating machines. There are 1000 times more desktops on the Web than there are in all the servers in the cloud.

The talk starts at about 2:42 into the stream: