Inventing the Future: learning the system

Dear Diary,

I’m off to Japan Tuesday for the big conference. Better take a snapshot of what I’ve been doing, because I expect my world to change by the time I get back. My first six weeks on this radical Croquet project were spent with very general learning of what’s what. Drinking knowledge from a firehose. For the next six we’ve been prototyping some of the features from conference papers written by my boss, Julian, and his counterpart at U. Minnesota. We’re going to demo these at the conference, and we go on right after Alan Kay’s keynote address. Yikes. Good thing Julian gives great demos! I imagine the conference organizers know that and put him in that slot accordingly. (Cast of characters here).

[details below the fold]

Finding yourself in Croquet can be disconcerting. The 3D environment is very different than the Windows & Menus interface of the desktop. (Also fathered by one of Kay’s teams back in the stone ages.) Julian wanted to add some familiar menus and windows, and some avatar navigation that would at least be familiar to 3D computer gamers (e.g., arrows and w,a,s,d keys for navigation).

Now, Jasmine has its own system, some of which is at a prototype level, and that’s built on Squeak, which is pretty rich with capability and even some books and wikis of documentation, but not as mature as widely used Enterprise Software. Croquet architect Raab is building a new 2D GUI system called Tweak to sit between these two. I didn’t know any of them. The documentation isn’t just in the code (e.g., interspersed as comments) – it is the code (no comments). On the user end of the problem, I didn’t expect that even Julian would create “the” perfect user interface for all circumstances right out of the gate. So instead of setting out to create a proper and extensible framework on which “the” user interface would be delivered, I merely hacked together a functional demo that illustrated the possibilities from the papers. The code sucks, as does the behavior if you push on it just a bit. I figure we’ll go through some iterations of what kinds of behaviors we need to support. Then we’ll know better how to build a framework that allows these behaviors to be separately specified by different people, exchanged and composed and decomposed for use in different situations. It will insulate people from the different event models underneath, and also let users specify behavior from within the Croquet environment itself. But now it’s a demo.

Our Jack Keel built most of the menus, which are in Tweak. Amazing, because there’s no documentation, and because Tweak has just the kind of framework I’ve been describing, which, alas, makes it very difficult to figure out from the code how to do anything. It’s so indirect. Jack’s a pro.

We had some code from Yoshiki Ohshima to demo some neat abilities to add annotations to items in the scene, and to other annotations, and filter them with a flip-away visor. But it was for an earlier version of Croquet. Not knowing what would be involved, we had our Ron Stewart port this code to Jasmine. We figured the old demo would be our fallback baseline, and we could add in new capability, such as storing the annotations in a searchable world-accessible server. It turned out to be a huge task, mostly because of instability in the different layers of event behavior frameworks and also immaturity in portable file system handling in Squeak. (See my no names, please! entry.) Anyway, he got it working and it broke as soon as we tried to combine it with Tweak. We figured that worst case, we could run them separately, from the same Squeak environment. That broke when folks at Minnesota started hooking up their worldbase, and they decided to rebuild from scratch. I would have consoled myself that Ron learned a lot in the process – but he’s taking a new job across the street, so it’s all for naught. Ron is a PhD in biology, and the famous biologist Jamie Thompson tapped Ron personally to run the bioinformatics database for his hot new lab on campus. Rats. Ron is a first class guy, both personally and technically, and I’m going to miss him – even after just three months. Ron also created the code that lets us load 3D mesh content created by the (relatively) ubiquitous 3D Studio Max modeler.

We contracted with a talented local tech shop, Clotho, to build a video screen above an avatar’s head, which displays the live Webcam video of the user associated with that avatar. Clotho founder Preston Austin and his student Seth Meyer did the work. They also had some earlier code from Yoshiki-san to work from.

I wrote the lousy keyboard navigation, some of the menus stuff, second-seated for Jack, Ron, and Preston, and integrated everything together with the content that Julian created. I hovered, but with my face and fingers in the code.

And I’ve been trying and mostly failing to coordinate with Minnesota. I think maybe Julian’s counterpart there, Mark, is in some personality aspects too much like me. We want to do it right, we want to do things without waste, we both impatiently take initiative, and we don’t know exactly where we’re going. Bricolage is very big among the folks in this project. Here’s a story. We were all building different top-level programs (called “morphs”. Don’t ask.) for running the various pieces we were working on. I unilaterally tried creating one combined thing in which the demo would be run. When Yoshiki-san’s annotation code failed in combination with the Tweak menus, I split them out again. Without knowing what the heck crazy thing I was doing, the Minnesota team went back to individual morphs. After they broke the annotation code, I went back to putting everything into a single morph. Whenever someone added capability to some private morph, I would make the same changes to the main demo morph. During a couple of important days of rapid progress, I was spending all my time copying over other people’s code. Mark recognized this and started using this demo morph. But I had in it a “particle fountain” that constantly emits sparks. It’s highly visible, and allows me to tell instantly when something has caused the system to fail, because the sparks stop coming. It’s quite ugly, and someone at Minnesota yanked it out as they were making other changes in the same code. I put it back, and they yanked it out again. I put it back again with a comment that it was useful for telling when you’re hosed. It stayed. Then I added a sky with moving clouds, which are even more visible, but far more tasteful than the particle fountain. Within a few hours, someone at Minnesota had removed the fountain. To me, this story illustrates that our colleagues really know what they’re doing, are attentive to the code and the situation, really understand the issues, and take initiative. What a pain. And it shows we’re communicating better through the code then we are by video conference, email, and text chat. A strong goal that many of us have for Croquet is to allow Croquet functionality to be built from within Croquet itself. With video avatars and Voice-Over-IP, we will all directly see who is working on what, and be able to easily discuss things with them while they do it, just as if we were all working in the same cubicle or bull-pen.

Another issue is that it is hard to add functionality in isolation. Squeak is Smalltalk, which has single inheritance. You can’t just say that some object combines this one (that I work on) and that one (that you work on). If you’ve got behavior that you want to be in two places you’ve to put it in a common base. If not everything that uses that base should share the new code, you’ve got to either refactor or copy the code around. And as written in Jasmine, some of the code does a lot. When you want something to do something a little different in some situations, you’ve got to reach into the existing code and add the new stuff. You can’t just add your new stuff separately. This is very useful for skilled cowboy programmers working alone on objects that are very separate from the objects that other people are working on. You can see what the code is doing, and feel the bits between your toes. But this is not ideal in collaborative projects of the sort we want to build. The solution is to build frameworks for particular aspects (like user event handling).

About Stearns

Howard Stearns works at High Fidelity, Inc., creating the metaverse. Mr. Stearns has a quarter century experience in systems engineering, applications consulting, and management of advanced software technologies. He was the technical lead of University of Wisconsin's Croquet project, an ambitious project convened by computing pioneer Alan Kay to transform collaboration through 3D graphics and real-time, persistent shared spaces. The CAD integration products Mr. Stearns created for expert system pioneer ICAD set the market standard through IPO and acquisition by Oracle. The embedded systems he wrote helped transform the industrial diamond market. In the early 2000s, Mr. Stearns was named Technology Strategist for Curl, the only startup founded by WWW pioneer Tim Berners-Lee. An expert on programming languages and operating systems, Mr. Stearns created the Eclipse commercial Common Lisp programming implementation. Mr. Stearns has two degrees from M.I.T., and has directed family businesses in early childhood education and publishing.


  1. Looks like an interesting conference. Obviously it’s very Croquet-heavy. Looks like you guys really are pretty much the dominant story in this subject. How many people do you expect will be attending?

    Interesting bit about your collaboration process.

    About single inheritance: yes, I can see how it’s limiting devlopment, especialy in this distributed-team environment. As long as your project is still in flux and you’re changing all kinds of paramenters, would it be too forward of me to suggest changing to a more flexible programming substrate

  2. I’m not entirely sure what to expect for the conference. This is the third year. I understand that last year had quite a few papers on Croquet, but most were not. This year about half.

    You raise an interesting question: Why Squeak? Ultimately, I imagine the answer is, “because that’s what Alan and Andreas were used to.” Squeak is Allen’s baby, and Andreas knows it inside and out. By contrast, the two Davids are old Lisp guys, and Mark and Julian were using other 3D environments before they joined Croquet.

    Another non-technical issue is license. Croquet has an “MIT-style” license, which means anyone can do anything with it, including giving it away, making money, redistributing, or whatever. Squeak allows us to do that. Curl would not.

    Another issue is portability. Squeak is mostly written in Squeak, with a very tiny and easilly ported byte engine. Squeak has been run on anything, including PDA’s (significantly, HP PDAs – not, as far as I know, Palms), and even embedded chips. Remember that HP is puttting a lot of effort into Croquet, and HP makes hardware.

    Although it is not nearly important as the above, it’s hard to beat Squeak for raw flexibility of the language. There are Python and, I think, Javascript engines being developed for Squeak. There’s an eToys scripting environment aimed at kids, which is really more of a prototype-style object-oriented language rather than a class-based one. It isn’t hard to build your own multiple inheritance on top of Squeak, using delegation. (If a method isn’t handled in the normal single-inheritance chain, go through a linear chain of delegate clases.) You can do all this because the compiler is accessible, open, and written in Squeak itself. (The Curl compiler was not as open. Don’t know about now.) The problem is that the current version of Croquet wasn’t designed to make use of this — it’s programmed “directly in” Squeak, rather than in its own language “on” Squeak. In fact, the current version takes over the “unhandled message” mechanism (that I would use for implementing multiple inheritance) and uses it for its own purposes. (It’s a cute hack that makes it easy to indicate that a message should be replicated among all the hosts in the collaboration.) The next version of Croquet might free this up again, but by that time we’ll have so much written “in” Squeak that no one will want to rewrite into a separate “Croquet” language that no one else uses. On the other hand, we will want to allow scripting of behaviors from within Croquet itself, and that may involve an eToys-like language. We’ll see.

  3. I was (mostly) joking about Curl. Obviously I think it has some nifty capabilities that would lend itself nicely to your project. However I think it has safely niched itself into a corner, and as a Laszlo guy (now) I suppose I’m supposed to hope that it stays there.

Comments are closed