Surfnet logo

Add-Ons for Collaborative Surfaces

This is an ongoing project.

In work on applications to faciltate collaborate around large interactive surfaces, we identified recurring needs for common software capabilities. Rather than build these into the applications themselves, we decided to support them as add-ons: small programs that connect to other more domain-specific programs. Our favoured development platform for collaborative surface applications is HTML5 running in web browsers, and this model fits especially well on that platform.

 

RA: The Remembrall Strata Gesture-Intent

 

We have three add-ons in development:

RA: the Remembrall (Peter Simonyi)

In our work on collaborative large surface applications, we found that much interaction was exploratory. While people discussed the situation, they asked the software “what if”? They used the software, and the data and mathematical models underlying it, to show what resulted from various speculations. This was commonplace, and reminded us of the kind of interaction described by Marian Dork et al. in their paper about the Information Flaneur, someone who strolls through data: http://mariandoerk.de/informationflaneur/

As we watched, we realized that this kind of interaction suggested a need for specific support. This is because once something had been explored, the users would continue exploring and later potentially return to an earlier state. But how? This needs a kind of super undo, whereby earlier states are remembered for later easy retrieval. This is the service provided by RA, a “Remembrall” that tracks state transitions, and allows return.

Strata: an Annotation Layer (Miran Mirza)

Another kind of behaviour we observed in large-screen collaborative applications was that people wanted to “draw” on the surface. Although their web buttons and graphs and numbers, people wanted to circle things, add arrows, draw lines connecting elements. It was tempting to use marked on the large screens; instead they drew pictures in notebooks. Our idea was that the ideal support would resemble a “telestrator”, the kind of device that sports broadcasters use to illustrate replays to help the audience better understand elements of they play. Our software, Strata, allows this: it is an add-on to web applications that allows users to “draw” on top of the app. It allows multiple touch, freehand drawing, and shapes, on a level (strata) above the app. It allows clearing, saving, and capture of whole images to be recalled later.

Gesture-Intent (Ravina Samaroo)

Lastly, in our observations of people collaborating, we observed many gestures in relation to the screen. People pointed, with one finger  or two, one person or two, and drew imaginary lines, all to help thinking and working together. Our idea was that certain gestures could be supported by software, either by Ra or Strata (above), or by the applications themselves. But the first step was to study what gestures people used, and what their intentions were. If there is a common pattern, then the approach will be feasible.