I came across this issue a couple of times and I decided to make a blog-post reminder for my personal glossary.

The “Mida’s Touch” problem refers to the fact that the eyes cannot be used directly as a mouse, because the eyes are never “off.” Thus one of the main problems when using the eye-gaze for selection purpose is to somehow combine it with a “clutch” that can engage/disengage eye-gaze control. A good clutch should be quick to operate, not increase the cognitive load unnecessarily and not disturb the user’s gaze-pattern, because the user will often be looking at some object when she wants to engage eye-tracking, and it would thus slow down the communication if she had to move her eyes to do it.

[More]

Pour l’enfant, amoureux de cartes et d’estampes,

L’univers est égal à son vaste appétit.

–Charles Baudelaire

Bly, S. A., and Minneman, S. L. Commune: a shared drawing surface. In Proceedings of the ACM SIGOIS and IEEE CS TC-OA conference on Office information systems (New York, NY, USA, 1990), ACM, pp. 184–192. [pdf]

————

The starting assumption of this work was that in group design situations, the activity of creating and using the marks on the drawing surface is as important as the marks themselves. During observations of design meetings, the authors identified three kinds of actions: draw, write and gesture. These were combined in different ways to support different uses: providing illustrations, or emphasis on a particular part of the talk, provide visual reminders, making references or other uses.

To support these mechanisms at a distance the authors designed a system called Commune where users could superimpose sketches to a common drawing surface while talking. They conducted a tight observation of use of the system by several pairs and they found that neither the talk nor the marks alone effectively communicate the issue under consideration.

One of the advantage that they saw in the use of Commune was the ability to be in the same place at the same time. While this is possible with the virtual superimosition of the marks that is not actually possible in real situations and the authors speculated that this could have been an advantage.

However, they also identified three problems: the stilii were difficult to use with sketches consisting of short line segments; second the drawing space was small and finally Commune constrained gestures to pointing actions.

Bekker, M. M., Olson, J. S., and Olson, G. M. Analysis of gestures in face-to-face design teams provides guidance for how to use groupware in design. In DIS ’95: Proceedings of the 1st conference on Designing interactive systems (Ann Arbor, Michigan, United States, 1995), ACM Press, New York, NY, USA, pp. 157–166. [pdf]

———-

This paper presents some findings from ethnographic observations of collaborators using gestures in face-to-face meetings. This work summarize previous research on the field like the work of Tang that found that 35% of hand movements in a group meeting are gestures. They developed a coding scheme for gestures and they defined the following top-level categories: Kinetic (the movements reproduces an action performance), Spatial (the movement indicates distance or location or size), Point (fingers point to some person, to some object or place), and Other.

The authors found that pointing was an highly frequent gesture that was used in the majority of situation in their task. The paper also reports important critics to telepointers and to the ClearBoard system such as the fact that they are weak gesturing devices as they do not have the rich dynamics of human hands, arms, or body.

A friend sent me an interesting link on a recent development of the European Air-Traffic control. Apparently voice channels currently used to dispatch communications are becoming increasingly insufficient to handle the mass of airplanes circulating at any given time. An experimental solution is called CONTROLLER PILOT DATA LINK COMMUNICATION.

This system is based on sending the instruction to pilot via interface, the controller click with mouse on flight level 250, and this is send to the cockpit, then the pilot see a flashing light, he acknowledges that he receive and send back massage. The operator can mark some kind of status message that will be meaningful for the second controller (en route sector is operated by executive and planner controller) the controllers can shift responsibility for the aircraft between themselves via interface input.

The interface design is reported below, while more information can be found using this link. The reasons why I found this interesting is that simple text communications will be used to coordinate actions between pilots and ground controls in very complex situations. This renew the importance of ‘minimal’ communication forms like chat in supporting collaborative work at a distance. My thesis is not wasted 🙂

Robertson, T. Cooperative work and lived cognition: a taxonomy of embodied actions. In ECSCW’97: Proceedings of the fifth conference on European Conference on Computer-Supported Cooperative Work (Norwell, MA, USA, 1997), Kluwer Academic Publishers, pp. 205–220. [pdf]

————-

This paper presents a taxonomy for embodied actions developed through ethnographic observations. The authors recogize the importance of  the body as the essential basis of all human action and interaction. The defining constraint of technology which aims to support collaborative work at a distance must be the essential corporeality of human cognition.

Embodied actions are defined as classes of cognitive practices that are pubblically and simultaneously available to the perception of the actor and ohers in a shared physical space.

The author defines a specific category for embodied actions in relation to the workplace where we can find pointing at something and shifting direction of gaze.

Precisely these all the defined categories function as a communicative actions in shared physical space because physical space enables the reversibility of perception. This is not a given fact in virtual space.

Fussell, S. R., Setlock, L. D., Yang, J., Ou, J., Mauer, E., and Kramer, A. D. I. Gestures over video streams to support remote collaboration on physical tasks. Human-Computer Interaction 19, 3 (2004), 273–309. [pdf]

————-

This paper summarizes two important study of the Fussell et al. on gestures. The authors recognize the importance of gestures for remote collaboration and distinguish between pointing gestures and representational gestures. This second group can be further divided in iconic representation, spatial gestures and kinetic gestures.

The article starts by critiquing previous approaches to enable remote gestures, sometimes oversimplyfing conclusions such as that systems like ClearBoard are too expensive and therefore impractical for most collaborative work. Also, the author choose to adopt surrogate approach to remote gestures, expressing the communicative intents through alternative means. The reason of this choice over a more direct view of the gesture are not discussed.

In a first study, the authors compared a face-to-face interaction versus a remote collaboration where the helper could indicate parts of a model using a telepointer mechanism. Contrary to the hypothesis they found that adding the pointer was not sufficient to improve performance over that of the video-only condition. They discussed possible explaination like the fact that the cursor tool was too limited in functionality. Also they hypothetized that the part of the task requiring effective pointing represented only a small percent of the total and therefore was not influent.

In the second experiment they evaluated the DOVE system. The compared video-only against the DOVE setup with manual erasure and with automatic erasure. They found that performance was best for this third condition. They also found that pointing gestures were mostly prevalent.

Kato, H., Yamazaki, K., Suzuki, H., Kuzuoka, H., Miki, H., and Yamazaki, A. Designing a video-mediated collaboration system based on a body metaphor. In Proceedings of Computer Supported Collaborative Learning conference (CSCL’97) (Toronto, CA, December 10-14 1997), pp. 142–149. [pdf]

———

This paper reports some interesting considerations and findings on the arrangements of video in remote teaching. The authors argue that the arrangements of the view should respect the face-to-face metaphor in order to be effective. Particularly, the authors argue that such systems should:

(1) the learners should be able to see the instructor’s pointer; (2) the instructor should be able to see that the learners are orienting themselves towards the pointed object as well as the pointer, when they are observing the instructor’s pointing; (3) the instructor should be able to reassure the learners by words or actions that the instructor is aware of the learners’ orientation; (4) the learners should be able to see the face of the instructor, when they want to know how the instructor is evaluating their behavior or when they want to draw the instructor’s attention; and (5) the instructor should be able to notice the learners’ orientation toward the instructor himself/herself as well.

A qualitative analysis of the technology revealed that problems are still presents as this solution does not solve completely the limits of video-mediated communication. Secondly, the representations might still be corrupted by the medium and finally, too many resources might be difficult to handle by the users.

Daly-Jones, O., Monk, A., and Watts, L. Some advantages of video conferencing over high-quality audio conferencing: fluency and awareness of attentional focus. Int. J. Hum.-Comput. Stud. 49, 1 (1998), 21–58. [pdf]

————–

This article does a great job in summarizing the work that at that time was relevant for video-mediated communication. The authors list four pragmatic needs that must be fulfilled through the transmission of auditory or visual information in human interaction: making contact; allocating turns; monitor understanding and attention and finally supporting deixis. The authors list useful resources for each of these needs.

The article describes how deixis is interrelated with eye-gaze and the value of visual data for collaborative work. The the article resumes the results in studies showing the impact of different video-communication technologies on distance collaboration summarizing that video mediation does not deliver the same benefits of face-to-face interaction.

The study reports an experimental setup where the authors compared two experimental conditions full audio and full video interaction. They conclude that the utility of a certain media should be considered with regard to the task.

Audio information is a sufficiently rich resource when it comes to coordinating fluent conversations. However the lack of video was found problematic for small groups resulting in less fluent communication.

Gaver, W., Sellen, A. J., Heath, C., and Luff, P. One is not enough: Multiple views in a media space. In Proceedings of INTERCHI’93 (Amsterdam, The Nederlands, April 24-29 1993), ACM Press, pp. 335–341. [pdf]

———-

The paper presents an interesting approach of using multiple cameras in a media space. The conducted ethnographical observations to understand how people switchend and why among multiple cameras. Their system allowed to focus the view on the shared workspace, looking at the settings of the remote collaborator’s office and the like. However, these view resulted of a minimal importance to the task. Even the face-to-face view was not used intensively.

Additionally, the authors found two big limitations of the system that impeded an efficient use towards a furniture arrangement task. First, a collaborator could not point on objects of the shared workspace. Second, a collaborator could not attract the attention (e.g., the view) of the other collaborator.