TeleKinect: Enabling Collaborative Interactions in a Virtual Space

Abstract:

TeleKinect is an advanced platform that facilitates collaborative interactions in a virtual space, enabling seamless long-distance collaborations. By utilizing real-time video transport capabilities, TeleKinect empowers users to merge foreground elements from remote locations into a unified virtual environment. In this position article, we present two innovative applications that leverage the TeleKinect framework to foster shared experiences over long distances. WaaZam focuses on enhancing social engagement through creative play, while InReach explores the manipulation of virtual objects and 3D data in a shared digital workspace.

Introduction:

In an era of globalization, individuals, teams, and families often find themselves geographically scattered. Remote collaborations have become an integral part of everyday life, necessitating the development of more effective systems to bridge the distance. While technology has made significant advancements, challenges such as disconnected physical and virtual spaces, limited gesture-based communication tools, and restricted manipulation of shared content still persist. It is crucial to distinguish between two types of remote experiences.

The first type involves crucial decision-making meetings, where the emphasis is on creating a sense of being in the same physical space, facilitating face-to-face interactions and interpersonal connections. This requires realistic representations of remote individuals in terms of size and gaze direction, as well as extending the remote physical space into the local environment. The second type encompasses creative sessions focused on collaborative creation and modification of digital content. Here, the focus is on the data itself and the shared space.

WAAZAM: Empowering Creative Collaboration

As our social circles expand across vast distances, the demand for synchronous creative interactions becomes more prominent. Unfortunately, existing communication platforms do not adequately support co-creative activities over long distances. Particularly in the realms of theatre and dance, where participants must directly coordinate with one another, the potential of creative telepresence systems remains largely unexplored. WaaZam is an innovative telepresence platform that prioritizes creative collaboration within a composited video environment. By incorporating depth analysis, object tracking, and gestural interaction, WaaZam offers users greater creative control during live sessions.

Throughout the years, artists and technologists have developed various strategies to foster engagement with interactive content. Remarkable examples include early artificial reality experiments by Myron Krueger and transformative mirrors by David Rokeby. In the research domain, composited video environments such as Reflexion by Agamanolis, PhotoMirror by Markopoulos, and the HyperMirror project have successfully merged distant spaces on a single screen. Building upon this foundation, our application focuses on identifying the tools and scenarios that best facilitate social engagement through creative play, especially between parents and children. Additionally, it enables collaborative customization of the environment, fostering a sense of togetherness.

INREACH: Bridging the Interpersonal Space and Shared Workspace

We aim to seamlessly integrate the interpersonal space and shared workspace into a unified, cohesive experience. InReach (Figure 3) presents collaborators with a shared virtual space where their live 3D recreated meshes are displayed side by side on a large screen, creating the illusion of an augmented mirror (Figure 4). This mirror allows users to observe themselves and their collaborators within the models and information they are collectively working on, breaking down the virtual barriers that separate collaborators in face-to-face settings. Collaborators can effortlessly point to data or 3D models, interact with digital objects using their bare hands, and manipulate them through translation, scaling, and rotation. We distinguish between one-handed and bimanual actions, showcasing these interactions in contrast to the traditional view offered by remote conferencing.

Figure 3:

Figure 4:

InReach proves particularly valuable in situations where users rely on bodily gestures to indicate and manipulate data while simultaneously visualizing their own presence in relation to that data. Relevant prior work in this field includes ClearBoard, a system that facilitates collaborative drawing on a shared virtual surface by two remote users. The concept of "the office of the future" envisions collaborative manipulation of virtual 3D objects from physical office desks, extending the real office through projected images of a remote office and virtual objects. MirageTable simulates the scenario of two collaborators working together at a table, enabling the creation of virtual replicas of real objects. Digital representations of each user's hands can then interact with these virtual objects within a physical simulation. ARCADE enables remote video-based presentations, allowing the placement and direct manipulation of virtual 3D objects by the remote collaborator.

Discussion and Future Work:

The initial feedback from users of both applications has been promising, motivating us to enhance our hardware precision, improve and evaluate gestural capabilities, simulate more realistic physics interactions, and explore our system in diverse contexts. In future work, WaaZam will present case studies that delve into specific user experiences, elucidating the challenges associated with creative coordination over long distances. Our researchers are actively studying ways to support and encourage storytelling, pretend-play, and improvisation among parents and children in divorced families and families with a traveling parent.

Furthermore, we are conducting a comprehensive user study to assess the effectiveness of customization features and explore the potential of these environments to foster social engagement through creative play and shared activities. As for InReach, our long-term goal is to investigate the integration of our system in industries that heavily rely on 3D models in their design processes, such as architecture, interior design, landscape design, as well as fields where collaboration revolves around bodily activities like theatre and dance. Additionally, we aim to leverage the shared virtual space to facilitate joint presentations involving two or more remote collaborators.

 

Hashtag/Keyword/Labels:

TeleKinect, collaborative interaction, remote collaboration, virtual space, shared experiences, WaaZam, InReach

 

References/Resources:

1. Benko, H., Jota, R., and Wilson, A. MirageTable: freehand interaction on a projected augmented reality tabletop. CHI 2012.

2. Cullinan, C. and Agamanolis, S. Reflexion: a responsive virtual mirror. UIST 2002.

3. Krueger, Myron: Artificial Reality, Addison-Westly, 1991.

4. Hiroshi Ishii, Minoru Kobayashi, and Kazuho Arita. 1994. Iterative design of seamless collaboration media. Commun. ACM 37.

 

For more such Seminar articles click index – Computer Science Seminar Articles list-2023.

[All images are taken from Google Search or respective reference sites.]

 

…till next post, bye-bye and take care.