Multi Touch Interaction
Multi Touch Input Software - in Java as UI
|
|
Exclusive information on Multi Touch Input. See the Video NowSee Video of Amazing Mult-Touch Interaction Technology from NYU Researchers |
|
Blog, Incredible Video Screenshots Look at screenshots of video demo of NYU researchers and discuss this new technology's capabilities. |
This is from an article written by the folks at UMASS-Amherst Department of Computer Science:
Multiple people can interact using a display surface together
If you ask people of all ages to draw a computer, most will draw a monitor, a keyboard, and a mouse. That is the style of human computer interaction that has been ingrained in our recent culture, where each person is immersed into his/her own computer display, removed from the world outside. But what if you can simply reach out and touch your computer, with bare hands, fingers, stylus and pens, just as you work on a table, a desk and a whiteboard?
Would the experience one has gained throughout a lifetime of interacting with the physical world be transferred to interacting with the digital world? What are the new usage scenarios if you can interact on the same computational surface simultaneously with colleagues? How would that change the nature of computational applications?
This is the focus of Chia Shen's (Ph.D. '92) research Human Computer Interaction on large, direct, multi-touch and multi-user co-interact surfaces. Direct multi-touch surfaces, such as a top-projected DiamondTouch (a research prototype from Mitsubishi Electric Research Labs, Cambridge MA USA, see www.merl.com/projects/DiamondTouch), are displays with superimposed input sensing and output displays; multiple touches can be detected simultaneously. By co-interact, we mean that multiple people interact with and on the same contiguous interactive surface, said Shen, Senior Research Scientist and Associate Director, MERL Research Lab. In this environment, the conventional interface metaphors used in today's predominantly desktop computers are not appropriate. Users should be able to focus on what their real world tasks are, not what the computer interface is doing. Tables predate computers, people have a whole set of preconceptions of how to use and interact with objects on a physical table. This human factor has strong implication on the design of interactive tabletops.
Shen's group faces many interrelated research challenges in studying a new genre of HCI tabletop computing:
DiamondSpin:
Orientation of tabletop documents People most often sit around a table, face-to-face or corner-to-corner. This is very different from the desktop display or electronic whiteboard usage scenario. People cannot interact effectively if we simply display a conventional desktop window system on a tabletop, said Shen. Imagine how one would operate a mouse from a 90 degree angle, or how will users read texts upside down all the time! Towards this end, Shen and her post-doc Dr Frederic Vernier, have developed the DiamondSpin Java tool kit (www.merl.com/projects/diamondspin). A real-time Cartesian to Polar coordinate transformation engine is at the heart of DiamondSpin, enabling flexible rotation and translation of documents displayed on the tabletop. Every user sitting at any side of a table can view documents and objects from a comfortable angle, and documents and digital objects can be naturally passed around the tabletop by a simple finger slide.
UbiTable and DiamondSpace: Content-in Content-out
One of Shen's research foci has been studying tabletops as general purpose display and interaction surfaces, rather than as special purpose computational devices. For example, imagine that tables at Starbucks and airport waiting areas are interactive surfaces. This implies that the digital tabletop must be simple to use for first-time users.
UbiTable (Ubiquitous Table) is a project in Shen's group to address the issue of easy, direct and fluid transfer and movement of documents between personal devices, such as laptops, USB devices, digital camera media, and digital tabletops. They are examining a concept called micro-mobility of digital content supporting a level of control for paper documents, but applied in the digital world. Another key issue in a walk-up environment is privacy the contents on the table should be treated as transient, when a meeting or a work session ends, the users ought to be able to walk-away, knowing that the table has been ‘wiped clean', with no residue of their own personal content for the next group of serendipitous walk-up users to access.
This transient model for files and documents used on the tabletop requires new methods for versioning and journaling of documents that have been brought onto the table dynamically from various personal devices.
DiamondSpace is a new research project that extends UbiTable into a multi-surface walk-up interactive room with digital tables and walls.
ExpressiveTouch: Multi-style, Multi-tool gestural input techniques On a direct multi-touch surface, multi-hand and multi-finger gestures as input methods are feasible. Our hands and fingers have the dexterity to express many levels of interactions, both symmetrically and asymmetrically.But fingers and hands are coarse in resolution when it comes to pointing and interacting with fine, pixel-level data. The mouse, indirect and relative, is one of the most precise pointing input devices. Moreover, people naturally like to use a pen or stylus for writing, scribbling, marking and annotating. Typing with a keyboard is the most efficient way for text input. Thus, ExpressiveTouch is concerned with integrating mutli-modal input devices. Two-handed bimanual operations hold great potential for human computer interaction in general. However, their appropriate design and effectiveness must be studied systematically. The operational relationship, balance and symmetry between the dominant and non-dominant hand are subtle, yet have key implications in the feasibility, acceptance and usefulness of bimanual input techniques.
Large direct multi-touch surfaces will soon be a reality. The key is the understanding of their role in the fabric of our overall interactive world, said Shen. Thus the research question is not how to replicate today's desktop computing on tabletops and electronic walls, but is to ask what are the potentials of these interactive surfaces, what are the appropriate and effective interaction techniques and user interface metaphors for them. In the past few years, together with her colleagues, post doc, and graduate student interns from Stanford, UC Berkeley, University of Toronto, University of Washington, University of Calgary and University of Paris, Shen's research has addressed some of these research questions and raised new challenges in pursuit of her vision of ubiquitous interactive tabletops.
Shen's research on shared tabletop systems has been presented and published in many major HCI forums including ACM CHI, ACM CSCW and UbiComp. She is leading the research at MERL on shared interactive surfaces (http://www.merl.com/people/shen). Before her HCI endeavor, she had over 10 years of research experience in real-time systems. Shen also led the MidART research project which was successfully incorporated into several large distributed industrial plant control systems, contributing to a multimillion dollar industrial control business during 2001-2003. MidART is real-time middleware for applications where humans need to interact, control and monitor instruments and devices in a network environment through computer interfaces. While at UMass Amherst, Shen was advised by Professor Krithi Ramamritham.