NDLI: Modeling multimodal integration patterns and performance in seniors: toward adaptive processing of individual differences

Please wait, while we are loading the content...

A system for fast, full-text entry for small electronic devices

Learning and reasoning about interruption

TorqueBAR: an ungrounded haptic feedback device

Georgia tech gesture toolkit: supporting experiments in gesture recognition

Capturing user tests in a multimodal, multidevice informal prototyping tool

Where is "it"? Event Synchronization in Gaze-Speech Input Systems

Combining speech and haptics for intuitive and efficient navigation through image databases

Demo: a multi-modal training environment for surgeons

Multimodal user interfaces: who's the user?

Mutual disambiguation of 3D multimodal interaction in augmented and virtual reality

Providing the basis for human-robot-interaction: a multi-modal attention system for a mobile robot

Towards tangibility in gameplay: building a tangible affective interface for a computer game

Architecture and implementation of multimodal plug and play

Large vocabulary sign language recognition based on hierarchical decision trees

Eyetracking in cognitive state detection for HCI

Interactive skills using active gaze tracking

Demo: playingfFantasyA with senToy

New techniques for evaluating innovative interfaces with eye tracking

Selective perception policies for guiding sensing and computation in multimodal systems: a comparative analysis

Multimodal biometrics: issues in design and testing

SmartKom: adaptive and flexible multimodal access to multiple applications

Hand motion gestural oscillations and multimodal discourse

A multimodal learning interface for grounding spoken language in sensory perceptions

Error recovery in a blended style eye gaze and speech interface

Crossmodal attention and multisensory integration: implications for multimodal interface design

Toward a theory of organized multimodal integration patterns during human-computer interaction

Sensitivity to haptic-audio asynchrony

A framework for rapid development of multimodal interfaces

Pointing gesture recognition based on 3D-tracking of face, hands and head orientation

A computer-animated tutor for spoken and written language learning

Using an autonomous cube for basic navigation and input

A multi-modal approach for determining speaker location and focus

Untethered gesture acquisition and recognition for a multimodal conversational system

Augmenting user interfaces with adaptive speech commands

GWindows: robust stereo vision for gesture-based control of windows

Distributed and local sensing techniques for face-to-face collaboration

A visually grounded natural language interface for reference to spatial scenes

Perceptual user interfaces using vision-based eye tracking

Sketching informal presentations

Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks

The role of spoken feedback in experiencing multimodal interfaces as human-like

Real time facial expression recognition in video using support vector machines

Modeling multimodal integration patterns and performance in seniors: toward adaptive processing of individual differences

Auditory, graphical and haptic contact cues for a reach, grasp, and place task in an augmented environment

Mouthbrush: drawing and painting by hand and mouth

XISL: a language for describing multimodal interaction scenarios

IRYS: a visualization tool for temporal analysis of multimodal interaction

Towards robust person recognition on handheld devices using face and speaker identification technologies

Algorithms for controlling cooperation between output modalities in 2D embodied conversational agents

Towards an attentive robotic dialog partner

Modeling multimodal integration patterns and performance in seniors: toward adaptive processing of individual differences

Content Provider	ACM Digital Library
Author	Lunsford, Rebecca Wesson, Matt Oviatt, Sharon Coulston, Rachel Xiao, Benfang
Abstract	Multimodal interfaces are designed with a focus on flexibility, although very few currently are capable of adapting to major sources of user, task, or environmental variation. The development of adaptive multimodal processing techniques will require empirical guidance from quantitative modeling on key aspects of individual differences, especially as users engage in different types of tasks in different usage contexts. In the present study, data were collected from fifteen 66- to 86-year-old healthy seniors as they interacted with a map-based flood management system using multimodal speech and pen input. A comprehensive analysis of multimodal integration patterns revealed that seniors were classifiable as either simultaneous or sequential integrators, like children and adults. Seniors also demonstrated early predictability and a high degree of consistency in their dominant integration pattern. However, greater individual differences in multimodal integration generally were evident in this population. Perhaps surprisingly, during sequential constructions seniors' intermodal lags were no longer in average and maximum duration than those of younger adults, although both of these groups had longer maximum lags than children. However, an analysis of seniors' performance did reveal lengthy latencies before initiating a task, and high rates of self talk and task-critical errors while completing spatial tasks. All of these behaviors were magnified as the task difficulty level increased. Results of this research have implications for the design of adaptive processing strategies appropriate for seniors' applications, especially for the development of temporal thresholds used during multimodal fusion. The long-term goal of this research is the design of high-performance multimodal systems that adapt to a full spectrum of diverse users, supporting tailored and robust future systems.
Starting Page	265
Ending Page	272
Page Count	8
File Format	PDF
ISBN	1581136218
DOI	10.1145/958432.958480
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2003-11-05
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Task difficulty Multimodal integration Senior users Speech and pen input Human performance errors Self-regulatory language
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Private speech during multimodal human-computer interaction

Modeling Multimodal Integration Patterns and Performance in Seniors: Toward Adaptive Processing of Individual Differences (2003)

When do we interact multimodally?: cognitive load and multimodal communication patterns

Toward a theory of organized multimodal integration patterns during human-computer interaction

Individual differences in multimodal integration patterns: what are they and why do they exist?

Multimodal system processing in mobile environments

Integration and synchronization of input modes during multimodal human-computer interaction

Toward a Theory of Organized Multimodal Integration Patterns during Human-Computer Interaction (2003)

Mutual disambiguation of recognition errors in a multimodel architecture

Modeling multimodal integration patterns and performance in seniors: toward adaptive processing of individual differences

Similar Documents

Private speech during multimodal human-computer interaction

Modeling Multimodal Integration Patterns and Performance in Seniors: Toward Adaptive Processing of Individual Differences (2003)

When do we interact multimodally?: cognitive load and multimodal communication patterns

Toward a theory of organized multimodal integration patterns during human-computer interaction

Individual differences in multimodal integration patterns: what are they and why do they exist?

Multimodal system processing in mobile environments

Integration and synchronization of input modes during multimodal human-computer interaction

Toward a Theory of Organized Multimodal Integration Patterns during Human-Computer Interaction (2003)

Mutual disambiguation of recognition errors in a multimodel architecture

Modeling multimodal integration patterns and performance in seniors: toward adaptive processing of individual differences