ACM ISS 2022
Sun 20 - Thu 24 November 2022 Wellington, New Zealand
Tue 22 Nov 2022 11:45 - 12:07 at Rutherford House Lecture Theatre 2 - Session 4: Mobile Chair(s): Ahmed Arif

Speech as a natural and low-burden input modality has great potential to support personal data capture. However, little is known about how people use speech input, together with traditional touch input, to capture different types of data in self-tracking contexts. In this work, we designed and developed NoteWordy, a multimodal self-tracking application integrating touch and speech input, and deployed it in the context of productivity tracking for two weeks (N = 17). Our participants used the two input modalities differently, depending on the data type as well as personal preferences, error tolerance for speech recognition issues, and social surroundings. Additionally, we found speech input reduced participants' diary entry time and enhanced the data richness of the free-form text. Drawing from the findings, we discuss opportunities for supporting efficient personal data capture with multimodal input and implications for improving the user experience with natural language input to capture various self-tracking data.

Tue 22 Nov

Displayed time zone: Auckland, Wellington change

11:00 - 12:30
Session 4: MobilePapers at Rutherford House Lecture Theatre 2
Chair(s): Ahmed Arif University of California, Merced
11:00
22m
Talk
Eliciting User-Defined Touch and Mid-air Gestures for Co-located Mobile Gaming
Papers
Chloe Ng University College London, Nicolai Marquardt University College London
DOI Media Attached
11:22
22m
Talk
Leveraging Smartwatch and Earbuds Gesture Capture to Support Wearable Interaction
Papers
Hanae Rateau University of Waterloo, Edward Lank University of Waterloo; Inria; University of Lille, Zhe Liu Huawei
DOI Media Attached
11:45
22m
Talk
NoteWordy: Investigating Touch and Speech Input on Smartphones for Personal Data CaptureHonourable Mention
Papers
Yuhan Luo City University of Hong Kong, Bongshin Lee Microsoft, Young-Ho Kim NAVER AI Lab, Eun Kyoung Choe University of Maryland
DOI Media Attached
12:07
22m
Talk
TetraForce: A Magnetic-Based Interface Enabling Pressure Force and Shear Force Input Applied to Front and Back of a Smartphone
Papers
Taichi Tsuchida Tohoku University, Kazuyuki Fujita Tohoku University, Kaori Ikematsu Yahoo Japan Corporation, Sayan Sarcar Birmingham City University, Kazuki Takashima Tohoku University, Yoshifumi Kitamura Tohoku University
DOI Media Attached