Illusionary Surface Interfaces, DFG

 

Funded by the DFG (Priority Program SPP 2199)
Duration: 2020-2024

 
Principal Investigators
Prof. Dr.-Ing Katrin Wolf, Berlin University of Applied Sciences and Technology
Prof. Dr. Albrecht Schmidt, LMU Munich

Illusionary Surface Interfaces is a research project within the Scalable Interaction Paradigms for Pervasive Computing Environments (SPP2199). The project aims to investigate how haptic sensations can be induced with less hardware as possible, as future technologies might rely on interfaces where haptic illusions are necessary to provide users with haptic feedback. Traditional computers typically refer (through visual affordances) to both the perceived and actual properties of the interface – suggesting not only fundamental functionalities, but also determining and communicating how humans possibly use the system. Such rich information visualization may, however, not suit the way we want pervasive computers and computational everyday environments to look and feel. We aim to create novel interactive experiences, that exploit multisensory illusions in order to extend the range of interface properties that can be displayed, using only everyday object surfaces as interfaces. In a manner similar to the “rubber hand illusion”, in which people can be induced to perceive a physical touch based purely on what they see, we will support visual and haptic feedback induced by augmented vision and sound. Instead of changing the objects’ physicality, we will visually and auditory augment them using “smart glasses” and projectors, while at the same time augmenting them haptically by inducing multisensory illusion. Technically, this includes sensing user interaction using machine learning tools and multimodal presentation of information.

 

Main Research Question
How can illusionary surface interfaces support creating better and more usable, tangible and touch-based interactions with pervasive computers?

 

Sub-Research Questions
How can we improve interactive surfaces and objects using multisensory illusions in our personal space or a control room?
What are effective and robust multimodal methods to enhance the haptic experience with everyday objects?
How can we comprehensively capture relevant and multimodal parameters that describe the user experience for physical interaction with artifacts?
How does physical interaction on a set of tangible and touch interaction elements impact human perception, experience, and behavior?

Project-related publications

Marco Kurzweg, Yannick Weiss, Marc O. Ernst, Albrecht Schmidt, and Katrin Wolf. 2024. A Survey on Haptic Feedback through Sensory Illusions in Interactive Systems. A Survey on Haptic Feedback through Sensory Illusions in Interactive Systems. In ACM Comput. Surv. ACM, New York, NY, USA, 39 pages.

Marco Kurzweg, Maximilian Letter, and Katrin Wolf. 2023. Vibrollusion: Creating a Vibrotactile Illusion Induced by Audiovisual Touch Feedback. In International Conference on Mobile and Ubiquitous Multimedia (MUM ’23), December 3–6, 2023, Vienna, Austria. ACM, New
York, NY, USA, 21 pages. https://doi.org/10.1145/3626705.3627790

Marco Kurzweg, Simon Linke, Yannick Weiss, Maximilian Letter, Albrecht Schmidt, and Katrin Wolf. 2023. Assignment of a Vibration to a Graphical Object Induced by Resonant Frequency. In Proceesings of INTERACT 2023 is the 19th International Conference promoted by the IFIP Technical Committee 13 on Human–Computer Interaction

Maximilian Letter, Ceenu George, and Katrin Wolf. 2023. A Survey of Computer-Supported Remote Collaboration on Physical Objects. In Proceesings of INTERACT 2023 is the 19th International Conference promoted by the IFIP Technical Committee 13 on Human–Computer Interaction

Maximilian Letter, Marco Kurzweg, and Katrin Wolf. 2023. Comparing Screen-based Version
Control to Augmented Artifact Version Control for Physical Objects. In Proceesings of INTERACT 2023 is the 19th International Conference promoted by the IFIP Technical Committee 13 on Human–Computer Interaction

Marco Kurzweg, Maximilian Letter, and Katrin Wolf. 2023. Increasing Realism of Displayed Vibrating AR Objects through Edge Blurring. In Proceedings of Mensch und Computer 2023 (MuC ’23). Association for Computing Machinery, New York, NY, USA, 16–26. https://doi.org/10.1145/3603555.3603570

Kai Kunze, Pattie Maes, Florian Floyd’ Mueller, and Katrin Wolf (Editors). Cognitive Augmentation. Dagstuhl Reports, 2023, Vol. 12, Issue 12, pp. 1–26, Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany, DOI: 10.4230/DagRep.12.12.1

Marco Kurzweg, Yannick Weiß, Stephen Brewster, Albrecht Schmidt, Katrin Wolf. Visuo-Haptic Interaction. Workshop at the International Conference on Advanced Visual Interfaces (AVI) 2022

Kirill Ragozin, Xiaru Meng, Roshan Lalintha Peiris, Katrin Wolf, George Chernyshov, and Kai Kunze. 2021. ThermoQuest: A Wearable Head Mounted Display to Augment Realities with Thermal Feedback. In Proceedings of the 20th International Conference on Mobile and Ubiquitous Multimedia 2021 (MUM’21)

Florian ‘Floyd’ Mueller, Pedro Lopes, Paul Strohmeier, Wendy Ju, Caitlyn Seim, Martin Weigel, Suranga Nanayakkara, Marianna Obrist, Zhuying Li, Joseph Delfa, Jun Nishida, Elizabeth M. Gerber, Dag Svanaes, Jonathan Grudin, Stefan Greuter, Kai Kunze, Thomas Erickson, Steven Greenspan, Masahiko Inami, Joe Marshall, Harald Reiterer, Katrin Wolf, Jochen Meyer, Thecla Schiphorst, Dakuo Wang, Pattie Maes. Next Steps in Human-Computer Integration. In Proceedings of the Conference on Human Factors in Computing Systems (CHI) 2020

Marco Kurzweg, Jens Reinhardt, Katrin Wolf. PLAY ME! Influencing Game Decisions through Suggestions made by Augmented Characters. In Proceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia (MUM) 2020, ISBN 978-1-4503-8870-2/20/11