News & Updates
June 21, 2017
CHI'15 paper cited in MIT master thesis
My CHI'15 paper Deformable Interfaces for Performing Music was cited in a master thesis from MIT by Irmandy Wicaksono. The title of the thesis is Design and Implementation of Multi-sensory Fabric as Deformable Musical Interface, and it can be downloaded at the link https://www.researchgate.net/publication/317639059_Design_and_Implementation_of_Multi-sensory_Fabric_as_Deformable_Musical_Interface
May 7, 2017
Most read conference article
With 51 reads my article SCI-FI: Shape-Changing Interfaces, Future Interactions published at NordiCHI 2016, was the most read article from all HCI publications at my former department DIKU at the University of Copenhagen.
January 23, 2017
Postdoc at UCSD (bioCaddie)
I have started working as a Postdoc researchers at the university of California, San Diego as part of a US government project funded by NIH on information retrieval for biomedical researchers. The name of the project is bioCaddie (https://biocaddie.org/) and my direct supervisor will be Dr.Jina Huh-Yoo (http://jinahuh.net/). As part of the bioCaddie team, my responsibility will be to do user evaluation of a data retrieval interface for biomedical researchers called DataMed (https://datamed.org/). DataMed is a big repository and dataset aggregator, especially targeted towards biomedical researchers and is the "dataset collector" version of PubMed (which is focused on aggregating pubblications in the medical field). As part of the user evaluation team I will be working on user experience (UX) tests, conduct interviews, analyzing users' workflows, performing heuristic evaluations, and organizing workshops following participatory design methods.
October 10, 2016
PhD defense: After Rigid Interfaces
After 3 years of work I finally defended my PhD thesis at the University of Copenhagen in front of the evaluation committee composed by Prof. Mikkel R. Jakobsen, Prof.Harald Reiterer, and Ass.Prof. Audrey Girouard. Below the abstract of the thesis:
Deformable and shape-changing interfaces are rapidly emerging in the field of human-computer interaction (HCI). Deformable interfaces provide users with newer input possibilities such as bending, squeezing, or stretching, which were impossible to achieve with rigid interfaces. Shape-changing interfaces can reconfigure their shape dynamically, providing users with new affordances and output modalities.
This thesis contributes to both the field of deformable interfaces and shape-changing interfaces through empirical research. In the area of deformable interfaces, this thesis presents two studies (1) a user study with a prototype of an elastic, deformable display, and (2) a user study of deformable interfaces for performing music. The first study reports a guessability study with an elastic, deformable display where 17 participants suggested fitting gestures for 29 tasks, including navigation and manipulation of 3D graphical objects. Results from the first study describe a user-defined gestures set for elastic, deformable displays, showing how participants used depth and elasticity of the display to simulate various deformations, rotations, and displacements. The second study investigates how musicians use deformable interfaces to perform electronic music. First, we invited musicians with different backgrounds (e.g., performers, DJs, instrument builders) to three workshops, where we made them explore 10 deformable objects and generate ideas on how to use those to perform music. Then, we implemented sensors in the five preferred objects and programmed them for controlling sounds with computer software. Finally, we ran a performance study where six musicians performed music with deformable interfaces at their studios. Results from the performance study show that musicians systematically map deformations to certain musical parameters and that deformable interfaces are generally used as tools to filter and modulate sounds.
In the area of shape-changing interfaces, this thesis presents two work (1) an analysis of sketches made by 21 participants designing either shape-changing radios and mobile phones, and (2) a large-scale analysis of 340 science fiction (Sci-Fi) movies that analyses behavioral qualities of shape change, and how they support particular functionalities of shape-changing interfaces. The first work presents an analysis of 42 sketches of shape-changing interfaces, specifically radio and mobile phone. The result of this analysis shows a range of interesting design elements, but also a lack of conventions on the use of metaphors with shape change and the need to extend present vocabulary. Also, the analysis shows how metaphors and dynamic affordances in shape change can be used to convey particular information (e.g., big-is- urgent, loud-is-up). The second work presents a large-scale analysis of 340 Sci-Fi movies that identifies instances of shape-changing interfaces. Results from the analysis reveals emergent behavioral patterns of shape change, namely Reconfiguration, Transformation, Adaptation and Physicalization.
In synthesis, the work presented in this thesis shows (1) implications of usefulness for deformable interfaces and how their new input modalities can redefine the way users interact with computers, and (2) how a systematic understanding of conventional design elements and behavioral qualities of shape change can help the design of shape-changing interfaces in the future.
June 17, 2016
SCI-FI: Shape-Changing Interfaces, Future Interactions
Our paper "SCI-FI: Shape-Changing Interfaces, Future Interactions" was accepted at NordiCHI 2016. The long paper is the result of a collaboration between the University of Copenhagen and the KAIST University of Daejeon (South Korea). Below the abstract of the paper:
Shape-changing interfaces (SCI) are rapidly evolving and creating new interaction paradigms in human-computer interaction (HCI). However, empirical research in SCI is still bound to present technological limitations, and existing prototypes can only show a limited number of potential applications for shape change. In this paper we attempt to broaden the pool of examples of what shape change may be good for by investigating SCI using Science Fiction (Sci-Fi) movies. We look at 340 Sci-Fi movies to identify instances of SCI and analyze their behavioral patterns and the context in which they are used. The result of our analysis presents four emerging behavioral patterns of shape change: (1) Reconfiguration, (2) Transformation, (3) Adaptation, and (4) Physicalization. We report a selection of SCI instances from Sci-Fi movies, which show how these behavioral patterns model functionalities of shape change and what they can do. Finally, we conclude by providing a discussion on how our results can inspire the design of SCI.
User-defined gestures for elastic, deformable displays
AVI 2014 - BEST PAPER AWARD
Elastic, deformable displays allow users to give input by pinching, pushing, folding, and twisting the display. However, little is known about what gestures users prefer or how they will use elasticity and deformability as input. We report a guessability study where 17 participants performed gestures to solve 29 tasks, including selection, navigation, and 3D modeling. Based on the resulting 493 gestures, we describe a user-defined gesture set for elastic, deformable displays. We show how participants used depth and elasticity of the display to simulate deformation, rotation, and displacement of objects. In addition, we show how the use of desktop computers as well as multi-touch interaction affected users’ choice of gestures. Finally, we discuss some unique uses of elasticity and deformability in gestures.
deformable interfaces for performing music
CHI 2015 - Music & Art
Deformable interfaces offer new possibilities for gestures, some of which have been shown effective in controlled laboratory studies. Little work, however, has attempted to match deformable interfaces to a demanding domain and evaluate them out of the lab. We investigate how musicians use deformable interfaces to perform electronic music. We invited musicians to three workshops, where they explored 10 deformable objects and generated ideas on how to use these objects to perform music. Based on the results from the workshops, we implemented sensors in the five preferred objects and programmed them for controlling sounds. Next, we ran a performance study where six musicians performed music with these objects at their studios. Our results show that (1) musicians systematically map deformations to certain musical parameters, (2) musicians use deformable interfaces especially to filter and modulate sounds, and (3) musicians think that deformable interfaces embody the parameters that they control. We discuss what these results mean to research in deformable interfaces.
Sketching Shape-changing Interfaces: Exploring Vocabulary, Metaphor Use, and Affordances
CHI 2016 - Embodied Interaction
Shape-changing interfaces allow designers to create user interfaces that physically change shape. However, presently, we lack studies of how such interfaces are designed, as well as what high-level strategies, such as metaphors and affordances, designers use. This paper presents an analysis of sketches made by 21 participants designing either a shape-changing radio or a shape- changing mobile phone. The results exhibit a range of interesting design elements, and the analysis points to a need to further develop or revise existing vocabularies for sketching and analyzing movement. The sketches show a prevalent use of metaphors, say, for communicating volume though big-is-on and small-is-off, as well as a lack of conventions. Furthermore, the affordances used were curiously asymmetrical compared to those offered by non- shape-changing interfaces. We conclude by offering implications on how our results can influence future research on shape-changing interfaces.
SCI-FI: SHAPE-CHANGING INTERFACES, FUTURE INTERACTIONS
NORDICHI 2016 - EXPLORING NEW INTERFACE DOMAINS
Shape-changing interfaces (SCI) are rapidly evolving and creating new interaction paradigms in human-computer interaction (HCI). However, empirical research in SCI is still bound to present technological limitations and existing prototypes can only show a limited number of potential applications for shape change. In this paper we attempt to broaden the pool of examples of what shape change may be good for by investigating SCI using Science Fiction (Sci-Fi) movies. We look at 340 Sci-Fi movies to identify instances of SCI and analyze their behavioral patterns and the context in which they are used. The result of our analysis presents four emerging behavioral patterns of shape change: (1) Reconfiguration, (2) Transformation, (3) Adaptation, and (4) Physicalization. We report a selection of SCI instances from Sci-Fi movies, which show how these behavioral patterns model functionalities of shape change and what they can do. Finally, we conclude by providing a discussion on how our results can inspire the design of SCI.
Technical Portfolio (Selected work)
This section of my website describes the technical work I have realised both in the academic context as a Bachelor, Master, and PhD student, and outside the academic context for personal projects and collaborations. The selected work are sorted chronologically from the most recent to the earliest.
MUDE: a multi-dimensional input deformable interface (2016)
MUDE is a deformable interface that is capable of sensing multiple deformations simultaneously. I have realised the version 1.0 of the MUDE prototype in collaboration with LEAP, a Copenhagen based company that produces electro-active polymers (EAPs). MUDE v1.0 features a soft silicone casing that is roughly the form factor of an iPhone 6 (i.e., 1cm x 8cm x 12cm). The silicone casing is embedded with three electro-active polymer (EAP) sensors placed at different heights inside its surface. The sensors are capable of sensing four distinct types of deformation when input is given, namely bend, press, stretch, and twist.
To classify the different deformations I used machine-learning techniques such as support vector machines (SVM). MUDE will be used in the future to carry out two user studies: (1) a study that borrows methods from psychophysics, in order to test users’ perception of various magnitudes of deformations, and (2) a study that tests users’ ability to control multiple inputs (i.e., deformations) simultaneously. This investigation will contribute the following to the filed of shape change: (1) empirical results on users’ perception of multi-dimensional input with deformable interfaces and (2) a series of interaction techniques and applications for multi-dimensional deformable input. These contributions will be oriented towards technology development, researchers, and designers, so as to provide a better understanding of the benefits and drawbacks of multiple inputs with deformable interfaces.
Deformable interfaces for music performances (2015)
For this CHI project I have build functional prototypes of deformable interfaces that were intended to be used as controllers for musical performances. The five interfaces were built upon ideas generated by nine musicians during three workshops (see the related paper for more details). The five interfaces were made of different deformable and malleable materials, embedded with various sensors (e.g., FSR, Flex Sensor 2.2", Rotary Pot, Conductive Rubber Chord).
The interfaces were programmed to work as MIDI controllers using an Arduino MEGA to interface and read the sensors' data with the computer, and using Pure Data (PD) to create a bridge to Ableton Live in order to allow musicians to easily customise and map sounds to the deformable interfaces. The musicians involved in the performance study highlighted how the deformable interfaces enhanced creative thinking during music performances and compositions for their intuitiveness and embodied relationship to sounds.
I+C+i. A revolution of education? (2012)
Before starting my PhD in Copenhagen in 2013 I have collaborated intensely with Prof.Sergi Jordà and the Tangible Interaction Group at the Universitat Pompeu Fabra (UPF, Barcelona), especially on the topic of technology-enhanced creative practices. The 21st of March 2012 together with Sergi and his group I run a workshop on rapid prototyping and creativity at the CCCB called "A revolution of education"; the workshop was mainly targeted towards teachers and scholars to understand how technology could facilitate and support individual as well as collective creative practices.
At the workshop we have used a technology that we call Arduino HID Shield. The Arduino HID shield transforms a simple Arduino board into a keyboard that can be used to give input to a computer. However, with the Arduino HID Shield any conductive material can be used in place of regular keys, and therefore allowing users to create customised interactive input interfaces. I have helped Daniel Gallardo and Carles F.Julià in the programming of the interface and the group in the organisation of the CCCB workshop. At the workshop we have invited teachers and scholars who had no previous technical knowledge either in programming or in electronics. Using the Arduino HID Shield both teachers and scholars were able to create customised, interactive input interfaces in roughly 20 minutes, such as game controllers, musical instruments, etc.
Insert Scorie (MA Thesis, 2010)
In 2010 I got my MA in New Technologies of Art at the Academy of Fine Arts of Naples. As a final work I presented an interactive sound installation called Insert Scorie. The installation consisted into nine semi-transparent tabletops mounted together to form a semi-circular table, where little toys could be placed and moved to generate interactive musical performances. The little toys used as input interfaces were tagged at the base with ReacTIVision fiducials, which could be tracked with the use of a PS3-Eye camera placed at the bottom of the tabletop. To create the interactive sound installation I have programmed an application using Processing to handle the input coming from ReacTIVision, and used Pure Data to create the sound engine that would let users generate music by simply moving the toys on the surface of the tabletop.
Furthermore, the toys were hand-crafted by primary and secondary school children during a workshop that we held at the academy of fine arts; the workshop wanted to teach children how to use discarded material (e.g., rubbish), and recycle it to create brand new toys. To create the installation I have coordinated a group of up to 40 students, all of which contributed to the building of the tabletop and the organisation of the workshop involving children. Insert Scorie was awarded the National Prize of Arts in 2010, and showed at the Ars Electronica in Linz in 2011 upon request of the media theorist Derrick de Kerckhove, who greatly appreciated the work and the philosophy behind it.
SCI-FI:shape-changing interfaces, future interactions
Published at NordiCHI 2016 (Göteborg, Sweden)
DEformable INterfaces for performing music
Published at CHI 2015 (Seoul, South Korea)