Pers Ubiquit Comput DOI 10.1007/s00779-013-0683-x

EDITORIAL

PUC theme issue: material interactions Mikael Wiberg • Jofish Kaye • Peter Thomas

Ó Springer-Verlag London 2013

With the advent of smart materials, ubiquitous computing, computational composites, interactive architectures, the Internet of Things, and tangible bits, HCI has increasingly recognized the role that non-computational materials play. As IT gets embedded in everyday objects, vehicles, and buildings, interaction design becomes not only a matter of designing interaction with digital objects but about how material arrangements work to create user experiences. What Robles and Wiberg [30] call ‘‘the material turn’’ in HCI has foregrounded both the materials from which we design interaction, and the design practice for giving form to material compositions under the notion of ‘‘craft.’’ As a direct consequence, ‘‘the material turn’’ is also a turn toward material interactions. In just about any design tradition, a deep knowledge about, and sensibility for, the materials being used is essential. By living in the world of the materials she uses, the designer sees potential and uses this knowledge in the act of design. Design then becomes a negotiation between form and function and between aesthetics and utility. In HCI, ‘‘the material turn’’ emerged from the concept of ‘‘tangible.’’ Ishii and Ullmer [18] talk about tangible interactions that bridge the digital and physical by creating graspable ‘‘bits’’ [12] that can be manipulated, accessed, and programmed [22]. Work on tangible interactions has helped inspire the re-examination of computation in more M. Wiberg (&) Department of Informatics, Umea˚ University, Umea˚, Sweden e-mail: [email protected] J. Kaye Yahoo! Labs, Sunnyvale, CA, USA P. Thomas Brunel University, Melbourne, UK

material terms (see [2, 5, 19, 24, 31, 35, 41, 43]). Computing is re-imagined as just another material, operating ‘‘on the same level as paper, cardboard, and other materials found in design shops’’ [2], and physical materials are now being re-imagined as substrates invested with computational properties (see e.g., [6, 26, 28, 29]). The result is the activation of existing properties of materials to create opportunities for interaction and experience. These two streams of re-imagination around computational and non-computational materials could be called ‘‘post-representational interaction design’’—or as we have chosen to call it here, material interactions. The notion of ‘‘Materiality’’ is now being applied in a number of different academic disciplines including physics [1], politics [4], philosophy [37], sociology [8, 9, 27], architecture [14, 39], and archeology [16]. It is of course not unusual that these disciplines use ‘‘materiality’’ to highlight the material dimensions of their object of study: It would, for instance, be hard to imagine archeology without materials being a central object of study. The interest in exploring computing through a material lens is also growing in computer science, media and communication studies, social technology studies, informatics, and HCI (see e.g., [7, 10, 15, 20, 21, 32, 33, 36, 40, 43]). These information systems-related areas have always focused on the immaterial dimensions of our world—on representations of things—and the differences between ‘‘real’’ and virtual, or ‘‘physical’’ and ‘‘digital.’’ It would seem that current developments in computing, such as cloud computing or ubiquitous computing, would take us even further away from the material. However, while Weiser [38] did envision a ‘‘disappearing’’ computer, he simultaneously envisioned computation as interwoven with the everyday world: ‘‘The most profound technologies are those that disappear. They weave themselves into the fabric

123

Pers Ubiquit Comput

of everyday life until they are indistinguishable from it.’’ Weiser calls for an approach that integrates the material and immaterial interactions, rather than maintaining distinctions between the material and the computational world, the real and virtual, the physical and digital. This theme issue on Material Interactions attempts to activate discourse around the notion of material interactions in HCI. Our hope is that this issue will provide a wellgrounded platform for HCI research and design that moves beyond the ‘‘material turn’’ to establish ‘‘materiality’’ as a theoretical perspective and as a methodological lens: A similar approach to the ‘‘Media as Material’’ perspective offered by Dourish and Mazmanian [11] or, for example, Belenguer et al. [3] who highlight the importance of considering ‘‘immaterial materials’’ in design. In creating the issue, there was a deliberate decision to include designoriented contributions to move the discussion on materiality forward practically, but the issue explores more than this: The issue suggests a direction for a new practice in interaction design focused on form-giving of computational things that explores notions of hybrid crafting and the crafting of interaction. The issue draws in particular from conversations at the CHI2012 panel Material interactions - From atoms and bits to entangled practices [41, 42]. In the first paper of this theme issue, ‘‘Giving form to computational things—Developing a practice of interaction design,’’ Vallga˚rda [34] takes the idea of shifting perspective from interfaces to materials one step further. Moving beyond a well-established focus on interface design in HCI, Vallga˚rda articulates a perspective which situates interaction design as the practice of giving form to artifacts or environments. She acknowledge how giving form to computational things is highly complex and different to most other form-giving practices due to its temporal form element—its ability to change between states. To enable this encompassing act, she proposes a trinity of forms as a framework to unfold the practice of interaction design, incorporating physical form-giving, temporal formgiving, and ‘‘the interaction gestalt,’’ that is, the form of interaction as performed by the user. She demonstrates how computational composites present a way to work with the temporal form and the physical form in a process not too different from any traditional form-giving practice and that a form-giving practice of interaction design is already well under way. The second paper of this theme issue, ‘‘Hybrid Crafting: Towards and integrated practice of crafting with physical and digital components’’ by Golsteijn et al. [13], makes this focus on form-giving in interaction design more explicit by focusing on ‘‘hybrid crafting.’’ Their paper describes novel means for crafting, including digital media integrations with physical construction. This is articulated

123

through the design of ‘‘Materialise’’—a building set that allows for the inclusion of digital images and audio files in physical constructions by using tangible building blocks that can display images or play audio files, alongside a variety of other physical components. The authors used this set in four hands on creative workshops to gain insight into how people go about doing hybrid crafting and discuss implications for the ongoing practices of both crafting and interaction design. These first two papers situate the current interest in material interaction by exploring approaches to building and creating. The third paper in this theme issue, with the title ‘‘Crafting interaction: The epistemology of modern programming’’ by Lindell [25], makes it clear that if we are to adopt a material-centered approach to interaction design, we should not only think about crafting with different materials, but we also need to revisit the computational and view it through a material lens. We need to understand code as a material as well. In approaching the computational as a material, this paper questions and re-conceptualize what we account for as a material. Instead of upholding distinctions between the material and the digital world, this paper focuses on understanding the digital as a material and its properties and potential in material compositions. Lindell writes, ‘‘this paper investigates the materiality of information technology, specifically programming language code from which interactive artifacts are made.’’ and in line with Vallga˚rda [34] and Golsteijn et al. [13], the author suggest that this understanding of code as design material allows the metaphor of craft to be used for the activity of programming. However, it is not only the practice of HCI that is in urgent need of methods for advancing a material-centered perspective. As the interest in materiality and material interactions grows, there is a similar need for research methodologies capable of guiding our explorations in this area. Accordingly, the fourth paper of this theme issue, ‘‘Methodology for materiality—interaction design research through a material lens’’ by Wiberg [40], highlights the need for a methodology to specifically explore interaction design research through a material lens. The paper proposes a structure and set of methods to work with material compositions in interaction design research, through the perspectives of attention to details, wholeness, materials, and textures. Grounding the discussion in design theory, the paper organizes current methods applied into a four-dimensional structure to guide deliberate choices of methods in different phases of interaction design research projects. The paper also illustrates how the proposed structure can be practically useful—both in advancing our studies of interaction design through a material lens and showing how it brings us back to the roots of our profession—that is, back to a focus on the materials, the fundamental components of any computational composition.

Pers Ubiquit Comput

The notions of ‘‘wholeness’’ and ‘‘texture’’ are both relational concepts pointing at how materials are brought into composition. Shad Gross, Jeffrey Bardzell, and Shaowen Bardzell argues in their paper ‘‘Structures, Forms, and Stuff—The materiality and medium of interaction’’ for the importance of understanding the relation between materials and interaction. They propose that the materiality of interaction can be understood through three different material vocabularies: tangible user interfaces (TUIs), theories of computational materiality, and craftoriented approaches to HCI. The authors argue that each of these offers a formulation of the materiality of interaction: as physical, as metaphysical, or as tradition-communicating. With this as a point of departure, they situate these three formulations in relation to debates on the nature of media, from philosophical aesthetics (the ontology of art, in particular), media studies, and visual cultural studies. In making this argument, they push forward the idea that the formulations of materiality, information, and meaning from HCI and those from the humanities have deeper underlying similarities than may be expected. Indeed, ‘‘material interactions’’ are arguably an object of study ontologically entangled by its very nature. A focus on entanglements in terms of interwoven perspectives and approaches might be useful for advancing compositional thinking in material interactions. The sixth paper of this theme issue, with the title ‘‘Intangibles Wear Materiality via Material Composition’’ by Hyosun Kwon, Hwan Kim and Woohun Lee, the authors argue for the effectiveness of intangible materials when they are coupled with tangible and computational mediums. The authors divide interaction design into the three categories of tangible material, intangible material, and computational material. Kwon et al. [23] argue that while the relationship between tangible and computational materials has been profoundly discussed since the origin of the tangible user interface, intangible materials, such as air, light, and magnetism, have been commonly disregarded as distinctive categorical materials in interaction design. By focusing on ‘‘intangible materials,’’ the authors add a perspective for establishing one of the main ideas behind this theme issue—that all we work with are materials, despite their metaphysical characteristics of being physical, digital or intangible. They propose a framework for material design which allows for material compositions based on configuration dimensions that correspond to the quality of the materials used. While the word ‘‘used’’ in HCI typically means applying a method or putting an interactive system into practice, the final paper of this theme issue by Miwa Ikeyima and Daniela Rosner, ‘‘Broken Probes: Toward the Design of Worn Media,’’ involves the definition of used meaning ‘‘secondhand,’’ or ‘‘previously owned.’’ It does so by describing the development and use of Broken Probes—artifacts whose

marks of degradation become unique identifiers with which to associate and retrieve digitally recorded histories. Sometimes we need to look back in order to move forward, and Ikeyima and Rosner [17] do so by offering the design and deployment of Broken Probes as a methodology for eliciting insights into how broken objects and acts of breakage may be given new life through their integration with ubiquitous computing technologies. Based on these developments, the authors introduce the genre of ‘‘worn media’’—a variety of computational material with which to frame and critically examine the manifestation of wear among digital things. While a final paper of a theme issue could serve the role of wrapping things up and culminating the arguments, this paper instead points forward by discussing how the genre of worn media sensitize designers and Ubicomp researchers to issues of incompleteness, impermanence, and imperfection—perspectives that help us account for the ethical, material, and historical terms of endurance in a digital age.

References 1. Barad K (2007) Meeting the Universe Halfway: quantum physics and the entanglement of matter and meaning. Duke University Press, Durham 2. Bdeir A (2009) Electronics as material: LittleBits. In: Proceedings TEI, ACM, pp 397–400 3. Belenguer JS, Lunde´n M, Laaksolhati J, Sundstro¨m P (2012) Immaterial materials: designing with radio. TEI 2012, ACM Press 4. Bennett J (2010) Vibrant matter: a political ecology of things. Duke University Press, Durham 5. Brownell B (2006) Transmaterial: a catalog of materials that redefine our physical environment. Princeton Architectural Press, Princeton 6. Butera WJ (2002) Programming a paintable computer. Ph.D. Dissertation. Massachusetts Institute of Technology 7. Coelho M, Zigelbaum J, Kopin J (2013) Six-forty by four-eighty: the post-industrial design of computational materials. In: Proceedings of the fifth international conference on tangible, embedded, and embodied interaction (TEI ‘11). ACM, New York, pp 253–256 8. Coole D, Frost S (2010) New materialisms: ontology, agency, and politics. Duke University Press, Durham 9. Dant T (2004) Materiality and society. Open University Press, Maidenhead 10. Doering T (2011) Material-centered design and evaluation of tangible user interfaces. In: Proceedings of the fifth international conference on tangible, embedded, and embodied interaction (TEI ‘11). ACM, New York, pp 437–438 11. Dourish P, Mazmanian M (2011) Media as material: information representations as material foundations for organizational practice. Third international symposium on process organization studies 12. Fitzmaurice G, Ishii H, Buxton B (1995) Bricks: laying the foundations for graspable user interfaces. In: Proceedings of CHI. ACM Press, New York, pp 442–449 13. Golsteijn C, Hoven E, Frohlich D, Sellen A (2013) Hybrid crafting: towards and integrated practice of crafting with physical

123

Pers Ubiquit Comput

14. 15.

16. 17.

18.

19.

20.

21.

22. 23.

24.

25.

26.

27. 28.

and digital components, personal and ubiquitous computing, theme issue. Material Interactions Vol X, X Gramazio F, Kohler M (2008) Digital materiality in architecture. Lars Mu¨ller Publishers, Baden Gross S, Bardzell J, Bardzell S (2013) Structures, forms, and stuff: the materiality and medium of interaction, personal and ubiquitous computing, theme issue. Material interactions, Vol. X, X Hedeager L (2011) Iron age myth and materiality: an archaeology of Scandinavia AD 400–1000. Routledge, London Ikeyima M, Rosner D (2013) Broken probes: toward the design of worn media, personal and ubiquitous computing, theme issue. Material Interactions Vol. X, X Ishii H, Ullmer B (1997) Tangible bits: towards seamless integration of interfaces between people, atoms, and bits. In: Proceedings of CHI. ACM Press, New York, pp 234–241 Ishii H, Lakatos D, Bonanni L, Labrune J-B (2012) Radical atoms: beyond tangible bits, toward transformable materials. Interactions Vol. 19, Issue 1, ACM press Jung H, Stolterman E (2011) Material probe: exploring materiality of digital artifacts. In: Proceedings of the fifth international conference on tangible, embedded, and embodied interaction (TEI ‘11). ACM, New York, pp 153–156 Jung H, Stolterman E (2012) Digital form and materiality: propositions for a new approach to interaction design research, NordiCHI ‘12, October 14–17, 2012 Copenhagen, Denmark Klemmer S, Landay J (2009) Toolkit support for integrating physical and digital interactions. Hum Comput Interact 24(3):315–366 Kwon H, Kim H, Lee W (2013) Intangibles wear materiality via material composition, personal and ubiquitous computing, theme issue. Material Interactions, Vol. X, X Landin H (2005) Fragile and magical: materiality of computational technology as design material. In: Proceedings of the 4th decennial conference on critical computing. ACM, New York, pp 117–120 Lindell R (2013) Crafting interaction: the epistemology of modern programming, personal and ubiquitous computing, theme issue. Material Interactions Vol. X, X Mate´rio (2005) Material world 2: innovative materials for architecture and design (Vol. 2). Birkha¨user, Basel, Switzerland. Miller D (2005) Materiality. Duke University Press, Durham Miller D (1997) Material culture and mass consumption. WileyBlackwell, Oxford Reed M (2009) Prototyping digital clay as an active material. In: Proceedings of the 3rd conference on tangible and embedded interaction. ACM Press, New York, pp 339–342

123

29. Reiger H (2007) Giving materials a voice. In: Proceedings of the 1 st Conference on Tangible and Embedded Interaction. ACM Press, pp 125–126 30. Robles E, Wiberg M (2010) Texturing the ‘‘material turn’’ in interaction design. In: Proceedings of the 4th international conference on tangible and embodied interaction. ACM Press, pp 137–144 31. Robles E, Wiberg M (2011) From materials to materiality. ACM Interact 18(1):32–37 32. Sundstro¨m P, Ho¨o¨k K (2010) Hand in hand with the material: designing for suppleness. In: Proceedings of the 28th international conference on human factors in computing systems. Atlanta, Georgia, ACM 33. Tholander J, Normark M, Rossito C (2012) Understanding agency in interaction design materials in proceedings of CHI 2012, Austin, Texas 34. Vallga˚rda A (2013) Giving form to computational things: developing a practice of interaction design, personal and ubiquitous computing, theme issue. Material Interactions, Vol. X, X 35. Vallga˚rda A, Redsto¨m J (2007) Computational composites. In: Proceedings of CHI. ACM Press, pp 513–522 36. Vallga˚rda A, Sokoler T (2010) A material strategy: exploring material properties of computers. Int J Des 4(3):1–14 37. Watts A (2007) Does it matter? Essays on man’s relation to materiality, New World Library, 2nd edn 38. Weiser M (1991) The computer of the 21st century. Sci Am 265(3):66–75 39. Wiberg M (2011) Interactive textures for architecture and landscaping: digital elements and technologies, information science reference, USA 40. Wiberg M (2013) Methodology for materiality: interaction design research through a material lens, personal and ubiquitous computing, theme issue. Material Interactions Vol. X, X 41. Wiberg M, Ishii H, Rosner D, Vallga˚rda A, Dourish P, Sundstro¨m P, Kerridge T, Rolston M (2013) Materiality matters: experience materials. ACM Interactions 42. Wiberg M, Ishii H, Rosner D, Vallga˚rda A, Dourish P, Sundstro¨m P, Kerridge T, Rolston M (2012) Material interactions: from atoms & bits to entangled practices. Panel at CHI’12, in proceedings of CHI0 12, New York, 2012, pp 1147–1150 43. Wiberg M, Robles E (2010) Computational compositions: aesthetics, materials, and interaction design. Int J Des 4(2):65–76