Language of Interaction (from Interaction08)

April 29, 2008 | Comment?

For my talk at Interaction08, I published a paper to accompany the talk, and then hosted it at a separate site. I’d like to bring it into here, continuing the theme of bringing everything under one roof. Inside are the video and slides of the talk. Enjoy!


We are interaction designers during a time of rapid technological change, placing us in the incredible position of collectively creating and curating a new language, the language of interaction. The explosion of products with interfaces means that people have to continually adapt and learn new things. They will have to read each interface and look for clues, common visual elements that they’ve seen before. They’ll subconsciously look for a language that comprises words, sounds, colors, shapes, icons, motion, gestures, priorities, hierarchies and more. We are the custodians of this language, creating and curating it organically. We need to start seeing it everywhere and learn from each other as we define the future relationship between people and technology.


Here is the presentation given at Interaction08 in Savannah.

And here is the video. Hopefully you can advance the slides along with the video.

We make things for other people to use, but we can’t show them how.

Everyone in our industry, this amorphous world of experience design, in one way or another makes things for people to use. We make websites for people to access their bank accounts, book flights, read about the world, find a date, find a free couch, or publish their thoughts and emotions. We create products that they carry around in their pocket or bag so they can contact a friend or take a picture. We are in the business of making products for actual people.

Although we make things, grow attached to them and release them to the world, we can’t go home with them to instruct their new owners on how they should be used. We can’t point to that button right there, no, that one, to guide people through the interface. The product or device or website has to, on its own, communicate the instructions for how to use it. This is what we mean by intuitive: the instructions for use are so successfully embedded in the interface that little conscious analysis is required to begin using it immediately.

But how do people learn how to use the products we make? From where do they and we learn the design lexicon necessary to interpret an interface? From other products. The challenge is that people are learning from a lot of really bad examples. They’re surrounded by a lot of products that haven’t been thought through, skewing their view of technology. Bad ovens, ATM machines, alarm clocks, gas pumps, cell phones, websites. Every bad experience reinforces negative expectations of products. We have to see what people face every day so that we can make products easy to use and love.

Note: From here on, I’ll use the word product to mean any piece of technology with an interface. This can be a website, a car stereo, a washing machine, a cell phone application, a kids’ toy, a software application, a dialysis machine, a car stereo, and so on.

People learn how to use technology from other technology.

People are becoming more technically savvy. They’ve learned from experiences with other products which in turn creates expectations for how the next product will work. When this happens, their style of learning becomes different, more at the surface level than at the abstract level. They mature from what is this? to how does this one work?

For example, many people by now know how to use a MP3 player, abstractly. “Load the songs I want on it, find the artist or album that I want to hear, and play it.” That’s an abstract MP3 player. “Enter in my checking account number and password, then transfer money from this account to that account.” That’s online banking, abstractly. This means that learning a product is evolving into learning how to use this instance of a product. “Where are the tools for loading music? How do I use the controls to find and play songs? Where do I transfer?” We have to decode the interface in front of us. We have to read it to decode the embedded instructions.

When we figure out a new product, we look for familiar visual cues to guide us, elements that we have seen before in other products and have learned in the past. It’s very natural and normal: we learn from experience. Our brains are always looking for the patterns in what we see, trying to find the consistency, looking for the language. We subconsciously latch onto the most familiar interface elements and construct the instructions from there.

For example, consider a stacked list of words or short phrases. On a website, when we see a stack of words, especially when they’re on the left side of the screen, we assume them to be navigation. These words can have no ornamentation at all, and yet their layout and position alone can communicate navigation. Now think about the table of contents in a book, a layout device that was invented about 500 years ago. It’s a stack of words that navigates a book. We gravitated easily to website navigation because it looked familiar to something we’ve seen many times before.

For example, consider the colors green and red. They have become standard metaphors for good and bad, in software, on status lights, on EXIT signs, on street lights, throughout the world of interface design. In fact, their universality made it easier to use them in product interfaces. But these metaphors are just arbitrary. There’s nothing innate in red that means bad or green good, unless we trace their meaning all the way back to the pre-language logic of fire bad, tree pretty.

Another example, consider an equilateral triangle pointing to the right. We generally interpret this in interfaces as play or forward, depending on the context. In fact, these two notions have the same root, the play icon first indicating the direction the tape moved forward in reel-to-reel tape decks. Now, the play icon is a standard; it will be difficult for any other symbol to supersede it, especially when combined with the two vertical lines that mean pause. This ubiquity has advantages, allowing the start/stop metaphor of play/pause to migrate into other product types. Washing machines from companies like LG now use the play/pause icon. Even though it originally meant make the tape play, when someone encounters a right-pointing triangle and two vertical bars on a washing machine, without any accompanying labels, they immediately understand that it means start and stop, because they’ve learned that from other technology.

All language evolves. The language of interaction is no different. We interaction designers need to be aware of this, notice when it happens, and introduce innovations from other fields that evolves the language for the better.

We depend on the interface more as products multiply.

Complicating this challenge for people to learn new products is that new types of products are continually invented. We are still in the early years of a technology explosion where soon the possibilities will overwhelm us. The variety of inputs (keyboard, mouse, touchscreen, gesture, touch fabrics, other sensors) and outputs (big screen, little screen, projection, ink-based displays, digital paper, display fabrics) leads to endless combinations of potential product. The ease of incorporating interfaces into products will create a lot of great ideas, and a lot of bad ideas.

This future radically increases the dependence on the interface itself, the one common element among all products. In fact, as materials science produces more input options, the entire product will be interface. In such a rich productsphere, a common language of all the elements of an interface will be the only way to avoid sheer product overload. The interface becomes the bridge from one product to another. The key will be knowing how to interact with any product.
Furthermore, globalization will increase the diversity of people using the products we create. While English is currently the default language in interfaces, icons, colors and gestures have a better chance at becoming universal than words do. As interfaces go global, so must the elements that comprise them.

We create and curate the language of interaction.

Here’s the part I find most fascinating, looking at it from a long-term perspective. The rapid rise in the web, communications, software, etc. over the last ten years is changing into the greater language around us, incorporating the words and symbols that we use in interfaces.

The types of elements that comprise the language of interaction are clear to us all. Words, icons, colors, shapes, sounds, motion, gestures, and speech are the individual elements. But we also need to notice and consider how these elements are treated in relation to each other. Priority, hierarchy, position, layout, and repetition all generate meaning on their own and thus also become elements of interaction. All together, these items form a lexicon.

This is just the beginning. As tech products become more and more enmeshed into our daily lives, our interactions with technology will become commonplace. Maybe even standardized. Maybe some day we’ll be able to approach any piece of technology and immediately use it because the language embedded in it has become universal.

But language is organic. It’s always evolving from countless decisions made by people to use words, avoid others, and make new ones up. This is exactly what we do when we create, design, improve or craft an interface. We select icons that we know will work, avoid icons that won’t, and try new icons that we hope will stick. Same with labels. Same with colors. And we are beginning to do the same with gestures. We need to notice everything because everything is becoming interface. We need to really see and deconstruct all the everyday technology around us so that we understand the context in which people are learning how to use products.

We in the user experience field are creating the language of interaction on the fly when we make the products that people use and touch. We are curating the language of interaction by analyzing it and improving upon it. We (ideally) review our work with real people to see what makes sense and what doesn’t. We toss the bad and keep the good. We are collectively building the foundation for how people and technology will interact for the next several decades, and if we’re not too dumb, centuries.

Interaction design is an amazing field to be in right now, with opportunities and challenges we can’t even imagine yet. Let’s have fun with it, learn from each other, get involved in the community, and do mind-blowing work. Thank you.


Add your comment below, or trackback from your own site. Subscribe to these comments.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>