Abstract Affordances

Stanley Milgram

This is an essay about linguistic relativism and how it influences hardware and software interface design and can inspire art as well. Design is vastly improved by adherence to standards and universals in affordances, which are the implicit functions in an object. But when affordances are changed at any level, even how we refer to them with language, it can sow confusion. As we enter the age of AI, this is of crucial importance. You wouldn't want a curvilinear object that is supposed to fit in your hand be dangerous in some way in its programming, or how it pings the internet. We can be creative with industrial design until it becomes harmful physically, as well as psychologically.

***

The Tivoli radio still uses the manual dial on its radios. As an elegant design solution, these types of controls can make an interface (seemingly) easy to use, and accordingly, are supposed to make life simpler. But the single knob or control can also simply hide complexity, as it originally did with the first iPod with the Click Wheel, which was subsequently incorporated into the smartphone as a virtual software control. What didn't get virtualized was the embodied memory of the control. After I got my first smartphone, I found myself using swiping motions on my e-ink Kindle.

The reflexive response for the design of new interfaces is that they should be touch-driven rather than twisting motions of the wrist. I don't think we want to remove analog controls because our brains are still pre-wired to those other gestures, as they contain vestigial wisdom and meaning. With purely digital controls, knobs that don't twist don't have the same tactile feedback, and what that feedback means to the human connectome. With the digital version of the knob, we aren't just selecting soft or loud; There could by myriad functions below it. Tactility is an important controller of brain activity--which is why I always promote the idea of playing traditional musical instruments because of the immediate feedback we receive from the hands as they touch the strings or keys. If everything you do in life is by using a screen, including music, the brain might not differentiate between touching a button on a screen to place your order for dinner, and playing a rhythm, even though this is conceptually interesting.

I like the idea that a range of values can be assigned using the knob control, and wire it to some other attribute, such as volume controlling color or saturation, or volume controlling opinion-fact, or mean-kind to a range of colors or saturation as a kind of "metaphor dial". Because of the relativistic nature of language and culture, assigning labels to 0 and 100 can be disorienting when placed in a left-right configuration. When we say "love and hate" it puts love at zero volume (i.e. something that is silent), and its polar opposite at full volume. "Hot and cold" puts hot at 0. Turning a knob to the right always will mean an increase in something. On faucets, hot is always on the left. It would be particularly difficult to change the order of many word pairs/axis points without causing confusion between the interface and its operation. Cold on the right was already grandfathered in; Changing it at that point would have safety issues. Metaphorically, while a word may get top-billing in a pair, assigning it to an axis point is another issue: one word is 0 and one is 100. Take loud and soft for example: If we turn the volume up, "soft" is loud. (Incidentally, the search result for "loud and soft" on Google supports the preference for that word order as opposed to "soft and loud"). Once these get burned-in we probably should not change them for our own safety, regardless of the 0-100 scale, and avoid good-bad. If we controlled this with a knob, we want good to be as loud as possible, but the left-right reverses this.

"For example, there's the Knob...perhaps by now you can guess what kind of action-sentence compatibility effect you could measure using the Knob. Researchers at the University of Rotterdam had people listen to sentences that implied clockwise rotation [and they all chose the same direction]."  Bergen, Benjamin K. Louder Than Words: The New Science of How the Mind Makes Meaning. New York, NY: Basic Books, 2012. Print., pp.82-86

Language can be a problem of managing cognitive dissonance as well and sets up biases accordingly. The hot-cold/left-right standards may automatically set up a colloquial bias for red-blue (We typically say "red and blue" not the reverse) on the same endpoints. Black-white would then put black at the same left axis point as red and hot in the other word pairs. (At least in English, but shouldn't matter, as in any language certain values will occupy the same axis points). Icons and symbols have standardized this to some degree, but with the advent of emoji, semiotics are too vague, and simply using the word becomes the better option. Words are also symbols in themselves.

All of this becomes important for UIs for AIs.

Spatial metaphors can also cause semantic conflicts, not only in language but in any temporal phenomenon such as music. We assume that tuning errors are "vertical"--that two notes sounding at the same time can be out-of-tune or out-of-key, but dissonant intervals can be spread out horizontally, such that B-natural will sound out of place within one or two bars in the context of D minor for example which has a Bb in its key signature. Of course, this can be "fixed" by making the wrong note "lean into" the correct note by anticipating or suspending it (appropriately called an "appoggiatura", meaning "to lean"). But if we are sensitive to the sequence or pattern of such "mistakes", then this little error on the B-natural will have to occur again so as to give more weight to the idea that it is intentional and not a mistake. But our burned-in sense of key, probably would not convince us that this is intentional. That's what keys are about in music: to establish at the outset what pitches will be used within a given time frame. "Accidentals" are used literally to mark things as intentional "accidents" and not mistakes. Music standardization is a good general standard for other domains as well. Once something is learned over the course of many generations, it is extremely hard-wired, and designers should take that into account, before making drastic changes that would cause confusion or dissonance. This is where consideration of standards and universals are useful.

Top-Billing

Even if binary judgments can be reconciled with a dial or knob (even one that uses numbers and not words), the fights are still being waged over the left-right binaries. The order in which we say things or understand them form alpha-omega value judgments. Always volatile, but flipping or subverting the soft-loud, left-right conventions can challenge the rigidity of our biases and assumptions--and even fun when you can assign colors to McCartney and Lennon, adding blues to red as you turn it up the Lennon "volume". But blue-red seems strange for mean-kind in relation to its political party equivalent. The Lennon-McCartney marquee is still a battle for vanity: No one says "McCartney-Lennon", even when McCartney was the primary writer on many songs. It is however logical in terms of the soft (McCartney)-loud (Lennon) axis, even though they could be both. Making Lennon "red" on a political axis is a repulsive thought, but the 60-year old colloquialism of "Lennon and McCartney" probably will never change.

Some left-right labels for a knob or dial:

Analog-Digital (turns up the pixels)
Bitmap-Vector (turns up the resolution)
Bottom-Up/Top-Down  (ways of working)
Computers (Love-Hate)
Conformity-Compromise
Easy-Hard
Fat-Lean
Fragile-Strong
Hard-Brittle
Hard-Soft
He-She
Kind-Mean
Liked-Not Liked
North-South
Old-New
Opinion-Fact
Pink-Blue
Quiet-Noisy
Rectilinear-Curvilinear
Shallow-Deep (depth of field)
Smooth-Rough
Hard-Soft
Soft-Strong
Sweet-Tart
Truth-Beauty