Amory Sivertson — Hidden Levels : Stick It to 'Em

In one sentence

The joystick succeeded as a global standard because it mastered "perceived affordance," creating a seamless bridge where human biomechanics and machine actions become one.

Why read this (who it's for)

This summary is for designers, engineers, and anyone interested in human-computer interaction (HCI). It explains why some technologies (like the joystick) become "invisible" while others (like the touchscreen) remain "monolithic." You will learn to identify why certain interfaces feel like an extension of the self while others feel like a barrier.

The big idea

The joystick is not just a gaming peripheral; it is a triumph of interface design. While many technologies aim to be "useful," the joystick aims to be "invisible." It achieves this through perceived affordance: the idea that an object’s physical form tells you exactly how to use it without a manual. You see a joystick and you know how to move something through space.

The evolution of the joystick—from the Wright Brothers' hip-tilting maneuvers to the dual-analog sticks of the 1990s—represents a constant refinement of how we translate human intent into digital action. Because it aligns so perfectly with our natural biomechanics (moving a hand to move an object), it has been adopted by the military for drone warfare and by the medical field for robotic surgery. It is the ultimate "man-machine" interface because it allows the user to stop thinking about the tool and start thinking about the action.

Key ideas

Perceived Affordance

• What it is — The quality of an object's design that makes its function immediately obvious. A joystick "affords" movement because its shape and position suggest how it should be gripped and moved.
• Why it matters — When an interface has high perceived affordance, the tool "disappears." The user stops looking at the hand doing the pointing and starts looking at the destination.
• In the book — David O’Grady notes that while knobs on the Magnavox Odyssey required a manual to understand, a joystick allows a user to understand the motion of moving through space intuitively.

The Separation of Movement and Looking

• What it is — A design principle where the action of moving through a space (navigation) is decoupled from the action of observing that space (camera control).
• Why it matters — In 3D environments (like first-person shooters), trying to do both with one input is "inelegant." Separating them allows for "mind control" style navigation.
• In the book — The transition to the "two-thumb" solution (dual analog sticks) in the late 90s allowed players to move and look independently, mimicking real-world behavior where we can look one way and move another.

The "Invisible" Art of Interface

• What it is — The goal of design where the interface becomes a seamless extension of the self.
• Why it matters — Great design isn't about the features of the tool; it's about how much the tool allows the user to forget the tool exists.
• In the book — The joystick is described as a "man-machine" interface that allows for "automatic" action, where the human and machine become one.

Familiarity as a Weapon

• What it is — The intentional use of existing, perfected interfaces to solve new, complex problems.
• Why it matters — Instead of reinventing the wheel, organizations can leverage the billions of dollars spent by the gaming industry to "perfect" a user experience.
• In the book — The U.S. Military adopted the FMCU (a controller that looks like a PlayStation/Xbox controller) because Gen Z and Alpha "grew up on these systems," allowing them to adapt to military tasks "almost instantaneously."

The Monolithic Nature of Touchscreens

• What it is — A "flat piece of glass" that offers zero inherent affordance, relying entirely on software to tell the user what to do.
• Why it matters — While touchscreens offer high versatility (shopping, maps, social media), they lack the precise, physical "body-machine" connection of a joystick.
• In the book — Niklas Nyland argues that touchscreens are "scarily monolithic" because they force every interaction into the same flat plane, whereas a joystick offers a physical, human-machine connection.

Frameworks & tools

The Affordance Audit

Use this to evaluate any new tool or software interface you are designing or adopting.

1. Identify the Action: What is the primary physical action the user needs to perform? (e.g., "Move a cursor," "Rotate a dial").
2. Check the Affordance: Does the physical shape of the tool suggest that action without instructions?
3. Measure the Friction: Does the user have to think about the tool (the "how") or the goal (the "what")?
4. The "Invisible" Test: If the user can perform the action while looking only at the target, the interface has succeeded.

Memorable examples & stories

The Wright Brothers' Hip Cradle

The first planes were controlled by the pilot tilting their hips in a wooden cradle and pulling a lever. It was a "full-body activity" that required months of training to master. Lesson: Early technology often requires us to adapt our bodies to the machine; great design eventually adapts the machine to our bodies.

The Atari 2600 "Stiff" Joystick

Amory Sivertson tried the iconic Atari joystick and found it "stiff to the point where it hurts my wrist." However, for the 1970s audience, it was the pinnacle of design because it was the first time a home console offered a "joyful" and "accessible" way to navigate a game world. Lesson: "Good" design is often a snapshot of what is possible at a specific moment in technological history.

The "Two-Thumb" Revolution

The move to dual-analog sticks (like on the N64 and later) was a response to the need for 3D space. It allowed for the separation of "looking" and "going." Lesson: When a new dimension of complexity is added (3D space), the interface must evolve to separate competing actions.

The Military's "Game" Controllers

The U.S. Army and the Armed Forces of Ukraine use PlayStation and Xbox-style controllers to operate drones and machine gun turrets. They chose these because the gaming industry already "did the job" of perfecting the interface. Lesson: Don't reinvent the wheel; if a tool is perfected for "joy," it can be perfected for "precision" in a high-stakes environment.

Key quotes

"The joystick... is an attempt to solve a fundamental issue, which is how do you provide human input to a device or a machine?" (David O’Grady, paraphrased) Why it matters: It frames the joystick not as a toy, but as a fundamental solution to human-machine communication.

"The joystick has what’s called 'perceived affordance.' You see one, and you know exactly how to use it." (David O’Grady, paraphrased) Why it matters: This is the secret sauce of why the joystick became a standard.

"The gaming companies... spent tons of money developing what is optimal, intuitive, and easy to learn. And the military is basically saying, 'Well, they did the job for us.'" (Jared Keller, paraphrased) Why it matters: It highlights the incredible value of "civilian" innovation in solving "military" problems.

How to apply it

This week

• Identify a "High Friction" Tool: Find one piece of software or hardware you use daily that feels "clunky" or requires you to think about the tool rather than the task.
• The Affordance Audit: Apply the "Affordance Audit" (from the Frameworks section) to that tool. Ask: "Does this shape tell me how to use it, or do I have to learn it?"

Ongoing

• Design for "Invisible" Interaction: When creating any process or tool, ask: "How can I make the interface disappear so the user can focus entirely on the goal?"
• Leverage Familiarity: When tasked with teaching a new skill, look for "pre-existing" skills the user already has (like gaming or smartphone use) and map the new task onto those familiar movements.

Common pitfalls

• The Touchscreen Fallacy: Don't assume a touchscreen is a better interface just because it's more versatile. It lacks the "perceived affordance" and physical connection of a joystick for tasks requiring high precision.
• Over-Engineering: The military's success with game controllers proves that sometimes the best solution is the one that already exists. Don't build a "custom" solution if a "standard" one is already perfected.

Connects to

• The Design of Everydaythings by Don Norman: Specifically his work on "affordances" and "signifiers."
• The Age of Wire and Wireless Programs by David Weinberger: For a broader look at how digital interfaces change human behavior.

If you remember only three things

• Perceived Affordance: Great design is intuitive; you should know how to use a tool just by looking at it.
• Invisible Interfaces: The best tools are those that "disappear," allowing the user to focus on the action, not the instrument.
• The Power of Familiarity: The gaming industry spent decades perfecting the "man-machine" interface, providing a blueprint for everything from military drones to robotic surgery.

Internalize it

• Reflection questions

  • In what areas of my life am I using a "clunky" interface (a process, a tool, a habit) that forces me to think about the tool instead of the goal?
  • How can I apply the concept of "perceived affordance" to my own workspace or the way I organize my daily tasks?
  • If I were to design a tool for a complex task, would I try to invent something new, or would I look for a "perfected" interface that already exists?

• Flashcards Q: What is "perceived affordance"? A: The idea that an object's physical design communicates how it should be used (e.g., a joystick "affords" movement). Q: Why is the joystick considered an "invisible" interface? A: Because it allows the user to stop thinking about the tool and focus entirely on the action. Q: What was the primary reason the military adopted video game controllers? A: Familiarity and the fact that gaming companies had already spent billions to perfect an intuitive, high-precision interface. Q: What is the difference between a joystick and a touchscreen in terms of affordance? A: A joystick has high perceived affordance (you know how to use it by looking at it), while a touchscreen is "monolithic" and relies entirely on software to tell you what to do. Q: What was the "two-thumb" solution's purpose? A: To separate the action of "looking" from the action of "moving" in 3D space.

Fidelity note

This summary is based entirely on the provided transcript of "Hidden Levels #2: Stick It to 'Em." I am highly confident in the technical definitions of "affordance" and the historical timeline of the joystick as presented by the speakers. The transcript covers the evolution from aviation to gaming to military/medical use, though it does not provide specific technical specifications for the FMCU controller beyond its visual similarity to a PlayStation/Xbox controller.