Tuesday, March 27, 2012

Intuition and Expertise

Originally published March 21, 2012: 

  Today I challenge Nobelist Daniel Kahneman on his own ground.  Wait!  Let me have my say before you cry chutzpah!  Then you can decide for yourself whether I'm bonkers or not.

  In a recent book for the lay reader, Thinking, Fast and Slow, Kahneman summarizes his life's work: understanding how we make decisions.  He postulates that we use two mental mechanisms for this purpose.  One is fast and intuitive ("System 1"), the other slower and more analytic ("System 2").  System 1, the focus of the book, probably developed evolutionarily so that we could reflexively act when we sensed the presence of a saber-toothed tiger.  We still use it to jump out of the way of a speeding car.

  But Kahneman asserts that System 1 also makes snap intuitive decisions in non-emergency situations.  These are more likely than not to be wrong, he says, because we are led astray by factors irrelevant to the problem at hand.  We are influenced by special insights we think we have and biases we do have. We may unwittingly replace a hard problem we are trying to solve with an easier problem we know how to solve.  Or we simply may be dyspeptic from a meal we just ate.  Kahneman lists many such extraneous factors, of which we are generally unaware, and demonstrates their deleterious effects by presenting data from numerous sociometric and psychometric experiments. 

  I have no quibble with him so far.  I agree that our distant ancestors, in learning to leap away from saber-toothed tigers, made it all too easy for us to leap to conclusions.  But Kahneman has given intuition an unwarrantedly bad name.  The thrust of his work has been to show that we largely act through irrational, error-ridden intuitive impulses.  He therefore gives short shrift to a huge domain where the intuitive impulse is rational and reliable: expertise-driven intuition, or expert intuition for short.

  Kahneman has relegated this domain to a tiny sliver of his book, only one of 38 chapters, where he describes his extended "adversarial collaboration" with Gary Klein, a defender of expert intuition.  In other chapters, Kahneman harps monotonously on examples of "experts" being derailed as readily as non-experts by extraneous factors.  Alas! the poor man has spent much of his life--and most of this book--studying "experts" like stock market analysts, CEOs and pundits.  Not my kind of expert or expertise, nor Klein's.

  As I use the term, expert intuition involves the years-long migration of expertise developed by System 2 into System 1.  Even Kahneman notes that while a chess novice might spend minutes or hours slowly thinking through the likely results of any move (System 2), a grandmaster can intuit the same results from a glance at the board (System 1).  Klein has studied how experienced firefighters can intuit from subliminal sensory cues when a building is about to collapse, without knowing quite how they do it, and act instantly on that intuition. 

  Kahneman and Klein did come to some agreement, among them conditions under which the intuition of experts can be valid, namely an environment that is sufficiently regular to be predictable, and an opportunity to learn these regularities through prolonged practice.  Those conditions apply to problem solving in engineering, so it is a good platform from which to launch my paean to intuition.

  I was fortunate to have been taught by a grandmaster of engineering intuition: MIT Professor Ernst Guillemin.  I still smile almost 65 years later, remembering how his gnome-like body would practically dance across the front of the classroom, showing us how he thought an electron would think and act in trying to make its way through a circuit he had drawn on the board.  "You have to think like an electron to understand this circuit!" he would exclaim. 

  Many years later my daughter Abby, then a pre-architecture student taking a structures course, was given the problem of constructing a truss from a given amount of poster board.  The function of the truss was to support the greatest amount of weight when suspended between two table edges separated by a given distance.  I told her, "You have to think like the truss!  Imagine yourself suspended face down with hands on one table, feet on the other, a weight tied to your midriff.  Where would you hurt most?  Use your limited resources to reinforce the truss in those places."  I am proud to say that her truss supported more weight than any of her classmates', even more than her professor's.

  In 1976, I wrote an article, "In Praise of Intuition," for a student engineering magazine.  I tried to define intuition as applied to engineering problem-solving and to show how it can be developed.  A precise definition of engineering intuition still eludes me, but, as then, I think it consists of at least the following abilities, successively refined over a long time:

  -- To know how to approach the problem without quite being sure how you know.

  -- To recognize what is peripheral and what is central, without having fully understood the problem.

  -- To perceive in advance the general nature of the solution.

  -- To instantly connect a problem in one field to analogous problems in other fields and import the analogous knowledge into the problem at hand.

  -- To sense when a solution must be right, just because "it feels right," and to suspect a solution that feels wrong and continue to gnaw at it.

  I suspect that Kahneman would vigorously argue against at least the last ability, calling it an "illusion of validity" likely to be biased by overconfidence.  But I hold my ground.  For engineering problems, and likely much more broadly, I think it is indeed possible to learn the abilities I listed above, including the last, through inculcation of certain habits of mind.  Among these habits are:

  Systematically fashioning knowledge and experience into an integrated whole.  Like a piece of a jigsaw puzzle, a new idea snaps into place more easily as you amalgamate your knowledge into a coherent pattern and as you gain experience at fitting pieces into that pattern. And the more complete the overall pattern, the more easily you sense what is missing.  This pattern of knowledge embeds itself into your intuitive thinking.

  Persevering in understanding.  Studiously avoid shrugging off the mysterious as beyond your ken.  What is novel and counter-intuitive on first exposure can become part of your intuition after repeated, usually laborious re-examinations.

  Looking at a problem and tentative results from multiple viewpoints.  If you are a mechanical engineer examining an electrical circuit, think of its mechanical analog, which you are more likely to understand.

  Projecting the general from the particular and checking the particular against the general.  Like Archimedes in the bathtub with his famed Eureka!moment, if you find that 3+9=9+3 and 4+2=2+4, immediately conjecture that x+y=y+x.  Conversely, if someone asserts that x+y=y+x, immediately make sure that 3+9=9+3 and 4+2=2+4.

  Focusing on the core of a problem without initially getting bogged down with matters of peripheral importance.  If your car's engine isn't running, forget for now about waxing it.

  Perhaps hardest, not fearing mistakes.  Don't be frozen into indecisiveness because "all the facts aren't in yet."  You will never have all the data you need, but you can still make a decision based on your expert intuition, knowing that you might be wrong.  The hard trick is to know when to act.  I believe that comes from learning the boundary between when a solution feels right and when it still feels wrong and needs more gnawing.

  I don't think expert intuition can be taught in a straightforward manner, beyond championing the rules above and demonstrating them in practice, as Professor Guillemin did for me.  If anything, teaching expert intuition is a process like the Zen Buddhist use of koans: that is, teaching intuition intuitively.  But even if not easily teachable, expert intuition can nonetheless be learned, by inculcating these habits of mind over tens of thousands of hours of practice. The discipline of doing so, Kahneman's pessimism notwithstanding, can inure one to many of the destructive influences he cites.  And there's a bonus: That discipline will apply beyond the field in which one spent all of those hours.  The habits of mind themselves will migrate into System 1, available for more general problem solving.