People largely operate on principles. A principle is a specific kind of mental structure: When the circumstances, established by perception, resemble some specific pattern, perform some specific action. Thinking by principle differs from rational evaluation. Rational evaluation consists of predicting the future outcome based on the specific circumstances and a range of possible actions; we perform the action that we expect will have the best outcome.
(It seems plausible too that rational thought may itself be nothing but thinking-by-principle using a narrowly and rigorously defined set of universal principles.)
It seems fairly clear to me that most animals think by principle; in animals, these principles are directly or indirectly* hard-coded into their brains as instincts. It also seems clear that principles themselves are easily susceptible to biological evolution: there will be variations in hard-coded principles, and the variations that diminish reproductive success will be naturally selected against.
*One example of indirect hard-coding might be imprinting. For example, whatever a baby duck first sees when it hatches becomes "mom"; in this case, the specific perceptions are not hard-coded, but the propensity for taking an imprint is hard-coded.
Thinking by principle is, in general, an extremely efficient way of dealing with a complicated world. For example, I'm a pretty good poker player. I almost never actually calculate the probabilities when I play. Instead I've developed a set of principles that guide my play. I'm almost pathologically slow at actually calculating probabilities (and arithmetic in general), a deficiency that prevents me from ever being a top professional player. My principles are sufficiently effective, however, that I almost always win against players at home, and I have a noticable advantage against skilled amateurs at casinos.
One advantage that human beings have over animals is that we can use our brains to think at multiple levels of abstraction. Therefore, we can create meta-principles: When a pattern of principles (perhaps in addition to a pattern of perceptual circumstance) are activated, perform some action. Since a meta-principle is a principle, we can extend the above definition: When a pattern of principles or meta-principles (or meta-meta-principles, etc.) are activated, perform some action. Furthermore, we can affect the contents of our own brain*: one of the actions we can perform is to positively or negatively reinforce a principle.
*Robert Anton Wilson eloquently describes this capability as that we are our own meta-programmers.
Thinking by principle allows us to operate effectively in situations where actually computing all the outcomes is practically impossible (even if it is tractable in theory). Our brains have only a limited amount of computational power; the computational compression afforded by principles is considerable. Consider, for example, chess-playing computers. It required a $10,000,000 computer, one that could compute 200,000,000 potential outcomes per second, to beat a human being, generally capable of evaluating at best only tens of moves per second. We can thus estimate that thinking by principle is about ten million times more efficient than by pure computation. (Actually, Deep Blue did have some thinking-by-principle as part of its program, so the computational efficiency of principles might be an order of magnitude or two higher.)
Thinking by principle, especially that principles can evolve socially, also allows us to act effectively in situations where actually computing the outcome is metaphysically impossible* or theoretically intractable (exponential or factorial complexity).
*For example, it is metaphysically impossible to win at poker by playing a computationally perfect game. A computationally perfect game ensures you will always break even in the long run, regardless of your opponents' actual play.
Furthermore, general principles (principles that almost always achieve an optimal or near-optimal (or correct or near-correct) outcome) are almost as valuable as universal principles (principles that always achieve an optimal or correct outcome), and there are many many orders of magnitude more good general principles as there are universal principles.
We can plausibly conclude, therefore, that the human brain is not a "rational thinking" machine; it is, rather, a machine that performs sophisticated and complicated management and application of principles. Furthermore, we can conclude that the principles in actual operation in a population is the result of an evolutionary process: heritable/transmissible variation in principles (and princple-sets) dialectically coupled with a negative selection process, a process that (somehow) removes egregiously bad principles.
There are, however, some drawbacks to using principles. Most importantly, Godel's theorem (and the related Church-Turing thesis) implies that there is no general algorithm that can evaluate a set of general principles for optimality. Any algorithmic or computational evaluation of a set of principles has to be done on a case-by-case basis, when it can be done at all.
Thinking-by-principle can lead to known irrational results. For example, if a principle is strong (i.e. the brain's "principle management system" assigns a high weight to the principle), a person might adhere to a principle even under specific circumstances where they can and do rationally know that contravening the principle would be optimal. Most people, for example, have a strong aversion to stealing; they will "irrationally" refrain from stealing even when they know they probably won't get caught, they know if they do get caught the consequences will not be severe, and the specific theft will not have any noticeable effect on the mutually beneficial state of a minimal-theft society. More specifically, many people will walk back into a store to pay for a stick of gum that has been inadvertently overlooked by the cashier. In most cases, these instances of irrationality are benign: the computational efficiency afforded by thinking-by-principle usually sufficiently outweighs the known failures, and natural selection helps remove principles with catastrophic failures.
Thinking-by-princple leads more problematically to the philosophical error of deonticism, the mistaken belief that principles have value independently of (as opposed to different from but necessarily related to) their outcomes. This error seems almost inevitable for any philosopher who understands neither evolution by variation and selection (every philosopher before Darwin, notably Emmanuel Kant, and many after) nor computational and complexity theory (i.e. just about everyone).
Principles have to be transmitted for evolution to work on them. Also, principles that establish global mutual benefit by sacrificing local "selfish" benefit (i.e. principles that optimize mutual benefit in Prisoner's Dilemma situations) have to have sufficient inherent strength to outweight the local "selfish" benefit. A person's brain, therefore, has to have some degree of "stickiness", the strength that received principles from one's parents, culture and society are adopted.
Like every trait, we expect the degree of stickiness to vary among individuals, the degree of stickiness to be heritable or transmissible (either genetically or verbally/culturally) and for some degrees of stickiness to be selected against. Too little stickiness, and each individual has to figure out the whole world from scratch, an impossible task, and therefore selected against. Too much stickiness, and a person cannot adapt to changing circumstances, also selected against.
There are three important concepts:
- People typically think using principles, not by direct rational evaluation
- People's brains have varying degrees of "stickiness" for adopting new principles
- Both the content of people's principles as well as their "stickiness" are shaped by selection, an inherently negative process