A theory of institutions

While individual human beings can conceive of "big" long-term, global goals, it's difficult to act directly on those big goals. Part of the problem is psychological: "small" short-term local goals have a lot more psychological motivating force. Hence, when people conceive of big personal goals, they usually break those goals down into a set of small goals. (Hence, for example, most task tracking software will actually show the stricken-out description of task when it is completed, rather than simply delete it; we get a small internal reward from seeing that task completed.) Big goals are harder to act directly on because we usually can't just tell what activity at the moment will efficiently achieve the goal. So, for example, if a group of people want to build a spacecraft to travel to the moon, they have to coordinate among themselves to figure out what each person should do; now, each person has a small goal, and the aggregate of those small goals will (hopefully) result in a person standing on the Moon.

Thus, human beings create institutions to achieve long-term, global goals by implementing short-term local incentives for individual people to do specific things.

An institution can be something concrete, such as a named, physically aggregated organization, such as Apple Computer, NASA, the Catholic Church; something semi-concrete, a part of an organization, such as the enlisted men and women of an army, or something abstract, such as capitalism or the (more-or-less) free press.

What makes something, concrete or abstract, an institution, then is that there are small, short-term, local incentives that map to big, long-term, global goals.

One way to characterize institutions by looking broadly at the small incentives and how they map to big goals. Some institutions will use more than one of these patterns (and almost all use some bureaucracy). Briefly, these patterns are:

Type	Short Term Incentive	Long Term Goal
bureaucratic	precisely follow rules and procedures	complicated but well-defined and well-understood, eliminating personal bias
creative	solve small problems	big, poorly-understood or poorly-defined
military	give and follow orders	dangerous and/or morally problematic
deliberative	voting	politically contentious
self-interested	personal gain	undefined evolutionary

For example:

Type	Examples
bureaucratic	DMV, IRS, an individual McDonald's restaurant
creative	scientific research, engineering, education, advertising, the general staff of an army, senior corporate management, team athletes
military	enlisted and field officers of the US Army, rank-and-file police and firefighters
deliberative	legislatures, direct democracy
self-interested	capitalism, markets, individual athletes, fine arts, literature

Most institutions use a combination of short-term incentives. For example, almost every institution has some rules that have to be followed precisely; thus, they have some bureaucratic incentives. Institutions often interact with each other, and this interaction can be institutional, and follow one or more of the patterns noted above. Some institutions, such as the relation of a military general staff (a collection of generals) to the field officers (colonels to captains) is often 50-50 creative and military (with a lot of bureaucracy). Sometimes, creative institutions are also deliberative.

Basically, individuals in an institution are rewarded or punished based on the small incentives. A functional institution is one in which the aggregate of small incentives achieves the big goal. A dysfunctional institution is one where the small incentives do not achieve the big goal.

So, for example, the Allied military 1939-1945 as an institution (consisting of a collection of collection of institutions) was healthy, because they won the Second Imperialist War. The US Military in Iraq was/is dysfunctional.

One feature of institutions is that they reproduce themselves. They are not simply the aggregate of the character of the individuals comprising them; the institution exerts considerable force changing the character of the individuals who participate in the institution. Thus, it is not just that the police recruit only people in whom the police mindset is already fully formed; individual police officers become socialized into the institutional police mindset.

Many social problems, therefore, come from three sources. First, the pattern of organization of an institution may no longer be well-adapted to the long-term goal. For example, capitalism is organized by self-interest, but this self-interest (IMnsHO) is no longer well-adapted to the long-term goal of economic growth.

Second, a poorly-adapted pattern of organization may be imposed on an institution. The most egregious examples are trying to turn creative pattern institutions, such as education and engineering, into bureaucracies. Bureaucracies are well-adapted to well-defined and well-understood goals, but education is poorly-defined, and both try to achieve poorly-understood objectives. For example, we do not yet have a good grasp on what it even means to be educated, much less how to educate people. For another example, even though landing a person on the Moon was a well-defined objective, we did not understand very well precisely how to do so. We cannot break down either of these objectives into precise instructions; at best, we can break the big problem into little problems. How do I educate this particular class of students in this particular subject? How do I keep a human being alive standing on the Moon? One reason I left software development was that the profession was becoming increasingly bureaucratized; I don't have a problem with bureaucracy per se, but software development is not well-adapted to bureaucracy.

Finally, the big goals of an organization may be something we don't like and don't want. The welfare and disability institutions, for example, usually fail to achieve efficiently providing benefits to as many people as possible, but their covert big goal is to actually deny benefits to as many people as possible, which they do in fact achieve. The big goal of capitalism (again, IMnsHO) has changed from providing economic growth to maintaining the power and privileged of the capitalist ruling class. And bureaucracies without external control on their goals simply become sinecures, with policies and procedures growing without rationale, because, why not? The goal becomes just to grow the bureaucracy.

Because we act intentionally to organize society, this theoretical framework helps, I think, to understand the nature and structure of institutions and their role in society.