Recently, a couple of comments (here and here) prompt me to talk about method.
I take Feynman very seriously: "The first principle [of scientific integrity] is that you must not fool yourself--and you are the easiest person to fool. So you have to be very careful about that. After you've not fooled yourself, it's easy not to fool other scientists. You just have to be honest in a conventional way after that." I try, indeed I try very hard, to apply this principle in my criticism both of others and of myself.
Feynman's assertion raises two questions. First, what does it mean to fool oneself? Second, can we distinguish between fooling oneself and not fooling oneself, and if so, how?
Feynman makes it clear that fooling oneself is a matter of method. In his view, when someone distorts an argument to support a preconceived idea, he is fooling himself. Note that Feynman does not say that having or investigating a preconceived idea is by itself fooling oneself; fooling occurs when the investigator allows his preconceived idea to distort the argument. So what does it mean to "distort" an argument? Feynman uses subsequent investigations of Millikan's electron charge experiments as an example: the investigators distorted the data to get values closer to Millikan's answer, i.e. their preconceived idea of what the answer "should" be.
If fooling oneself is a matter of method, then it should be possible to tell the difference between fooling oneself and not fooling oneself by looking at the method. Science has a lot of interesting procedures to avoid fooling oneself. The most obvious is the double blind method: neither the subject nor the person collecting the data knows whether the person is being treated or is acting as a control. Since neither the subject nor the investigator knows what the answer "should" be (the preconceived answer is that there is a difference between the measurements of treated and untreated (control) subjects), they cannot bias the measurements. The double blind method seems like an effective technique for removing one kind of fooling oneself.
It is easier to fool oneself in philosophy and non-experimental argumentation, simply because there isn't an easy way like the double-blind technique to remove bias. But there are other ways. Does the author employ or avoid well-understood logical fallacies? (In my more cynical moods, I suspect that the primary project of academic philosophy is to teach students to write so turgidly as to prevent the detection of fallacies.) Does the author critically examine opposing viewpoints? Does she try to represent those opposing viewpoints fairly and honestly? These methods are not as tight and effective as double-blind testing, but they can, in my opinion, do a lot of work detecting and correcting fooling oneself.
Which brings me to the "criticism" of my own work cited above. First, I am always looking to see if and how other thinkers, especially thinkers with whom I disagree, fool themselves or avoid fooling themselves. Which means I am looking not at their conclusions, but at their methods. For example, when I examine Plantinga's Modal Argument, I look not at his conclusion that God exists, but at his method: I argue that he has made a methodological error, a logical fallacy. The question is not whether I am "motivated" to investigate his argument because I disagree with his conclusion, but rather whether my motivation has distorted my criticism, distorted my own argument.
A lot of the criticism of my work in the comments tends to come in three categories. First, people who just insult me. I have enough self esteem that insults from random people on the internet do not cause me the least distress. The only negative effect of this sort of criticism is irritation that I have to waste my time reading and possibly moderating a useless comment. Second, people who just assert their disagreement with my conclusion, which is again a pointless waste of my time (and if the commenter intends to affect my views in any way, a waste of his or her time). I really don't care that people disagree with me. It would be a waste of my time to write about what everyone agrees with; indeed, the more controversial a topic, the more interesting.
What I really do want to know is: am I fooling myself? If you do not believe I care whether or not I'm fooling myself, why even bother to comment? I honestly don't understand it. No one but me reads the comments, especially in older threads. Why even bother to register your dissent? If I want to fool myself, or I don't care about fooling myself and others, and registering dissent actually mattered, I would just delete the comments, which I can do without detection. (And you don't know I haven't, eh? But I leave them all in, unless they simply repeat an earlier point and I want to make clear that I am unwilling to waste any more of my time reading and moderating a commenter.)
I will give nontrivial attention only to criticism that addresses my method, i.e. how I construct my argument. Am I making a logical fallacy? Have I failed to address an important opposing viewpoint or argument? Do I make unexamined or unsupported assumptions? (Note too that this is a blog, not an academic journal; many posts here represent preliminary speculation, not fully-formed arguments.)
By the way, it's really important to cite and summarize opposing views and offer a basic analysis of how the opposing view affects my own argument. If you don't cite, I have no idea what you're talking about; even if I Google a vague referece, for example Edward Feser, I don't know whether what I find is what the commenter is referring to. If you don't summarize or analyze, I have to substitute my own judgment of the opposing viewpoint for my critic's, which just introduces bias. I have a limited amount of time, and if the work of someone such as Feser superficially appears to be worthless (and his work does superficially appear to be worthless), I'm not going to waste my time without some evidence that a deeper analysis is worthwhile.
I'm open to criticism; I really do believe that I can possibly be fooling myself, that I can possibly be ignorant of really good arguments that contradict my position (which is why, for example, as a communist I study capitalist economics). I don't even mind rudeness per se, but if you are rude, especially without direct provocation, your support should probably be stronger than otherwise. But if you're not interested in helping me figure out how I'm fooling myself, don't bother commenting. I take Heinlein to heart: "Never wrestle with a pig. You both get dirty and the pig likes it."