In the science fiction story, "And Then There Were None," Eric Frank Russell depicts a coherent anarchist society. Two key features of the "Gands" (from Gandhi), the inhabitants of Russell's anarchist planet, are first, their absolute resistance to any form of coercion, and second, their lack of any absentee ownership. Several scenes and expositions illustrate these concepts. Anyone may take over abandoned property; the original owner, Russell says, will have left it for something equally free. In the story of Idle Jack (a parable of how the Gands deal with free riders), when Jack is driven to steal because his neighbors no longer will give him social credit, he faces no after-the-fact punishment for his theft; people are simply more vigilant about protecting the physical property in their possession. (When he has finally exhausted all social credit, Idle Jack finally kills himself.) The Gands take the prohibition against the initiation of coercion seriously: it is always illegitimate there to be the first person to use physical force; there are no socially constructed exceptions whatsoever.
The Gands do not use money; indeed, they have contempt for the concept. Instead, they have a system of "obs" (short for obligations), a form a social credit. If you do something for me, then you plant an ob on me; to kill the ob, I must do something for you, or do something for someone else you've planted an ob on. One could, I suppose, write down obs, but because obs are not coercively enforced by the community, there's no legal reason to do so; obs, therefore, are usually simply remembered. Communities are, therefore, limited to a size where the community can keep track of everyone's obs. Russell seems to anticipate, by a number of decades, the modern understanding that intra-community trade was historically founded on social credit.
The Gands fundamentally do not ever legitimatize coercion. Even direct self-defense against immediate physical attempt at coercion, while legitimate (as in the para, is still somewhat disreputable. Instead, the Gands prefer passive resistance. When the Terrans, who have visited the world of the Gands after centuries of neglect, attempt to coerce the Gands, the Gands do not fight back; they simply refuse to be coerced. When the Terrans, for example, attempt to abduct some of the passengers from a passing bus, The Terrans find the passengers have chained themselves to the bus. When one of the Terran characters attempt to coerce a Gand at gunpoint, the Gand sensibly points out that if he were dead, he would be of no use to the Terrans. As Heinlein says, you can never enslave free persons; you can only kill them.
There are some problems and limitations to this notion of anarchism, but on the whole it is coherent, and it is difficult to frame a moral objection to the society. But without this absolutist attitude towards the initiation of coercion, and the concomitant impossibility of legitimatizing absentee ownership, Libertarianism is just a thinly-veiled rationalization for the worst sort of tyrannical oligarchy. The contempt that many on the left have for Libertarians is not a desire to dominate others, but a reaction to hypocrisy.
"Government" is a just label given to that social institution that has a monopoly on the legitimate physical initiation of coercion. It does not matter what organization calls itself the government; the real government consists of the institution, i.e. not only the organizations but also the social and cultural constructs, that legitimatizes the initiation of coercion. When we have abstract property and absentee ownership, physical possession no longer defines ownership. Ownership, rather, becomes a matter of social agreement, and we must, therefore, socially construct what it means to make an agreement. We must construct what agreements actually mean (and even today, resolving disputes about what agreements actually mean occupies the time and effort of a million lawyers), when an agreement is legitimate or illegitimate, and what the consequences are when the agreement is breached.
Libertarianism entails government, a social institution that legitimatizes the initiation of coercion. Libertarianism is just disingenuous about labeling: what they label as "government" is just one part, the police, of the institutional structure. However, the owners of property, because they are privileged to define the terms of tenancy, play the major role in legitimatizing the initiation of coercion; therefore, they are the government. Furthermore, by absolutely privileging absentee ownership, Libertarianism would create a government more tyrannical than most. It is a government of the owners of property, by the owners of property, and for the owners of property; tenants have no rights whatsoever of their own.
If you want to argue for an anarchy, argue for an anarchy: Argue that the initiation of coercion in the strict physical sense is always illegitimate, and no institution can legitimatize it. Or, if you want to argue for an oligarchy, then argue for an oligarchy. Argue that those who are able to accumulate abstract property and absentee ownership deserve to rule or perhaps deserve mastery, and those who are not able to accumulate abstract property deserve slavery. But please do not call the the initiation of coercion always illegitimate on one hand and then legitimatize it on the other. It is an offense against both tyranny and liberty to try to cover an oligarchy in the cloth of freedom.