Would the state, as Marx predicted, really wither away as socialism evolves into communism?
We have to be more precise about what we mean by "the state". There are two distinct senses. First, the state can just mean the government: a well-defined, bounded organization that holds and exercises a monopoly on the use of force. The second sense is the government acting to implement a class dictatorship, acting forcibly to secure the interests of some class of society.
Under socialism (in the sense of the political-economic system that acts as the transition between capitalism and communism) there will still be a bourgeoisie, a class of people who own capital; it does not seem feasible to instantaneously place the ownership of all the means of production directly under the direct control of the government. Therefore, the government will have to act to secure the interests of the people who work for a living, in opposition to interests of the owners of capital. In this sense, a socialist government will be a "dictatorship of the proletariat" as opposed to the current system where the government acts forcibly to secure the interests of the owners of capital and is thus a "dictatorship of the bourgeoisie".
(Technically the "state" refers not just to the government, but also the political superstructure, the social and psychological social relations that underlie and support the government. The commercial news media, for example, are part of the larger state, because the content and framing of the news reinforces and supports the role of government to secure the interests of the owners of capital. The organized mainstream political parties are also part of the larger state, as are the bourgeoisie themselves who finance the campaigns of the major candidates.)
The goal of communism is to erase class distinctions. If there are no classes, then there are no class interests for the government to secure, so the function of government to secure some class interests disappears.
But I think Marx wants to make a stronger claim: that there is no government under communism; that individual people under rational and mutually beneficial economic relations will not need a well-defined organization that has a monopoly on the use of force. The only need for force will be against people who are literally insane, and individuals or small groups can easily handle these few unfortunate individuals.
I think this claim is unrealistic, even under ideal conditions.
The Prisoner's Dilemma shows that some mutually beneficial outcomes require the mutual decision of both (or all) parties to forcibly bind themselves to an outcome. It's possible to optimize the outcome of iterated Prisoner's Dilemma games using the Tit for Tat strategy. However, relying exclusively on this strategy requires a near-universal individual adoption of deep social constructs. Furthermore, the Tit for Tat strategy limits the scope of economic decision-making to time scales sufficiently small that the number of times a game is iterated becomes hard to predict.
The latest financial crisis illustrates the limitations of Tit for Tat. Greenspan's theory of financial deregulation essentially assumes that the threat of Tit for Tat will adequately prevent one financial institution from betraying another by refusing to pay its debts. However, individual people within those institutions created complex financial instruments that delayed the decision to cooperate or betray (pay or default on debts) past the careers of those creating, selling and buying the instruments. It's easy to sell — even buy — a financial instrument you know is bad when the short-term consequences of that decision appear beneficial and the long-term consequences will fall on another's head.
Industrial production in Marx's time was much simpler than it is today. The most complicated industrial endeavor in the 19th century was, as far as I know, ship-building or railroad-building, endeavors that employed only thousands of people and operated on time-scales of only years. Today we manufacture aircraft, computers, and millions of automobiles, that employ tens or hundreds of thousands of people (when you count all the suppliers of all the components) and operate over time scales of decades (especially counting continuing research and development). A ship-building factory could be created using the financial capital of a handful of people; a factory to produce one kind of computer chip requires, or so I'm told, a billion dollars, or about 200,000,000 hours (assuming $5/hour) of socially necessary labor time. The Windows Vista operating system cost ten billion dollars to develop.
Once we start going into space to exploit the massive resources of the solar system, the problem will get even worse. Ventures will initially take decades to pay off the initial investments, and it will take perhaps a century or longer until the exploitation of space is economically self-sufficient.
These large-scale economic endeavors presently require and will continue to require a degree of centralized economic planning by people with considerable specialized expertise and the individual incentive to make decisions that are beneficial on time scales longer than their individual careers or even lives. It doesn't seem feasible to counting on purely individual decision making, even if that decision-making were ideally socialized.
(It's important for those who profess abhorrence to centralized economic planning to understand that this centralized planning is already happening. The capitalist class is even now engaging in high-level macroeconomic planning among themselves and relying on government power and economic participation to enforce compliance with this planning. The goal of socialism and communism is not to implement centralized economic planning, but to make the centralized economic planning directly responsible to the needs of all the people, not just the capitalist class.)
If there were a revolution, then, we would have to make a momentous decision: either restrict the scope of our technological and economic endeavors to a mid-20th-century level that could indeed rely on non-state mechanisms such as Tit for Tat to regulate individual behavior and anticipate the withering away of the state in the stronger sense. Alternatively we would have to permanently (at least for a century or two, and no one can make predictions on that time scale) institutionalize state economic planning to manage large-scale technological and economic endeavors.
Being a technophile, I would prefer the latter. But the institutionalization of state economic planning automatically creates a ruling class, those who do in fact perform the planning, and a ruled class of everyone else. Which means that class struggle itself must be permanently institutionalized. Which is a decidedly non-trivial — perhaps unsolvable — problem in political science.