In general laws and regulations work only when the prescribed behavior is in most everyone's mutual self interest. When everyone is better off doing something than they would if nobody did it, laws compelling people to do it will work. For example, because everyone is better off respecting ordinary personal property rights than they would be if nobody respected property rights, laws against theft will work in the long term. (Laws and regulations are necessary when, if most everyone is better off doing something, a few people can get an even larger advantage by not doing it.) In other words, laws and regulations are effective and necessary when the underlying situation is a Prisoner's Dilemma.
The finance industry would seem to be a classic Prisoner's Dilemma. It seems that everyone would be better off with a financial system where everyone is reasonably honest, prudent and realistic about things like the value of assets, the risk of transactions than if nobody were honest. From 1940 to the mid-1970s, the finance industry was heavily regulated, and the regulations did not become almost completely ineffective until the 1990s. (It's not the only or even the primary cause of the crisis, but the 1999 repeal of the provisions of the Glass-Steagall Act (thanks, President Clinton!) separating commercial from investment banking is a notable event.) While heavily regulated, finance and banking were consistently profitable, and for six decades we avoided an internal collapse of the banking and finance system, and weathered several external shocks (notably the 1982 Latin American debt crisis and the 1997 Asian financial crisis). The worst internal crisis of the financial system was the Savings and Loan crisis of the 1980s-1990s (caused directly by regulatory failure), but the crisis was not severe enough to threaten a 2009-like collapse of the financial system.
Even though the mutual advantage of financial regulation seems clearly apparent, and the mutual disadvantage of deregulation equally apparent, the finance industry and capitalist ruling class lobbied for decades to weaken, remove and undermine financial regulation. And, when they were successful, they almost immediately drove the global financial system into catastrophic failure. And even now, in the face of catastrophic failure, the consensus in the entire political infrastructure (i.e. both the Democratic and Republican parties) that represent the capitalist ruling class is against tough financial regulation.
Why? It's not like the bankers have no idea of mutual self-interest. They haven't been lobbying for the right to use violence as a means of competition. Bank of America knows that the "selfish" power to take over Mom & Pop corner bank by force of arms is not worth losing the mutual benefit of never having to fight an all-out war with Citigroup. But it's clear that the capitalist ruling class does not consider financial regulation to be in its mutual self-interest.
To a certain extent, there's a problem with international banking. The US government can regulate only American banks, and regulations establishing mutual interests work only when everyone is regulated; if a few can exploit others' cooperation without penalty, mutual cooperation just becomes mutual being-a-sucker, and it's in nobody's interest to cooperate. But the entire West had highly regulated banking, and there were effective mechanisms in place to punish countries that didn't effectively regulate their banks. The effort to overturn and undermine financial regulate was an international effort. (Indeed, the capitalist ruling class has embraced internationalism far more thoroughly and effectively than the working class.)
If effective financial regulation that demonstrably affords stability, consistent economic growth and capitalist privilege is not seen as in the mutual interests of the members of the capitalist ruling class, we must examine more closely what they conceive their own interests to be. We must ask: what is the political character of the capitalist ruling class?
A football game is a zero-sum game. One team wins, one team loses. It doesn't matter how well or how "honorably" you played, winning is winning and losing is losing.
When a receiver beats a cornerback in the end zone, and has a high probability of catching the game-winning touchdown pass, the cornerback has an obvious self-interest in shoving the receiver so he cannot catch the pass. In a direct, immediate sense, the cornerback won't do so because if he did, he would be almost certainly face a pass interference penalty: the immediate drawback of the penalty outweighs the advantage of preventing the catch. The question, however, is why does the cornerback consent to the existence of the penalty in the first place?
To some extent, it's in the teams mutual interest to disallow pass interference: if one team's cornerback can push the opposing team's receiver, then the opposing team's cornerback can push the one team's receiver. If we look at Rawls' "veil of ignorance", before the game neither team knows on whose cornerback and receiver the game will depend. But that's only part of the story.
Before the game, it's possible that the coach knows the other team's receivers are generally more competent than his own cornerbacks, and the other team's cornerbacks are more competent than his own receivers. Even with the "veil of ignorance", the coach knows that generally, the pass interference penalty will hurt his own team more than the opposition; he would have a higher probability of winning the game if the penalty were not enforced. Since football is a zero-sum game, there's no scope for an evaluation of mutual self-interest; there's no value (at least in the immediate sense) to "playing well", only winning (even "winning ugly") or losing.
It's true in general that from the coach's immediate perspective, there is no such thing as mutual benefit in any sense. All rules and regulations impose a pure loss on the coach, by requiring him to make extra effort to not just win in the most efficient manner possible, but to win according to the rules. If pass interference is allowed, then he doesn't need to spend resources training his cornerbacks to protect against the pass without interfering with the receiver.
Rules and regulations in football work not because they are in the mutual interest of the coaches (who are typically charged with actually winning games) but because they are in the mutual interest of the owners. The owners are less interested in winning games per se; they are more interested in making a profit by entertaining the audience. The owners of the St. Louis Rams are not particularly pleased by losing fifteen out of sixteen games in 2009-2010, but they will still make more profit because the rules of football make games more entertaining than if there were no rules at all: no one is going to pay (at least not for long) to watch a mob of 22 enormous guys in a free-for-all, even if the Rams' players could consistently win such a fight. In other words, the owner is better off losing an entertaining game with rules than winning a boring game without rules.
This mutual interest would still hold even without the artificial separation between coaches and owners interest (or even between the coaches and players). If all the coaches owned their own teams, their own interest would still be to entertain the fans, and they would be better off banding together, creating mutually beneficial rules and punishing teams that broke those rules. The mutual interest permeates the entire structure: even though they lost most of their games, the players still earn a nice salary and have the opportunity to improve or be traded to better teams, the coach will get some good draft picks this year, the owner still made buckets of money.
But imagine if everything else were the same, but at the end of the season, the team — coach to water-boy, but not the owners — that won the Superbowl received absolute power for a year, and everyone on the losingest team in the NFL were literally taken out and shot. People would still play football, but suddenly, the mutual interest of an entertaining system of professional football would be outweighed by the individuals' interest gaining absolute power and in not dying, even though an entertaining system of football would remain in the owners' interest. We would then expect that the coaches, players and staff would do everything in their power to overturn and undermine all the rules of football. Every rule affects some people more than others, and the mutual interest of entertainment is of no value to a dead man.
The last paragraph describes how the capitalist ruling class sees the world. Win, and you win nearly absolute power for life; lose, and you're taken out and shot. Stuck in the middle and you have the chance of rising to the top but you're terrified of falling to the bottom. Losing means not just not winning, but facing the prospect of being ejected from the capitalist ruling class, a fate — in the minds of the ruling class — literally worse than death.
The capitalist ruling class is not directly interested in creating and maintaining a robust, stable, well-functioning economy. At best, a functioning economy is a means to an end, but if they can use other means to gain personal power and avoid death, they will use all their considerable cleverness and power to actually employ those means. The stakes are just too high — in their minds — to relax and seek mutual benefit. No mutual economic benefit is worth giving up the opportunity for nearly absolute power nor worth risking death.
In much the same sense, monarchs, royalty and the nobility during feudalism did not have a direct interest in international peace and a well-functioning political state with a contented and prosperous population. Political stability was sometimes in their indirect interest, but overall, the lure of absolute power and the risk of ejection from the feudal ruling class meant that if they could keep or enhance their power through political instability, brute-force oppression or war, they would do so. No attempt at mere "regulation" was ever successful in the long term; the propensity for the feudal ruling class to pursue their own interests at the expense of the people's was not substantially affected until the feudal ruling class was simply eliminated and political power socialized (to some extent) in capitalist pseudo-democracy.
In a similar sense, finance regulation cannot work in the long term unless and until the ownership of finance capital is socialized, until the capitalist ruling class — built on the private ownership of finance capital — is eliminated and financial power socialized in democratic communism.