Why did the sub-prime crisis happen?

It was a subject of discussion in the office yesterday, as we tried to come up with plausible reasons why banks and other lenders were so eager to lend money to borrowers who could not reasonably pay back to the loans. We came up with a very short list of "a) sheer idiocy" and "b) some form of government policy". Apparently option "b" is correct:

Consider the low lending standards that were a significant component of the mortgage crisis. Lenders made millions of loans to borrowers who, under normal market conditions, weren't able to pay them off. These decisions have cost lenders, especially leading financial institutions, tens of billions of dollars.

It is popular to take low lending standards as proof that the free market has failed, that the system that is supposed to reward productive behavior and punish unproductive behavior has failed to do so. Yet this claim ignores that for years irrational lending standards have been forced on lenders by the federal Community Reinvestment Act (CRA) and rewarded (at taxpayers' expense) by multiple government bodies.

The CRA forces banks to make loans in poor communities, loans that banks may otherwise reject as financially unsound. Under the CRA, banks must convince a set of bureaucracies that they are not engaging in discrimination, a charge that the act encourages any CRA-recognized community group to bring forward. Otherwise, any merger or expansion the banks attempt will likely be denied. But what counts as discrimination?

According to one enforcement agency, "discrimination exists when a lender's underwriting policies contain arbitrary or outdated criteria that effectively disqualify many urban or lower-income minority applicants." Note that these "arbitrary or outdated criteria" include most of the essentials of responsible lending: income level, income verification, credit history and savings history — the very factors lenders are now being criticized for ignoring.

The whole article is here. H/T to Brian Micklethwait (who was kind enough to link to an earlier QotD).