Mark Blumenthal has a long, wonky, but worthwhile post examining the relative strategies and quality of Iowa pollsters. Key finding:
The point here is not so much to guess at which poll has the “right” demographic mix, the right percentage of first-time caucus goers or even about how many Iowans will caucus. Again, knowing with precision who will show up on January 3 is all but impossible. The point is that the Iowa polls have varied widely in terms of the kinds of people they sample. Put another way, no two polls share the same conception of the likely electorate, and some of the differences have been enormous. Yet those of us that consume the data compare results across the various polls as if they all measure the same “likely caucus goers.” They do not.
Put another way, even within the small subgroup of “Iowa polls,” when we compare their results, we’re not just comparing apples and oranges, but apples, oranges, bananas, grapefruits…