Yesterday, numerous news outlets – including the Wall Street Journal and Reuters – published stories about a startling new study claiming that some 30% of employers plan to stop offering health insurance to workers as a result of the Affordable Care Act. There’s no doubt that this figure, derived form a study conducted by McKinsey consultants is noteworthy. Previous other reputable studies have said far fewer businesses would drop coverage, as the White House was eager to point out in the wake of news coverage stemming from the McKinsey study.
Rather than just report what was contained in the McKinsey study, however, I wanted to know more about the study itself. So I called up McKinsey today to ask about the methodology behind the employer survey. A McKinsey Quarterly article about the study said the results were based on a survey of 1,300 employers varying in size from variety of industries and regions. What the article didn’t say was anything else about how the study was conducted. Here’s what we would need to know to evaluate the integrity of the McKinsey study and make any judgments about the ACA based on it:
* What were the precise breakdowns of size, geographic location and industry for the businesses included in the survey? This would tell us if the sample was representative of American business as a whole. Small businesses, for instance, might be more likely to drop coverage due to the structure of the ACA.
* How were the businesses chosen? An unbiased sampling method here is key. If the list of businesses was culled from Chamber of Commerce memebrship or McKinsey client lists, this is important to know. Ditto if the list was generated in a more randomized way.
* What was the response rate? And how were businesses surveyed? If 13,000 businesses were contacted, but only 1,300 responded, such a 10% response rate could call into question the results. Also, there is, for example, a huge difference between surveys conducted in person, over the phone and over the Internet.
* Lastly, this tidbit was included in the McKinsey Quarterly article about the survey:
“…our survey educated respondents about [employer sponsored insurance] implications for their companies and employees before they were asked about post-2014 strategies.”
In other words, those conducting the survey may have primed respondents to say they would keep or drop coverage.
I asked McKinsey if a third party paid for the survey and they said no. On my other questions – about the methodology and what script was used to “educate respondents” – they declined to comment.
