Pushing Statistical Derp

It turns out that the entire October jobs report is derp, and needs to be thrown in the trash. Yesterday, I wrote about how the job numbers were very skewed in statistical analysis. According to this story on the AP however, it seems that the Government Statisticians have violated the cardinal rule of statistics and methodology: pay attention to the sample size.

What is derp? Well, Economist Noah Smith has the most perfect definition of it over on his site, using Bayesian Theory. Check it out. But I’ll just shorten it up and say that derp the assertion of an idea when evidence says you should update your ideas.

And Bayesian theory is perfect to describe the Statistician in the government as pushing derp. Priors, the sample survey response rate on unemployment was 76.4%. This month, it was 83.5%. If you’re already face palming, it gets better. The 83.5% of survey respondents reported for an EXTRA week, because of the government shut down.

Methodologically, it goes like this, and yes I’m over-simplifying it, to make a point:

Let’s say that I’ve always sent a survey out to 1000 people, and that I’ve always gotten 764 responses. I’ve done this for month after month, and on average, it’s more or less the same.

Now let’s say that because I was furloughed for a couple of weeks, I send the surveys out late to 1000 people. And, because of this difference, including lag time, I get 835 back. Now I’m comparing the results of 764 respondents of the past, with 835 respondents of the present. And, because I ALWAYS asked them about labor life for the past 4 weeks, they include the four weeks, PLUS the time that they were expecting the survey, but didn’t get it – because I was on furlough.

Baynesian theory is about modifying your priors based on new statistical evidence. This is exactly what the BLS did NOT do. Comparative (longitudinal) surveys have to be statistically based on the same sample size, and if time dependent, on the same time frame, or else they are complete garbage.

So, while I’m sure the Government Statisticians at the BLS went to very respected schools, and have very respectable Master’s and Ph.D.s, they seem to have forgotten the rules of statistics and, infinitely more important, the rules of methodology.

Unfortunately, we will not get a “real” jobs report to see what the social facts of labor are, until December. All the anomalies won’t really shake themselves out of the analysis until January.

Advertisements
This entry was posted in Public Policy, Sociology. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s