Lies, Damned Lies, and Statistics

Of the many economic data releases, perhaps the most widely followed are GDP, unemployment, and the CPI. Having worked at Lehman Brothers in Fixed Income Research and in my own time following research reports, I can attest to how important these series are in wall streets analysis of markets. Like any good data analyst, I have never taken the series at face value and have always questioned the methodology behind them. The reason, of course, is that the major weakness of the inherently positivist approach that is econometrics is that the usefulness of the analysis can never surpass the reliability of the data.

So how reliable is the data? Anecdotally, it seems to be quite out of sync with reality. The government reports that "inflation" is running at 3% whereas we see much larger rises in daily living expenses. The government reports that over a million jobs were added in 2007, yet we hear of many layoffs and general expansion pessimism in the media. What gives? Before we take a closer look at the data, let us first examine from a purely economic sense whether we expect the data to be reliable.

Disclaimer

First, a brief disclaimer is in order. As the reader may know, I have some philosophical differences with the Keynesian framework. As part of this article, in addition to discussing the methodology, I also present the Austrian critique of the Keynesian framework. Although I do hope the reader finds the Austrian critique insightful and well thought out, I just wish to point out that one does not need to accept Austrian economics in order to appreciate the rest of the discussion. I add the Austrian critique only for completeness.

What Are The Incentives?

The crux of micro-economics is incentives analysis. As such, what has always fascinated me about government economic reporting is the obvious conflict of interest. Take, for example, the CPI. The government has numerous liabilities indexed to the CPI (Social Security, TIPS, etc). In addition, actual inflation and inflation expectations are both very important in setting lending rates, for various business calculations, and in the consumers evaluation of the economic climate. They are one of the key factors that influence the macro economy. It is no wonder they are one of the key factors in any statistical analysis of macro trends. Yet, it is the government that defines the methodology, collects, and reports the data. I ask you, dear reader, what are the incentives in this situation? Are they not for government economists to understate inflation in the CPI as much as possible? Ex ante should we not expect exactly that? Not only are liabilities decreased, cheaper credit is enabled, and expectations are anchored.

A Closer Look At The Data

Having established our expectations in the matter, a closer look at the data is warranted. I borrow from John Williams of shadowstats who maintains alternate data for all the important series. A detailed analysis of each series is beyond the scope of this article. I will link to more detailed discussions.

1. CPI: is supposed to capture rising prices. It was doing a reasonable job pre-Carter, but thanks to numerous methodology changes, it now consistently understates rises in prices. See the Boskin commission for an introduction. The use of hedonics and substitution is perhaps the most misleading and manipulative practice. I have discussed these elsewhere.
   
   I have also argued that rising prices are not inflation; they are the effect of inflation.
   
   
2. GDP: is supposed to be indicative of economic growth. Unfortunately, it is not the whole picture. GDP is really only indicative of economic output. Economic growth arises from changes to the production structure that cannot be capture by GDP. I have discussed this elsewhere.
   
   Despite being an inaccurate measure of growth, GDP is perhaps the most important economic series. It has many subtleties that John Williams highlights. Pay close attention to "imputations" where the government pretty much makes up new sources of income.
   
   
3. Unemployment: is really two separate series. One is the unemployment rate, and the other is non farm payroll. They are based of two separate surveys. The unemployment rate is based off the household survey and jobs growth is based off the payroll survey. See John Williams analysis here.
   
   The payroll survey has an additional component known as the birth/death model. It is an ARIMA model of small business "births" and "deaths" and their associated job changes as the payroll survey does not include those. Like with any time series analysis, it is always backward looking. The tendency is to project current trends into the future. Thus, at key economic turning points, it will project the previous trend and not predict the change. The BLS freely admits this. What they are not so free about is the model itself. It is a closely guarded secret. Today, with the economy at a clear turning point, the birth/death model has been responsible for some bizarre predictions and overestimations.
   
   The housing survey too is not short of its problems. Most notably the way unemployment is defined is very misleading. For example, "discouraged workers," people willing to work but who have given up looking for a job as they do not expect to find one, are not considered unemployed. This number has been rising recently indicating that the economy may be worse than we suspect, but the unemployment figure dropped 0.1% this month.

Conclusion

The various government reports, although unreliable, should not be confused as inaccurate. The institutions that collect government data, as far as I know, collect accurate numbers. The problem is in how they are presented. Methodological changes result in understatements when desirable (CPI, unemployment) and overstatements when desirable (GDP, jobs). As the famous quote goes "there are three kinds of lies: lies, damned lies, and statistics." The statement "succinctly describes how even accurate statistics can be used to bolster inaccurate arguments." In order for any analysis to be useful, it must account for the quirks in the data.

I remind the reader of the alternate data.