Higher participation rates, as well as more timely and accurate responses, are necessary to improve the performance of our economic surveys.

E arlier this month, President Trump fired the head of the Bureau of Labor Statistics (BLS), generating a large response of economists coming out in her support. But the firing raises a larger substantive issue: The reliability of the government’s data has been deteriorating, and the best way to improve it is by using the economic incentives implicit in “data markets” that drive economic statistics. 

Rather than devote taxpayer funds toward beefing up BLS, cutting taxes for those engaging with government surveys would be a more productive way of improving our economic data. A key problem for government surveys in general has been low response rates. Financial incentives for both survey participation and accuracy through taxes would channel the benefits of those using the data to the sample members who supply it. 

In the private sector, if you hire a head of sales, and sales then drastically deteriorate, a natural response would be to seek a replacement. Less accountability for poor performance exists in government, regardless of which party is in charge. Uncertain data quality and large revisions to government statistics, whether increasing or not, are a performance problem that greatly harm the taxpayers and investors who rely on them when making important economic decisions. 

One argument against firing senior BLS officials is that BLS and other government agencies have conducted their surveys in the same fashion over many decades, and it’s therefore not their fault that the statistics can’t be trusted anymore. Instead, the issues with data quality are due to outside forces, such as low response rates and, in particular, delayed responses causing the revisions. Indeed, response rates for establishment surveys have declined over time, from close to full participation historically to minority participation rates recently.  

But poor participation in economic surveys is not beyond the control of those running the surveys, and thus ultimately is the responsibility of BLS, or any other government agency in charge of our economic data. Behind the failure is a more general lack of understanding of how the “data markets” underlying surveys work, something my own research brought up some 30 years ago and is very relevant today in improving the declining performance of governments statistics. 

The performance of surveys is driven by an implicit labor market. The supply side is made up of those sampled. They are expected to work in terms of allocating time and effort to supply their data but, unlike other work, are expected to do so for free. You get what you pay for. If responding to an unpaid survey interferes with paid work, it’s not hard to guess which will get priority. Historical declines in response rates, and future ones if not addressed, are partly due to growing incomes from regular work which increase the opportunity cost of free survey work.   

The government agencies such as BLS and Census Bureau that make up the demand side of this labor market also have poor incentives to deliver high quality surveys. The earnings or employment of their staff is not typically dependent on survey performance, although the recent firing of the BLS commissioner is perhaps an exception.   

Higher participation rates, as well as more timely and accurate responses, are necessary to improve the performance of our economic surveys. But this will come from providing better economic incentives for those sampled, as opposed to staffing up BLS or changing statistical methods. Statistics implicitly assume survey workers value participation, accuracy, and are willing to work for free — but the evidence clearly shows that they are not. And when they avoid the unpaid work, statisticians think there is a free lunch by simply imputing the missing data.   

Improved financial incentives for larger survey participation and better accuracy are both needed and feasible. For better performance of survey respondents, paying more for both participation and accuracy can easily be done, the latter by monitoring a small share of the sample and financially rewarding accuracy. After all, such incentives are used by the IRS to reduce measurement errors for reporting taxable incomes. In addition, making it easier to respond to surveys would improve performance, particularly so for small businesses, which the new chief counsel for advocacy at the Small Business Administration has stressed. 

One natural way for the government to provide better incentives or rewards for sample members would be more favorable tax treatment tied to participation and even accuracy, such as through tax deductibility or credits. Currently, a respondent does not benefit directly from his work; he learns nothing from his response. Instead, benefit mainly accrues to those who learn from the entire data set of which he has contributed a small part. Paying respondents for their contribution to improve our understanding of the economy therefore generates a mutually beneficial exchange, just as wages do for any regular work.   

Better incentives will do more to improve the quality of our economic statistics than changing our statistical techniques. Favorable tax treatment for sample members is one viable tactic. As with so many other things, when it comes to data collection there, is no free lunch. Our economic statistics are in need of better economics.