Big Data Quality, Then and Now

A decade ago, just before the beginning of the data science hype cycle was the big data hype cycle. At that time I had the privilege of sitting down with Ph.D. Statistician Dr. Thomas C. Redman (aka the “Data Doc”). 

We discussed whether data quality matters less in larger data sets, if statistical outliers represent business insights or data quality issues, statistical sampling errors versus measurement calibration errors, mistaking signal for noise (i.e., good data for bad data), and whether or not the principles and practices of true “data scientists” will truly be embraced by an organization’s business leaders.

This episode is an edited and slightly shortened version of that discussion, which even though it is from ten years ago, I think it still provides good insight into big data quality, then and now.

 

Extended Show Notes: ocdqblog.com/dbp

 

Follow Jim Harris on Twitter: @ocdqblog

 

Email Jim Harris: ocdqblog.com/contact

 

Other ways to listen: bit.ly/listen-dbp