Brave question, Angus. One danger of the 'broader question' is in losing sight of the interdisciplinary nature of evaluation and entrenched historical trend-analysis/silver bullet approaches. Key cause/effect evaluation points need to be teased out of the data, and you really have to iteratively evaluate these findings using both statistics AND rational thinking (which is where some real interpretive skills are key). I'd be scared of the 'less data' approach, unless in a reporting context, as this inevitably plays into lower costs and then more noise on systems that are inherently noisy. The opposite (too much data) can equally be problematic,of course. I enjoyed the book too, but do agree with some reviewers that Nate Silver is sometimes too 'celebrity-fying' and misses some key aspects of modelling & interpretation. It ALL comes down to good data & pertinent interpretation.Some of the best discussions I've had in this regard are those of interpreting, teasing out, and ranking of proximate and ultimate causation/effect.

Lovely discussion, thank you gentlemen. I will have to read this. We will always need human creativity, and intelligent, thoughtful, considered human assessment of data - hence not automated data. If you enjoy Ted Talks see: Abraham Verghese/ A doctor's touch | Video on TED.com. A parallel universe in the medical industry.

I would agree with you, sometime less (or at least the right amount of data) is better than an avalanche of numbers . Will make an effort to read this book.

Thanks for the thoughts - It is certainly a both-and proposition, rather than a choice between data or human judgement. I am currently working on a project which involves big data for geosciences in which we are taking the view that computers have much more to offer in the 'heavy lifting' of data analysis etc., than we currently utilize; the key to this will be the ability to enable interactive validation and visibility of the processes being undertaken.
Regarding less data, one component is the noise factor: more can be a distraction or completely misleading. Another component relates to quality - error/uncertainty as well as the data's ability to represent mechanistic (over statistical) relationships.
What is the best way to determine data quality and significance? What should we measure more or better? I think there are useful outcomes for better science and industry that will come from trying to answer these things.

Have a look at two papers that were given at the CMP in Ottawa last year both on Geomet. The one given lays out a methodical and "less is more" approach to obtaining good relevant data in a Geomet program. The other also a good paper and I'm sure good data had a huge number of data points which was more of a "sample everything" approach. If the thought is put in ahead of time to select samples that represent the overall trends savings can be made in both the amount of data that needs to be looked at and the costs that are associated with it.

I will try to track them down; I also look forward to looking up the TED stuff  - it will be interesting to see what other science fields are doing in the data analysis area.