Risk versus uncertainty
- Risk can be mathematically modeled to yield a probability
- uncertainty cannot be mathematically modeled
Conditions for quality data
Why google’s Search data is better than Facebook profile data
- subject feels she has privacy privacy
- subject feels she is not judged
- subject sees tangible benefit from being honest
The hedgehog versus the fox
- The hedgehog approaches reality through a narrative/ideology while the fox thinks in terms of probabilities
- The hedgehog goes very deep in an area while the fox employs multiple different models
- The fox is a better forecaster than the hedgehog
- The fox is more tolerant of uncertainty
Big data
- More data does not yield better results and predictions
- Deciding the right kind of data from the abundance available
- To do prediction it is important to start from intuition and to keep model simple
- qualitative data should be weighted and considered
- Be self aware of your own biases
Prediction
- Similarity scores – clustering in Netflix and baseball
- Be wary of confirmation biases
- Be wary of overfitting using small sample size – Tokyo earthquakes and global warming
- Correlation does not equal causation
- short hand heuristics to reduce the computational space – for example chess
Related references
- Irrational exuberance, Robert Shiller
- Expert political judgement, Philip E. Tetlock
- Future shock, Alvin and Heidi Toffler
- Principles of forecasting, J Scott Armstrong
- Predicting the unpredictable, Hough