The limbic and reptilian parts of the human brain have had more time to evolve. Compared to these two parts, the frontal lobe of the human brain where logic resides is a more recent phenomena. Ironically, the logical functions carried out by this part of the brain, which makes man distinct from other animals, are the ones most easily replicated by machines.
The entire human body, not just the deliberate thinking portion of it, should be considered to be a neural network. Using only the thinking portion of the human body for decision making purposes is sub-optimal. This is especially true for a human that has actively engaged in calibrating his body for a specific purpose. Prime examples are deliberate cultivation and heavy reliance on muscle memory by professional athletics, chefs, actors, music composers and detectives.
Intuitive gut feel can be considered muscle memory cultivated over time for specific functions yet expressed as formalized equations. To free up time, individuals can actively convert what they “intuitively know” into formalized equations and have the corresponding functions delegated to machines. Thereafter they could either further compound the effects of this process by building up muscle memories in other domains or sit by the beach and do nothing.
Humans will always have a role available to play in the future regardless of society’s degree of automation.
From Sujit on trading
- not necessary to get numbers further back than six months
- stock market subjected to fractal distribution
- it is possible to generate returns of up to 140% per year by trading on stocks that are moving within a range
- going all in on each position each time leads to a very low Sharpe ratio
- Sharpe ratio should be calculated separately for method and for SnP benchmarked against US treasury interest rates. The difference is the actual returns
On Rambo Last blood
A movie is a reflection of the culture and attitude of an age. Rambo was a very popular cultural icon during the eighties and the early nineties when memories of the Second World War and the Cold War against the communist were still very fresh in the minds of the people in America.
If you looked at the world today through the eyes of someone like Rambo, you would have been able to easily draw facts to back the narrative painted by Trump prior to being elected president.
When operating in an environment of uncertainty, a decision maker formulates multiple often competing narratives in the head that best explains majority of the facts presented. He calibrates the weightage assigned to the probability of each narrative as new pieces of data become available. He simultaneously utilizes multiple ones that are assigned high plausibility in his decision making to strive for the best possible expected outcome . It is a cognitively demanding iterative activity that goes on indefinitely.
- common themes between movie and Trump’s narrative
- Mexico drug cartels
- Mexico prostitution rings
- The world is a dark place
- illegal border crossing
- poor border fence
- white male
- Rust belt
- Protagonist is in his 70s
- Freedom fighter who fought the communist in Vietnam and Russia
- guns and lots of blood
- man of steel
- manifest destiny
- Cognitive biases
- Narrative fallacy
- Framing bias
- selective bias
- Expert political judgement, Philip Tetlock
For us to be able to successfully apply artificial intelligence on any domain, the following needs to be true
- The behavior the system to be modeled must not be stochastic
- The state of the system must be decipherable by the data scientist
- it should be possible to understand the state in which the system is at through interpretation of data gathered
- The domain can be modeled
- the parameters for modeling the domain must be well defined
Only when all three premise are true can we determine where the adjustment should be made when a model fails to predict an outcome
The financial markets is stochastic in the short run.
The underlying parameters are constantly changing and thus hard to model due to the emergent nature of impacts caused by human activities. The data is qualitative and thus hard to convert into clean quantitative datasets.
While the price movements are obvious it is hard, it is hard to attribute impact to the various parameters.
As such, it requires human neural networks that consumed all these qualitative data to perform the prediction/decision making.
If the issue of aging population is an inevitable affliction of all industrialized countries and majority of countries will become industrialized within the next 30 years, then we should be expecting our population to collapse by 2050. Based on this premise rather than being worried that majority of workers will get replaced by Robots and made irrelevant, we should instead be worried that robots are not replacing tasks handled by forthcoming retirees fast enough,
Risk versus uncertainty
- Risk can be mathematically modeled to yield a probability
- uncertainty cannot be mathematically modeled
Conditions for quality data
Why google’s Search data is better than Facebook profile data
- subject feels she has privacy privacy
- subject feels she is not judged
- subject sees tangible benefit from being honest
The hedgehog versus the fox
- The hedgehog approaches reality through a narrative/ideology while the fox thinks in terms of probabilities
- The hedgehog goes very deep in an area while the fox employs multiple different models
- The fox is a better forecaster than the hedgehog
- The fox is more tolerant of uncertainty
- More data does not yield better results and predictions
- Deciding the right kind of data from the abundance available
- To do prediction it is important to start from intuition and to keep model simple
- qualitative data should be weighted and considered
- Be self aware of your own biases
- Similarity scores – clustering in Netflix and baseball
- Be wary of confirmation biases
- Be wary of overfitting using small sample size – Tokyo earthquakes and global warming
- Correlation does not equal causation
- short hand heuristics to reduce the computational space – for example chess
- Irrational exuberance, Robert Shiller
- Expert political judgement, Philip E. Tetlock
- Future shock, Alvin and Heidi Toffler
- Principles of forecasting, J Scott Armstrong
- Predicting the unpredictable, Hough
Conversations with Yi (EverString)
The forthcoming trend for engineering
Machine learning is increasingly becoming commoditized. DevOps becomes more important. Demand for specialized service where DevOps is encapsulated will further increase as demand for engineering tasks further outstrips engineering supplies.
On lead generation market
Companies in the lead generation space have need for scalable web crawlers. This helps offset the cost of retaining three in-house engineers.
Lead generation space has consolidated. There were priorly 120k such companies. There is 7k companies in operation. Majority of players are generating leads by scraping LinkedIn.
Consumer space require constant development of new features. Enterprise space requires service heavy. Enterprise space requires not just lead generation but entire channel marketing service suit (physical mail, online advertising, email marketing)
Lead gen hard to retain. The list becomes less valuable once it’s been used. 80% yearly churn is normal. One company reduces yearly churn to just 10% this by reducing second year subscription from USD800/yr to USD200/yr. further discount to USD100/yr if they don’t like. Recurring service is for grabbing fresh leads from same data source.
On Tele conference
Zoom’s product team compared with UberConference has developed a better understanding of the true conference needs of their users in various context. They have worked harder to ensure their product work seamlessly in identified scenarios. A typical example is the ability to join s conference bybthe press of a button on their mobile phone while driving instead of having to type the typical 4 pin digits.