A bottle neck was observed this evening with the data pipeline for trends.getdata.io.
Gradual addition of more new data sources eventually lead to more jobs being added to the queue than could be processed per period time.
Phenomena of a similar nature was observed with getdata.io during the beginning of the year when our user base started ramping up.
The higher order principle is that bottlenecks are inherent within systems. They are emergent by nature. Situation will deteriorate exponentially once threshold is crossed.
The same principle applies with financial systems, flight systems and boat sailing.