Secrets Hidden in Plain Sight

In our contemporary world of Big Data, political correctness and endemic obesity, if you are aware of something counterintuitive, it can be openly discussed and clearly visible without attracting broad attention.  However, the relative security of an overt secret will vary with the times.

We like to conceive of ourselves as rational beings but then how do you explain the herd mentality?  On one hand, we are logical.  On the other, we cast our lots with others to the point of resembling lemmings secure in following the masses off a cliff.


There are physiological drivers of our unhealthy addictions to sugar, gluten and alcohol.  But what of ideas?  Are they embraced due to a self-secreted cocktail of happiness chemicals: endorphins, dopamine, serotonin and oxytocin?  Simon Sinek suggests as much in his recent book Leaders Eat Last.  I agree that what first survived and then evolved into Homo sapiens is relationship-driven.  This suggests and perhaps explains why people put complex interrelationships ahead of their own rationality and, hence, our fear of failing to be politically correct in our every utterance.

Using that as a working hypothesis, it is easy to believe that people, in an attempt to remain accepted and acceptable, will strive to ignore any counterintuitive notion until after it has become both broadly embraced and common knowledge.  So then, how does a new idea ever get to be commonly accepted?  Indeed!

There is an extraordinary amount of new data being created every day.  Just recently, I read that IBM estimates that half of the data in existence today was created within the last two years.  However, I do not notice us, as a race, becoming wiser at the same rate.  Therefore, the volume of new data is obscuring the truth, not elucidating it.

I was fascinated by the chart that Nate Silver used in the conclusion of his book, The Signal and the Noise.  I have reproduced it below:

I’ll let him explain it. 

“Our views on how predictable the world is have waxed and waned over the years.  One simple measure of it is the number of times the words ‘predictable’ and ‘unpredictable’ are used in academic journals.  [The data shown in figure C-2 is based on searches conducted of the JSTOR catalog of print journals.  I searched for cases in which the word ‘predictable’ or ‘unpredictable’ appeared in the journal at least once (but not both words in the same article), breaking down the results by decade of publication.]”

The previous century started with a balance, let’s call it uncertainty, between the two positions.  The Great Depression weakened not only the world economy but also the confidence of academics (and presumably people at large) to a greater and greater extent, peaking during World War II at almost two mentions of “unpredictable” to every “predictable.” The financial boom of the 1950s, innovations around the moon race of the ‘60s and at least imaginings, if not an expectation, of a Star-Trek-like future in the early ‘70s spurred confidence that the future is knowable.  Exclusive usages of “predictable” soared, doubling in 30 years. 

Although the degree to which our sense that the world is more predictable than unpredictable has fallen over the subsequent 40 years, it remains almost exactly opposite of what it was in pessimistic 1945.  As you can see from the chart, confidence in predictability remains double that of our willingness to admit to chaos. 

I have spoken about and written that the climate is not yet ripe for broad acceptance of the Theory of Constraints.  Despite a steadily creeping fear that, just maybe, those things we seek to forecast using the modern sea of available data are not, in fact, forecastable, the herd continues to demonstrate by its expenditures that it is firmly entrenched in the camp of expensive forecasting and advance planning software. 

If we further consider that the understanding of the masses trails that of academia, which trails pioneers in the practical world, we are a long way from global acceptance of and significant demand for TOC.  Hence, TOC is a secret to be exploited and not yet something to be pushed on others (except those one has authority over and who need to know), probably for the next several generations to come.