If you want to know where you’re going, look at what’s happening now.

Nova Spivack of Bottlenose continued Thursday’s presentations with his talk about “predicting the present” and spoke about technologies to impact CI; he also reflected upon previous conversations as many issues are timely, relevant and important for many attendees.


Three phases of history

  • Age of agriculture: Remember the past. Oral traditions were key; tradition was valued.
  • Age of Industry: Predict the future. Major shift due to need to plan long-term projects like the railroad infrastructure. Focus on progress, innovation, and logistics.
  • Age of Information: See the present. Information moves quickly, has a short shelf life, and we have moved into a present-oriented civilization; “now centric.” This will change everything about how we think and work, and we will decide what to do in the present based on the present—we are on the cusp of this evolution; this could take time to evolve.

If you want to know where you’re going, look at what’s happening now, said Spivack.

It is harder to get reliable information about the present on which to base intelligence; unstructured data like emails contribute to an unprecedented explosion of information. Unstructured data is hard for machines to understand; data fields are not easily discerned and new forms of understanding information are required.

The growth of data scientist analysts grow at the rate of 1/6th as what is required. Increasing fragmentation (social media, for example) is a challenge in keeping up with demand.

By 2018 the US will face a shortage of up to 190,000 data scientists with enough proficiency in statistics to use big data effectively.



If information that needs to be processed grows faster than available processing power, demand grows at current rate, and supply of resources is lacking—how do we bridge this unavoidable gap? There will be a moment when “you just can’t cope any more” with the flood of data and sources. Where is consistency in tools? Some can be helpful, but the situation is difficult: Getting the news is no longer a disciplined process.

In the event artificial intelligence (AI) “gets smart,’ it won’t happen quickly enough. AI also cannot consider all variables and the complexity of given situations. Currently there is a shift from network systems to neural systems; but human intelligence is not only neural. There is more computation occurring in the human brain than what we realize and appreciate.
Complexity does not equal intelligence; consider the length of time it takes for humans to grow and mature.

The problem is unavoidable; and AI will not resolve the issue of information floods and shortages of resources to manage information. We will need to iterate our strategies regularly.


How to handle the problem and navigate the course?

  • We need to assume our map is changing…because it is.
  • We need a system like a GPS to measure our environment and reroute us.
  • The wisdom of crowds can be helpful.

To operate in the present, real time data mining will help based on existing models and theories. Humans are still required to make long range projections—machines can report on consumer preferences and attitudes to help ‘read the tea leaves’ such that humans more accurately make predictions.

Indicators are helpful, but for actual decision making, it is important to consider “what is the present?” For a person, the present is in nanoseconds. We sense things, process them, and reach conscious awareness so we make decisions.

For companies, the present is traditionally a “larger thing” as building models and generating response has inertia—it takes time. Previously, “moments” were measured monthly or quarterly; this time is becoming compressed and is now more commonly measured in hours as social media influences business practice and may require fast response (crisis management, need for tactical decisions, etc.)

Real time measurement is required: Where is all conversation happening and what is the influence of each party discussing a situation? Sometimes, waiting and doing nothing is better than taking action depending upon whether or not influencers are engaging in conversations. Such decisions are measurement decisions, and not “gut” decisions.


Measurement techniques

Consider the wisdom of crowds. What are a large number of people saying? Election predictions, for instance, serve as an example. Certain lay people are better than experts at analyzing and predicting the Superforecasters have an open mind, are alert to nuance, seek new information, and adjust earlier conclusions with new data.

Google is an example of a superforecasting technology—it measures “who links to who” and develops rankings. Quality pages are ranked and costs of ads determined. Google trends analyzes search queries to make predictions about what is important in the world.

Bing predicts takes mass data and examines it against what they want to predict; it does a better job than Google.
Biases and problems in algorithms are important factors to keep in mind; people can change bias, machines cannot.

Prediction is moving toward anticipation. “X is going to have value Y at time T.”

Anticipation is moving toward “X is trending to value Y right now.”

We speak in terms of anticipation, not prediction. Anticipation focuses on our present environment.

Why is real-time anticipation important?

Consider the example of self-driving cars. Real time information like the weather and traffic conditions are valuable success factors for self-driving cars; the “Internet of Things” and the resulting data will notify consumers about their need to do things—buy a product, schedule a doctor visit, etc. This is the direction of the future.

Companies like Amazon, UPS, and others are making market changes based on real time information to determine and meet consumer needs.

Data will illuminate trends without product and service providers needing to ask questions; but the correct data set must be used for the right situations. Also, consider what are accurate vs inaccurate sources (some conversations tracked may include intentional incorrect data to manipulate perceptions).

This is where the analyst comes in—in helping to determine which sources to fact check.

Spivack provided a variety of examples in which attendees learned real-time data is important and relevant.

Analyst driven solutions rely on rules but miss any attacks that don’t match the rules, notes Spivack.

Machine learning is becoming an increasingly important development for CI practitioners; Boolean thinking is insufficient.

Said Spivack, analysts speak to data; data scientists let the data speak.