The key steps of private decision making —Collect, Viz, Compare—

It is said that the 21st century is the Big Data Era. We can approach various kinds of data, for example, web log data, massive sensory data form machines, public statistics and so on. All our activity is recorded and output in various forms. 

In the business world, data driven decision making is becoming popular. Data is being stored as form of easy use and the fusion of massive data and fine statistical modeling is achieving success every day.

By comparison, can we say that we can handle and make use of all of data for our own decision making, private decision making? Aren't we so overwhelmed by data? For example, when you join meetups, I'm afraid most of you might not utilize the information about other participants. If you made full use of the information, you could obtain a chance with your bright future. In our everyday life, we can't breathe in the flood of data.

I'm convinced that Collect, Viz, and Imagine are the key steps for making full use of massive data. First, Collect is namely the step of collection data. This includes acquiring data through public web APIs and spidering data around WWW. Second, Viz is the visualization of collected data. As you can see in this blog, we can easily visualize data as various forms with programming. Finally, Imagine is to come up with hypothesis from the data and your experience. I'm afraid that this step goes well with Bayesian statistics. At present I'm not good at Bayesian modeling very much , so I usually make use of natural bayesian inference with my brain and my experience.

In the next post, I will describe an example how we Collect Viz and Imagine with R.