Big Data, The Moving Parts: Fast Data, Big Analytics, and Deep Insight

    Newer Older

    Given that the term itself is simplistic, it should be understood that there at least three significant aspects of Big Data that make it unique, beyond "an order of magnitude more data than what you have now", which was one early definition.

    Instead, Big Data refers to the integrated employment of the following capabilities:

    Fast Data. Recognizing that traditional methods for moving, processing, and querying data were not sufficient, the Big Data industry has created an entirely new set of techniques -- and adapting some of those that existed -- so that organizations can actually process the full universe of information that they possess in enough time to actually get inside the windows of key business processes and critical decision trees. Thus, Fast Data techniques provides the ability to 'see' all (or enough, anyway) of what you know in a short enough time to actually do something with what you've learned. Fast Data techniques, at least so far, have grown exponentially faster at approximately the rate of Moore' Law, just barely keeping up with Big Data growth volume in my research.

    Big Analytics. This is where the qualitative differences between traditional business databases and Big Data become more apparent. Where Fast Data is about new techniques to process and transform raw information considerably faster than ever before, Big Analytics is about turning information into knowledge using a combination of existing and new approaches. As you can see from the moving parts visual above, some of the classic players in analytics are in use here including MATLAB, SAS, and R. But some of the most interesting aspects of Big Data can be found in relatively new entrants such as Apache Hive and Mahout, the latter which brings to bear automated machine learning to finding hidden trends and otherwise unthought of or unconsidered ideas. In fact, an entire industry is growing up in smart information management systems that will "not rely on users dreaming up smart questions to ask computers; rather, they will automatically determine if new observations reveal something of sufficient interest to warrant some reaction, e.g., sending an automatic notification to a user or a system about an opportunity or risk."

    Deep Insight. The powerful yet unfocused tools of Big Analytics are not sufficient to reap the rewards of Big Data. That requires taking the sum of the information at hand, applying analytic processes to it, and finally generating new knowledge and insights using a a specific, situated method. Insight must be in the domain of the business to be useful, and this part of Big Data is where the technology is connected to ground truth in a feedback loop. That is, the tools of Big Analytics are just tools by themselves. It's not until they are directed at deriving a particular type of result that they are actually useful in a business context. Insights must also be connected to specific objectives (examples depicted in the moving parts visual above) in order to have high levels of impact.

    From The enterprise opportunity of Big Data: Closing the "clue gap" | ZDNet

    The bottom line: Big Data + Usability + Broad Access = Scalable Competitive Advantage

    See Also:

    How social media and big data will unleash what we know | ZDNet

    Big Data Predictions for 2012 | Collaboratory

    Harnessing Social Business Intelligence: Nine Strategic Uses

    The Disruptive Shifts: Calling Out Social Business Amongst Mobile, Cloud, Consumerization, and Big Data

    keyboard shortcuts: previous photo next photo L view in light box F favorite < scroll film strip left > scroll film strip right ? show all shortcuts