EMC Greenplum President Bill Cook explains how big data is the "engine for creating new economic value."
Big data analytics explores the granular details of business operations and customer interactions that seldom find their way into a data warehouse or standard report.
By 2020, the quantity of electronically stored data will reach 35 trillion gigabytes, a forty-four-fold increase from 2009. According to IDC's 2011 Digital Universe Study sponsored by EMC, we reached 1.2 million petabytes, or 1.2 zettabytes, by the end of 2010. That's enough data to fill a stack of DVDs reaching from the Earth to the moon and backabout 240,000 miles each way.
For alarmists, this is an ominous data storage doomsday forecast. For opportunists, it's an information gold mine whose riches will be increasingly easy to excavate as technology advances.
When data volumes skyrocketed in the early 2000s, storage and CPU technologies were overwhelmed by the numerous terabytes of big data to the point that IT faced a scalability crisis. Then, we were once again snatched from the jaws of defeat by Moore's law. Storage and CPUs not only developed greater capacity, speed and intelligence; they also fell in price. Enterprises went from being unable to afford or manage big data to allocating budgets on the collection and analysis of it.
What used to be a technical problem is now a tremendous business opportunity.
Digital data is ubiquitous; it is in every industry, in every economy, in every organizationmaking big data relevant for leaders across every sector.
Putting those zetabytes of structured and unstructured data (from operational systems, cell phones, Twitter, Facebook and other modern sources) to work can play a role in helping reduce costs, improve customer relationships, develop new products, accelerate and synchronize delivery, ask and answer deeper questions as well as enhance and simplify decision making.
Big data is about putting all that information to work. Access to the many zetabytes of data in the world is meaningless to your business until you start applying next generation analytic tools and methods to generate insights that enable you to do something that helps move your organization forward.
The McKinsey Global Institute suggests that if US Healthcare could use big data creatively and effectively to drive efficiency and quality, the potential value from data in that sector could be more than $300B in value, two thirds of which would be in the form of reducing national health care expenditures by about 8%. In the private sector, McKinsey estimates that a retailer using big data analytics to the full has the potential to increase its operation margin by more than 60%.
Big data allows you to innovate beyond what you can currently comprehend to solve problems and invent products or services that you've never thought of before!
Big data is part of an IT continuum and is a crucial piece of an enterprise information strategy.
Simple reporting, spreadsheets and even fairly sophisticated drill-down analysis have become commonplace expectations of business intelligence. However, there are types of analysis that BI can't produce and that is where big data comes into the equation.
Big data analytics explores the granular details of business operations and customer interactions that seldom find their way into a data warehouse or standard report, because a growing share of this information is unstructured data that can't be warehoused or analyzed in neat columns and rows. Also, this data is constantly in motion so its velocity defies the current RDBMS model.
When you are after things like predictive analysis, natural language processing, machine learning or advanced statistical techniques, even if you want to analyze and mash up unstructured content in the BI mix - you need new technologies and ways of working with data to get at the insight that can bring value to your organization.
By leveraging the power of big data analytics you develop an "information advantage" and can transform from a reactive organization that uses data to understand the lessons of the past, into a predictive, proactive organization that uses the insight contained in big data to anticipate and execute on opportunities of the future.
The possibilities of big data continue to evolve rapidly, driven by innovation in the underlying technologies, platforms, and analytic capabilities for handling data, as well as the evolution of behavior among its users and as more and more individuals live digital lives.
Using big data technologies and analytics will become a key indicator for competition and will likely create new competitors or ways of competing in your industry. As you move forward, think about a couple of things: