Overview: Big Data and Hadoop


CIO Insights: Big Data and Hadoop

What is big data being used for?

Big data investments are on the rise. Businesses are using big data to address such issues as enhanced customer experience, process efficiency, new products/new business models, and more. While some of the top hurdles or challenges for with big data among businesses have been determining how to get value from it, defining a strategy, and obtaining the skills and capabilities needed, in 2012, 58% of companies had invested in big data technology or planned to. In 2013, this number increased by 6%.

How has Hadoop changed the landscape, and what are key vendor road maps telling us?

Hadoop has become a key platform driving big data interest. Apache Hadoop is a varying se of standard open-source software projects that provide a framework for using large amounts of data across a distributed network. Many distributions exist and the ecosystem is growing as products emerge in multiple categories. Hadoop 2.0 will broaden target uses by consolidating clusters, moving models from file systems only to databases, and improving SQL and differentiating players.

What are the barriers to effective Hadoop use, and what steps can you take to avoid them?

Cost of acquiring skills, integration with current infrastructure, lack of security, immature analytical applications, and undefined value proposition are the biggest barriers to Hadoop use. Typical stages and milestones of big data adoption span from unaware to strategic and transformative. During these last stages are when businesses reap the most value from big data.

To avoid some of the barriers during Hadoop integrations:

  • Audit your data—find ‘dark data’ and connect it to opportunities to identify projects
  • Identify emerging functionality for application to your business
  • Consider cloud pilots to minimize capital expenditures
  • Design for security, information governance, and broad data warehouse integration