Big Data Definition:
Big Data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process the data within a tolerable elapsed time.Big data sizes are a constantly moving target, as of 2012 ranging from a few dozen terabytes to many petabytes of data in a single data set.
In a 2001 research reportand related lectures, META Group (now Gartner) analyst Doug Laney defined data growth challenges and opportunities as being three-dimensional, i.e. increasing volume (amount of data), velocity (speed of data in and out), and variety (range of data types and sources). Gartner, and now much of the industry, continue to use this "3Vs" model for describing big data.In 2012, Gartner updated its definition as follows: "Big data is high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization.] Additionally, a new V "Veracity" is added by some organizations to describe it
If Gartner’s definition (the 3Vs) is still widely used, the growing maturity of the concept fosters a more sound difference between big data and Business Intelligence, regarding data and their use:
Business Intelligence uses descriptive statistics with data with high information density to measure things, detect trends etc.
Big data uses inductive statistics and concepts from nonlinear system identification to infer laws (regressions, nonlinear relationships, and causal effects) from large data sets to reveal relationships, dependencies, and to perform predictions of outcomes and behaviors
Reimage Your Business With Big Data and Analytics:
Any data, big or small, can be a powerful force to accelerate competitive advantage. As an evolution of your overall business intelligence and analytics strategy, a tactical approach to Big Data can put you on the path to reimagine your enterprise.
Big Data Trends Changing the Face of Business:
Failure to Manage and Leverage Data Costs Money:
An increasing number of firms, large, small and in-between, believe that falling short in the management and use of data costs them in both revenue and efficiency. According to CompTIA's Big Data Insights and Opportunities report, organizations say the top five costs associated with shortcomings in managing or using data are:
1. Wasted time that could be spent in other areas of the business
2. Internal confusion over priorities
3. Inefficient or slow decision-making; a lack of agility
4. Inability to effectively assess staff performance
5. Lost sales and reduced margins due to operational inefficiencies
Analytics and Hadoop Skills Aren't as Important as Basic Infrastructure Skills…for Now:
The skills gap is an ever-present consideration, but CompTIA's survey found companies still place the greatest importance on skills related to managing servers, data centers, storage and information. Analytics skills and knowledge of data-specific tools like Hadoop are in demand, but firms rank them lower. CompTIA expects that to change as more organizations begin exploring their big data options.
Machine Data and the Internet of Things Takes Center Stage:
While sentiment analysis and crunching click-stream data is still an important thread in the big data world, machine data is taking on increasing importance, says Ron Bodkin, CEO and founder of Think Big, a big data consulting firm. From RFID tags and industrial equipment to jet engines and consumer electronics, the world around is generating ever increasing amounts of data. Companies are beginning to use that data to improve products, drive efficiencies, identify defects and enhance security.
Compound Applications That Combine Data Sets to Create Value:
The new cornucopia of public and private data is providing a new opportunity to mash up multiple big data sets to gain new insights beyond what a single big data set allows. "The highest value of big data is coming from combining big data sets together," says Ron Bodkin, CEO and founder of Think Big, a big data consulting firm. For instance, Land O' Lakes' WinField business, a provider of agricultural seed and crop protection, combines big data sets—including weather data, soil moisture data, soil type data, seed data and more—to help its growers achieve maximum yields.
There's an Explosion of Innovation Built on Open Source Big Data Tools:
From an open source core, companies are building an array of big data platform technologies, tools and components. "The open source core of big data continues to be the center of the action," says Ron Bodkin, CEO and founder of Think Big. "It's fundamentally being driven by the open source model, but the innovation is around the organizations that are taking the open source components and pushing it forward."
More vendors are providing tools to ease the pain of implementing big data solutions, including General Electric, which is providing tools to help manufacturers harness their data, to Microsoft, which is working closely with Hadoop distribution provider Hortonworks to help business users and analysts access big data sets through Excel.
Companies Taking a Proactive Approach to Identifying Where Big Data Can Have an Impact:
Many early big data projects were skunkworks projects intended to prove the value of big data, but that's changing. "We've seen a lot of viral spread of success," says Ron Bodkin, CEO and founder of Think Big, a big data consulting firm. "But we think there's a better way than relying on skunkworks innovation. It's about taking a more proactive approach to identifying where big data can really have an impact. We do think it's important to have a proven test case, but with executive sponsorship you can get results a lot faster."
There Are More Actual Production Big Data Projects:
Test-bed projects have dominated the big data industry for the past several years, but these days there are a lot more actual production projects, says Ron Bodkin, CEO and founder of Think Big, a big data consulting firm. For now, he says, these projects are largely about achieving data scalability and cost containment, like building a data lake, but the innovators who got started early are beginning to turn their attention to leveraging their new analytics capabilities for business transformation. "They're spending a lot less time chasing down data and a lot more time actually looking at data and answering questions," Bodkin says.
Large Companies Are Increasingly Turning to Big Data:
Large enterprises took to big data initiatives in a major way in 2012. Fifty-three percent of 1,217 large companies surveyed in a global study conducted by Tata Consultancy Services (TCS) undertook a big data initiative during the year. They also have a great deal of confidence in their initiatives—43 percent predicted a return on investment greater than 25 percent.
Most Companies Spend Very Little, A Few Spend A Lot:
Most companies aren't spending a lot on their big data initiatives, but some are investing heavily. Tata Consultancy Services (TCS) found that large companies undertaking big data initiatives are spending a median of $10 million. Twenty-five percent of companies spent less than $2.5 million on their initiatives in 2012.
On the other hand, 15 percent of the companies surveyed by TCS spent more than $100 million on big data initiatives in 2012; 7 percent spent more than $500 million. TCS found that companies in the telecommunications, travel, high tech and banking sectors spent the most, while companies in life sciences, retail and energy/resources spent the least.
Investments Are Geared Toward Generating and Maintaining Revenue:
Perhaps unsurprisingly, business functions that generate and maintain revenue get the most investment when it comes to big data initiatives, according to Tata Consultancy Services. In fact, 55 percent of spending went into four business functions: sales (15.2 percent), marketing (15 percent), customer service (13.3 percent) and R&D/new product development (11.3 percent). Nonrevenue-generating functions get less investment: IT (11.1 percent), finance (7.7 percent) and HR (5 percent).
The Greatest ROI of Big Data Is Coming from the Logistics and Finance Functions:
While revenue-generating functions like sales and marketing are garnering the lion's share of investments (together they represent 30.2 percent of the big data budget), Tata Consultancy Services (TCS) found functions like logistics and finance (which get just 14.4 percent of the big data investment pie) expect a much greater ROI on their investments.
In fact, when TCS asked the companies it surveyed to rate 75 activities in eight business functions on their potential to benefit from big data initiatives, companies around the world listed as many logistics activities in the top 25 as sales activities.
The Biggest Challenges Are as Much Cultural as Technological:
While companies still struggle with technological challenges around big data, companies told Tata Consultancy Services that the biggest hurdle to achieving success in their big data initiatives is getting business units to share information across organizational silos. The technological issue of dealing with the volume, velocity and variety of data still ranks highly, but getting at the data in the first place takes precedence. Companies are also struggling with which data to use to make better business decisions.
>>CHANGE TO BIG DATA OR SUFFER>>>