Digitization has made significant strides in recent years - racks of documents and piles of files have been replaced with zettabytes of data stored in the servers of data warehouses. The dawn of the ‘Internet of Things’ 1 age has led to a data deluge; both corporate entities - in sectors such financial services and insurance, healthcare, pharmaceuticals retail and manufacturing - and individual users are churning out all this data. Trends such as the growing use of mobile devices and social media networks are generating considerable amounts of data on the individual users’ side; meanwhile driving data generation on the corporate side are factors such as regulatory requirements that mandate storing internally generated structured data in the financial services industry. According to a report by storage solutions provider EMC, India will generate and share almost 40,000 petabytes of data in a year, and this figure is expected to surge 60 times to reach 2.3 exabytes in the next 10 years. Based on EMC’s analysis, the growth rate of data in India is double the growth expected worldwide2.
The proliferation of such large pools of data has led to the invention of a new industry term - big data. Big data, as defined by Gartner, has three primary characteristics - volume (amount), velocity (speed of creation and
utilization), and variety (types and sources of unstructured data) 3. In common terminology, big data refers to large datasets that cannot be addressed using legacy data-management tools. Such datasets require next-generation techniques and technologies to aggregate, manipulate, analyze and visualize big data.
The onset of the big data phenomenon automatically gives rise to challenges of data management. In such as scenario, a question that most CIOs need to answer is, “How can this data make economic sense to the organization?”
Making economic sense - the utility of big data
Big data can unravel unseen opportunities for an organization. The availability of managed and structured data can help decision makers in several ways4:
- Introduces Transparency: Making data readily available can reduce search and processing time and add significant value to an organization. In many cases, making data available to relevant stakeholders in a timely manner can reduce time-to-market and improve quality.
- Helps discover reasons for variability: Organizations can collect and manage increasingly detailed performance data on every aspect and use it to analyze variability in performance. This can help in identifying root causes and take requisite corrective action.
- Helps identify patterns and facilitates customization: Big data allows enterprises to analyze patterns across customer profiles and create specific segment-tailored products and services to meet the varied and specific needs of customers.
- Encourages a scientific approach to decision making: Companies can use algorithms and conduct sophisticated analysis to facilitate decisionmaking processes such as streamlining inventories and pricing in response to real-time in-store and online sales. Big data analytics enhances the quality of decision making and helps decision makers take increasingly informed decisions.
- Facilitates innovation: Big data can provide insights that can help improve existing products and create new products and services.
Enterprises can leverage big data in a number of ways to create value. However, the key to unfolding the full potential of big data is an enterprise’s ability to organize, manage and analyze data using effective big data
technology.
Key technologies for big data
A wide range of technology is available to facilitate the aggregation, management and analysis of big data. While the list continues to expand, some of the key technologies currently available are as follows:
Indian IT players have also boarded the big data bandwagon. The Infosys SocialEdge platform generates actionable insights from big data for immediate use by firms5. TCS has also zeroed in on big data as a key technology likely to shape the computing world of tomorrow6. Cognizant is also looking at developing solutions to cater to the challenges that big data gives rise to. Many programs launched by the Indian Government, including the nationwide Unique Identification Authority of India (UIDAI) program, is providing IT players with the opportunity to market their big data services in the domestic market7.
The big data market - exploring the unexplored
The global big data market is well-poised for growth; companies have started recognizing the potential of big data and have, thus, started tapping this space and its related analytic capabilities to gain operational efficiency.
Currently, global vendors such as IBM, Intel, and HP continue to dominate the big data market8. Indian IT-BPO players are yet to ride this growth bandwagon successfully and are gain a strong foothold at the global level. As recently as five years ago, selling analytics to Indian clients used to be challenging. This trend is now changing, and the industry is witnessing growth in adoption; yet, it the market is yet to witness widespread deployment. Even on the M&A front, global giants such as Opera Solutions and IBM have been really aggressive, while Indian players continue to lag behind.
KPMG in India’s point of view
Globally, big data has now spread its wings and is no more restricted to big enterprises or internet companies. Interest in this space is growing, which is leading to a shift in the business paradigm of an already new segment. A number of new technologies are emerging in an already crowded space. Transcending the era of Hadoop, one of the oldest solutions in this space, the market is moving to Advanced SQL and NoSQL platforms. This could make big data available to mainstream SQL, Java, and web developers.
In addition, trends such as cloud computing - which allows players to rent storage and computing capacity - is likely to become an attractive deployment option and strengthen the adoption of big data solutions.
Such trends are creating ripples in the industry on the one hand; on the other, a few challenges must be addressed. A shortage of skilled professionals is a major roadblock in the development of the big data market. Companies are unable to find suitable talent to fully exploit this space. Globally, it is estimated that the demand for data scientists could outstrip supply by more than 1.5 million9. Other than talent shortage, bandwidth limitations can hamper the efficient use of computing and storage resources in a cluster. Further, maintaining data security and privacy can be an arduous task. Protecting sensitive data and creating safeguards to prevent abuse is the need of the hour.
As such, companies need to create a balance to effectively address challenges and explore opportunities. From a client perspective, companies need to formulate a strategic roadmap - this involves creating a big data strategy, training resources within an organization, breaking through silos and provisioning for data sharing. In addition, it is important that companies undertake systematic planning to determine the best possible tools and techniques for complex dataset analysis. From a vendor standpoint, IT-BPO companies in India need to gather momentum and tap the global big data market. Companies may revisit their M&A strategies to acquire appropriate assets in global markets. A case in point is Wipro, which recently acquired Promax Applications Group (PAG), an Australia-based analytics company10. In their quest against talent shortage, companies may also collaborate with academia to train and certify data scientists of tomorrow.
As industries mature, companies will increasingly use data to gain a competitive edge. The next decade is expected to see growth in the demand for big data services. Thus, it is important for Indian IT-BPO companies to come to the fore and strengthen their position in the global big data market. Adopting a ‘wait-and-watch’ approach could be tantamount to playing the role of an extra rather than taking the lead.
Sources
1. The Internet of Things refers to sensors and intelligence embedded in items such as consumer devices or physical assets. It is connected to the internet
2. “Indian CIOs prepare to take on the big data challenge”, Dec 2011, Business Standard
3. “Creating a Winning Strategy with Big Data”, Jun 2012, CIO.in
4. “Big data: The next frontier for innovation, competition, and productivity”, June 2011, McKinsey Global Institute
5. Infosys Resource Center, accessed in July 2012
6. “Big Data: Storage Solutions”, July 2012, TCS
7. UIDAI Website
8. Services Angle, Big Data market , accessed in July 2012
9 “Creating a Winning Strategy with Big Data”, June 2012, CIO.in
10 “Wipro acquires Australia analytics company”, May 2012, The Times of India