Industry voice: Understanding big data – and why it's vital for big business decisions

Big data is such a complex and game-changing tool, it is not surprising that businesses are wary and sometimes confused by it. The benefits are significant, and with so many potential uses, it is important that organisations fully understand it before engaging with it.While data does not always have to be "big", a good way of describing this recent trend is multiple sets of data which are too large and complex to be processed through traditional tools.The key for organisations is combining the right data sources to answer business questions. Data can be any size, the critical point is relevance. It can be about almost anything in any format, from customer data, financial data, social media, manufacturing data to sports data, and when analysed, can provide insight and understanding of complex issues. In an increasingly IT-focused digital age, data is being collected from more sources and locations.In the last few years we have seen an explosion in data. There are very few industries which are not using data and could not benefit from the insight it provides. Until recently, the focus for much of this insight has been for marketing, but it is increasingly being used for different applications. One of the most exciting use cases is in sport. The Bolton Wanderers Football Club is using data blending and visualisations to help them understand the movement of players and improve their game.Before being able to analyse and learn from data, businesses need some key questions answered: where is data captured and stored, how is it processed, what is the right data to use to answer the most pressing questions, and what do businesses get from it?Where is data stored and captured?Data can be stored almost anywhere. When it comes to data, it is often so large and from multiple sources that it needs to be stored across multiple databases which are then clustered together. The benefit of a system like this is the scalability. To increase the size of this type of database, companies can simply install more storage and put in place enough hardware to manage it.There are generally two main ways data is stored: SQL and NoSQL. SQL (Structured Query Language) is a type of programming language designed for data. From the 1970s until recently, SQL-based databases were the dominant force. However, SQL has begun to lose its appeal as the means to store data because the code is not fully portable. It can also be a bit restrictive as the standard is not always maintained leaving businesses unable to blend certain data sources together. NoSQL (Not only SQL) was designed to solve these issues. NoSQL supports SQL along with multiple other languages, adapted to the demands of data. With NoSQL, speed comes first, and unlike SQL, there is no structure so the system is horizontally scalable. This makes growth very easy. If an organisation has enough space to store data then further databases can be added to grow the overall data cluster. For this reason, NoSQL is the system of choice for heavily data dependent organisations such as Google, Amazon and the CIA. HadoopHadoop is a software ecosystem which enables SQL and NoSQL databases. When introduced it dramatically speeds up processes by clustering databases in parallel. Because the data is stored in separate places, a data analysis or blending procedure which might take 20 hours can take just three minutes.As data requirements have grown, Hadoop has enabled this growth, allowing for the management of structured (SQL) and unstructured (NoSQL) data.Hadoop is one of the key factors for the current data revolution we are experiencing. When combined with data analysis and blending software, Hadoop can be used by largely anyone able to understand the software, often without the need for a data scientist. Data captureOnce an efficient means of storage is available, the capture can be relatively simple. Current dependence on IT has resulted in multiple opportunities to capture data. A simple example illustrates this...In most business environments, data and interactions are increasingly becoming electronic. Even when people have face to face meetings, agendas, minutes and summaries will be distributed electronically. By monitoring interactions and gathering data, organisations are able to maximise their productivity. This was achieved by Sandy Pentland, a professor of computer science, who monitored interactions between workers at a call centre. Pentland used the data gathered to restructure coffee breaks and enhanced the flow of ideas, varying who interacts with who and changing conversations.Equally, data can be collected from existing programs and software. For example in healthcare, data is being developed to be used to provide treatment tailored to genes. When samples are taken, typically results are returned in a digital format. The most innovative healthcare companies will feed this information into machine learning systems which will be able to suggest or eventually prescribe treatment. The opportunities for collecting data are almost endless, especially because of our growing dependence on IT. In order to gain real insight, however, this data needs to be processed into a format which can be easily understood and used to drive results. To gain more insight across the organisation, it also needs to be accessible to the data analyst within the company, so there isn't a need to rely solely on IT to handle these projects and produce results.How is it processed?There are a number of tools available to process data, dependent on business requirements. Traditional tools for analysis of data have their issues, and are left unable to process the size of the modern day data environment. For line-of-business analysts, manually inputting or processing data is too time consuming and will result in data being out of date by the time it is acted upon. Then organisations will fall behind their competitors in their ability to make data-driven decisions.For many businesses, the major benefit of data is that it can be blended. As data is often captured from so many sources, and often saved in different formats, it is important for businesses to be able to effectively blend the right data, organise and place insight at their fingertips. Many data sets will come from multiple sources. For example, customer information for a retailer might be made up of data taken from in-store and online purchases, social media likes/dislikes, call centre information and many more. All of this data needs to be united in one place for analysis to provide a deep understanding of shopping behaviour.Data blending is therefore a critical tool, enabling multiple data sources to be joined in one place. For example Database USA, a provider of business mailing lists, blends data from hundreds of data sources in order to ensure their lists are up to date and accurate.What do businesses get from data?There are an endless number of specific industry uses for data, as well as some key means by which almost any business can benefit, from increasing productivity, to intelligent decision making and personalised customer relationships.The most traditional use of data is marketing. Data-driven marketing has been around for a little while, but it is growing in its use across verticals. Gartner predicted that by 2017 the average CMO will be spending more than CIOs on IT services. This has sparked much more of a link-up between marketing and IT departments, who are also helping to combine marketing and CRM databases.Data allows marketers to make their contact with customers and potential customers more personal. Knowing that your core customers are made up of specific demographics and having an understanding of individual customer preferences makes personalised marketing achievable. One of the most exciting uses of data I have seen recently has been in healthcare. We are starting to see medical teams developing tailored treatment for patients based on their specific gene data metrics. Doctors will be able to correlate genes to hereditary illnesses and consequently suggest preventative treatment, and the most effective treatment.Data has the potential to impact so many different industries. Another key area where we are seeing growth is in financial services. There are multiple use cases for data in the finance industry, from detecting fraud to risk analysis. We are increasingly seeing investment companies carrying out financial forecasting using data and predictive analytics. Direct benefitsFor almost any industry, productivity, business intelligence and strong customer relationships are important to growth. These can be dramatically enhanced by the right use of data. The growth in the use of data in recent years has produced an environment by which the day-to-day life of organisations can be made leaner and more effective.1. Increased productivityOne of the most important data benefits is the ability to save time and increase productivity. Organisational leaders who leverage data properly can obtain an overall view of the internal movement of ideas and data, spotting bottlenecks and over-performing units which help them to make changes accordingly. A McKinsey report highlighted that acting on social business data can result in a 20-25% increase in knowledge worker productivity. Analysing and acting on the data can have a dramatic impact on productivity and help to reduce costs. Making data and analysis an everyday feature of company life will encourage staff to further use it to help save time on menial tasks and get more out of their time.2. Business IntelligenceMaking intelligent business decisions is important to any organisation, yet to the growing number of organisations becoming more adept with data, it is the data that makes the decision intelligent. The amount and types of data companies are processing means that they can put together predictive models, and use their data to decide on the next best action, removing the guesswork.Financial companies are increasingly using data to make critical decisions by predicting the movements of financial markets and the outcome of their actions. Equally, having all of the information to make a decision in one place can dramatically improve the process. In most organisations, critical data is usually available; however, businesses that have not moved into the data world are often unable to access the right data. Unsurprisingly, the Economist Intelligence Unit reported that nine out of 10 business leaders believe that their decisions would be better made had all the relevant information been available to them.3. Increased customer personalisationClosely tied with marketing, using data to personalise customer relationships is a growing trend. Smart merchants are segmenting and differentiating their treatment of customers dependent on customer profiles. Knowing if a customer is a big spender or likes a certain type of product means contact with that individual can be tailored to theses preferences. Companies using data in this way are therefore able to invest their marketing into relevant areas, focused on higher value customers or increasing overall loyalty. Data provides an opportunity to gather information from users which can then enhance the user experience, building brand and improving decision making, bringing customers into the boardroom.Acting on dataTraditionally, data analysis has been a slow and labour-intensive process. The growth of data awareness has meant that today's analysts can spend more time acting on data, rather than processing it. New IT tools, such as data blending software, have provided businesses with the ability to be creative and use insight to find the information they need. Empowered by these exciting analysis tools, line-of-business analysts are already doing more than they could just a few years ago. Data is not merely just a marketing tool; it is a full-scale business tool which can be used across organisations. I would expect to see decision making with data become almost as essential to businesses as IT in the future.Stuart Wilson is regional vice president of Alteryx UK

Click here to view full content