Marketers and chief data officers must contend with ever-growing volumes of data in their day-to-day lives. This truth has resulted in “big data” morphing from a buzzword into an undisputed fact of modern corporate life. Between consumer-generated data and business-generated data, the sheer amount of data generated and collected worldwide has exploded.
Recent research from Northeastern University found that the amount of data worldwide will grow to 44 zettabytes by 2020, with 2.5 exabytes of new data created every single day. This equates to 2.5 quintillion bytes of new data from social media, web traffic, and other sources created every hour of every day. The explosion of data translates into a massive expansion in the amount of data that companies collect, store, and manage, which results in a greater interest in big data technologies.
And this is a good thing. Big data technologies can make a powerful impact on your business, especially from the perspective of managing increasing volumes of data and creating business value. But as with any technical solution, brands need to proceed with caution. If you’re not careful, you could end up with a solution that costs a lot of money and doesn’t do what your business really needs.
Technology Isn’t Always the Solution
Modern big data technologies are built to solve specific problems. These specialized solutions are extremely powerful and can streamline operations dramatically where the very large datasets commonly defined as “big data” are in evidence. That said, big data technologies are nevertheless software built specifically to manage and derive value from datasets that are too large or too complex for traditional data processing applications. These are technical solutions to a very technical set of problems.
One thing that big data solutions cannot do is contend with organizational issues. These include challenges like siloed departments, “turf wars” over who “owns” specific datasets, and so on. Seek out vendors providing big data technologies and you will find a host of companies willing to accept your business. But beware – if you begin the purchase process for a big data solution without knowing what you want to achieve, you risk losing both time and money and getting stuck with an expensive piece of software that doesn’t provide business value.
Goals First, Technology Second
It may be an old piece of advice, but if you start by defining the problem you want to solve, you’ll end up with a better result. A clearly defined business problem focuses any technology search on solutions that will solve your specific use-case first and foremost. You’ll likely gain other efficiencies along the way, but the core of your search should ideally focus on a small number of specific objectives.
If you have a lot of unstructured data, for example, you might seek out a data lake. For massive volumes of structured data, perhaps a purpose-built data warehouse can help. If you need to blend high volumes of structured and unstructured data that travels at batch and streaming cadences, might I suggest a data hub instead? These are just a few examples of technologies that align to clearly defined use-cases businesses need to solve for.
Starting with a business problem also includes enlisting the aid of any other potential stakeholders in the organization. If your company is like most, then your team isn’t the only one struggling with data volumes. If you work in marketing, consider asking sales and service about their data problems. Engaging the IT team is imperative, if for no other reason than they can tell you whether that flashy solution will play nice with the rest of your systems.
If you can gain alignment with the various stakeholders who stand to benefit from a big data solution, then you also build a stronger case to seek out a new technology. Multiple departments willing to share a joint solution tends to be an easier sell than marketing or sales asking for a new department-specific system.
By defining your business requirements upfront – i.e., what your goal is – you may also find that the issue is less one of technology and more one of process. In that case, you could find efficiencies that otherwise would have gone unrealized because of a too-heavy focus on deploying a new and shiny technology. If it is a problem in need of a technology solution, then you also can more clearly define its true drop-dead requirements.
The idea that deploying a new technology will solve all your big data problems is a dangerous one. This kind of thinking results in companies spending thousands of dollars on unnecessary systems that provide no business value. Better to start with a clearly defined problem and only then seeking out technology solutions – if that is even what is needed. What this all boils down to is the simple reality that you need to solve the right problems, and be certain that you are doing so.
