How-To Guide: We Have Big Data, Now What?

The term “big data” is both commonplace and misunderstood, a buzzword that means different things to different companies. The idea of access to radically more information that translates to improved company performance is immediately appealing, prompting organizations of all shapes and sizes to hastily jump on the big data train. This demand gave rise to big data pioneers like Hadoop, which empowered businesses to capture and store massive amounts of information, from users’ search terms and social media updates to spending patterns across geographies.

Today, many organizations have adopted some form of big data storage, but lack the technology and staff to utilize the data in a meaningful way. While resource-rich companies like Facebook and Amazon have seemingly mastered the art of analyzing user activity data in order to match users’ interests and demographics with targeted advertising campaigns, even their systems are far from perfect. For example, one issue is that data scientists can’t always access and analyze entire data sets, so they are forced to extract and process much smaller subsets that could lead to an incomplete or misleading conclusion. Ever seen an advertisement on your newsfeed that you find weird? This may be why. For example, I wear comfortable, flat and conservative shoes, so when an advertisement for Kim Kardashian shoes appears on my Facebook page, it clearly misses its target audience. Or when my friend, a partner at a consulting firm, receives an ad to join that same firm as an analyst, the advertisement’s targeting analytics are clearly off.

The Data Scientist Bottleneck

In 2012, Forrester claimed that big data‘s potential is not about technology, but about smart people with the right skillsets. Up until now, Forrester was completely right. This people-oriented prediction may seem positive at first glance, but in reality, a lack of easy to use technology has forced us to depend on people with very specific skillsets and limited access to big data insights across lines of business. Here is why:

Big Data Can Be Cumbersome: Big data is notorious for being difficult to use, meaning that employees in your HR or marketing department aren’t suited to analyze big data on their own.

Data Scientists Are Rare: If your company has a data scientist at all, they are likely a small team inundated with an unreasonable amount of incoming requests.

Data Analysis Is Extremely Time-Consuming: Between limited resources and massive unstructured data sets, data scientists could spend countless hours analyzing information for only meager results.

The bottom line: the lack of easy to use self-service tools for big data combined with a shortage of data scientists creates a big data bottleneck and an impediment to business growth.

“Now what?”

In order for data-driven decisions to be a reality at your organization, big data must be taken out of the storage and computing layers and made accessible for business users across all departments, from HR and marketing to accounting and operations. This will empower employees to make better informed business decisions, such as defining an audience for a hyper-targeted marketing campaign or developing a smoother check-out process based on past customer transactions. And they can make these business decisions without waiting in the queue for an overloaded data scientist or inundated IT department.

To make this level of accessibility possible, businesses must leverage technologies that make big data as simple as a Google search. That’s why we founded Arimo and created a platform that empowers anyone to be a data geek.

By democratizing data intelligence, everyone wins. Business users without programming skills or a background in statistics can instantly access and process data on their own, while data scientists are able to scale themselves across an organization and play a more cross-functional role. This can have a drastic impact on productivity, encourage collaboration across teams and increase business effectiveness.

Only the Beginning

The democratization of data intelligence is a giant step forward, and we are beginning to see the early results on what happens when everyone in an organization adopts a data-driven mindset.