Why We Need to Think Small About Big Data

Why We Need to Think Small About Big Data

Big Data is huge (literally and in terms of its popularity) but to really see its value, we need to think about it at the micro level.

In January, two Princeton University students decided to apply virus propagation modelling to Facebook’s growth to project its future. Based on their results – and the research methods that relied on big amounts of historical data related to the rates of growth and decline of disease spreading within society – they were able to conclude that Facebook’s membership would decrease by 80% by 2017 (maybe because of some Facebook vaccine?). Now, we should keep in mind that this study was not peer-reviewed and was more done as a way to shock the world than to relay a point. What wasn’t accounted for was the fact that Facebook has reached a critical mass. Long story short: Facebook isn’t going anywhere.

The Facebook Virus Model

What we are supposed to expect from Facebook in the next few years based on Virus Propagation Modelling.

A few years ago, during the incipience of the concept of ‘big data’, Google decided to use the data they had from search histories to predict when flu season would arrive. (Why all this focus on sickness and disease when it comes to data?) Though the initial experiment was a success, Google has seen more missteps than triumphs in recent years when it comes to predicting the next plague.

Big data is exciting (or sickening depending on how you interpret its aforementioned applications). Recently, a phenomenal op-ed piece written by Gary Marcus and Ernest Davis (which I would implore everyone here to read) was published in the New York Times. The article focused on major issues plaguing the application of big data to social modelling. (Much like the issues described above.) Right now, there is so much hype around the concept of big data, that people are forgetting that it needs to be looked at on a small scale. Leave the mass modelling to governments and think tanks. Businesses should be looking at their own internal version of ‘big data’ (relatively speaking) and see how it can aid in achieving operational efficiency.

What exactly is ‘Big Data’?

Big data is, for the most part, exactly what it sounds like; it is large quantities of data with virtually limitless applications. Data sets collected are often (and by definition) so large that traditional methods of computing become virtually impossible, hence the advent of so many analysis tools in the marketplace today. In the last two years, we (humans) have created 90% of the information that exists in the world today (statistic courtesy of IBM). Every day, we are creating more, and it is both overwhelming and exciting.

Think Small About Big Data

Photo Credit: Shutterstock. Used under license.

But as with many new applications, we are more excited by the prospect of what it can do on a large scale (and rightfully so) than how it can be applied to smaller applications. The irony is that in this case, at least at present, the real value in big data is thinking small.

Just how small are we talking?

Well, Google was trying to leverage big data in order to predict sickness in different geographical locations. That’s a big use of big data. A smaller use would be finding correlations in engagement spikes and optimizing a content strategy with the information you can pull from these reports.

Every day, businesses are handed a new set of big data from every one of their digital components. Whether it is your Facebook page, a Twitter ad campaign or your website’s analytics, there is a new trove of data waiting to be combed for valuable gems.

What in the world am I talking about, you ask?

This is what in the world I am talking about.

Say I export my data from Facebook once every week. At the end of the month, I have thousands of data points that can be analyzed. Don’t believe me? Log in to your Facebook page and export your most recent month’s page level data. Every day there are dozens of data points that you can study. Even more so when you look at your page level data and you can double that if you run ads on Facebook. So when I say thousands, I mean thousands.

Say you run a regression (sorry stats-haters) on your data in order to identify outliers (both positive and negative) as well as influence points in your data. Then, looking closely at those outliers and influence data you identify correlations that exist. Maybe at every one of these points, you noticed that an image was shared as opposed to text. Maybe 95% of these posts have fewer than 40 words. Maybe more than three-quarters feature a link or a question. This is all extremely valuable when optimizing your brand’s content strategy.

Why am I emphasizing your brand? Too often, we get bogged down in aggregate data. What’s that? It’s people who look at five million Facebook posts, note that images have higher engagement and tell brands that they need to share images and nothing but images. That might have been true for the majority of posts that were examined by that one analyst, but it might not work with your audience. Using your own version of big data to run these same (fairly simple) reports can add a huge competitive advantage when it comes to sharing content that resonates with your audience.

What if I don’t understand all of this?

The beauty of these kinds of programs is that they have been designed to appeal to the masses. It would be foolish for a company like Facebook or Google to nix the idea of targeting the statistically-inept. After all, that would make up the majority of us. Basic analyses of your data are everywhere. Tools exist to examine and analyze top-level data and even some of your free insights on those same networks provide you with information that can be extremely valuable in crafting these strategies.


You don’t need to be a statistician to use big data. When you see the potential with regards to the applications of big data, you can understand why there is such excitement surrounding it. Imagine the possibilities that can be unearthed by looking at these data in new ways.

At the end of Marcus’ and Davis’ piece, they conclude that there is reason to be excited about big data, but that there were more marked breakthroughs in the 19th and 20th century in the form of medicine and other major societal advancements. Of course we can’t dispute that these discoveries changed everything about human life. But big data is just getting started. We have no idea where it can take the human race (both metaphorically and physically) and what the future holds.

All we can say for certain is that when it comes to business, the applications are clear and extremely useful.

The following two tabs change content below.
Corey Padveen is a data-oriented marketing professional with a focus on statistical analyses of human behavior. This specialization has led him to speak and present at dozens of conferences around the world, to write for a variety of reputable online and print publications, and recently, to publish ‘Marketing to Millennials For Dummies’ as part of the world-renowned ‘For Dummies’ series. He regularly shares real world examples and findings from his research, and discusses how members of society are evolving as consumers, communicators, and a global network as a whole.
Comments are closed.