Big Data and the Online Economy

Big Data in the Online Economy

Everyone is talking about “Big Data” and the amount of interest in this area from investors and even the government is currently huge. Online/internet companies like Google, Amazon, Facebook have be

en the lead adoptors of this technology and are also in the forefront of innovations in this space. It would not be too far-fetched to claim that “Big Data” is one of the engines driving the online economy .

What benefits does this massive data processing capability provide?

For companies, one of the primary benefit is improved operational efficiencies in targetting the right customer base – whether it is Google and Facebook using your browsing/searching history and online activity to sell targeted ads, Amazon providing better recommendations for product purchases, or Target using your purchase history to send you coupons. In the semi-online world, “big data” is enabling new discoveries in the health sciences field and other scientific endeavors, helping intelligence agencies with better analyses, driving more operational efficiencies in supply chain management, Khan Academy is using it to better evaluate students, insurance companies are using massive data sets to predict risk better and quicker, hubway is using it to provide improve locations and capacity to best serve cyclists, sports teams (money ball!) are using it to evaluate players, and even presidential election are using it to win elections! There is also a push towards using big data for more data oriented decision within organizations with the hope that it will result in better business outcomes.

For customers, these advances in big data processing have made the online browsing experience more customized. Imagine LinkedIn without its “Jobs You may be interested In” feature or Amazon, YouTube, New York Times etc. without its recommendation engine. New frontiers are also being explored. A startup travel company in the Boston area is exploring the idea of using individual browsing and search histories and online social activity to provide you with travel destination recommendations which you most likely to enjoy. Big data analytics combined with “crowd sourced” intelligence is another approach leading to some very interesting products. A blog post earlier in this semester dealt with

The Downsides of Big Data
There is also a potential downside to this advance – the social and legal implications of governments and organizations posessing so much information about individuals is not completely clear yet. There is literally a “data grab” going on and there have been some obvious cases of invasion of privacy. A retail chain knowing when women are pregnant before their families do, or a group of students working on a class project being able to predict a person’s sexual preferences based on the person’s facebook activity are just two instances of this. If enterprises – with their primary objective of profit making – know so much about you, how will they use this information? In the best case scenario, people will get information really relevant to them. In the worst case scenario, it could lead to some kind of “online discrimination” based on your online activity or even something insidious as information control – companies or governments deciding what information people see and when. Whichever way it plays out, it is definitely going to be an interesting ride in the near future.

References and Articles:


  1. Tom Deane

    Very interesting topic Shirish, and Big Data is definitely a buzzword these days. I agree with you that there should be more controls and restrictions on people’s right to privacy in a world where so much data about individuals is flowing online. I still remember the feeling of uneasiness two years ago when I searched for my name on Spokeo and found an incredible amount of information including age, current and past addresses, family members with name and address, etc. I was curious about how they got this data when I never consciously agreed to be listed and I’m careful with where I enter personal information. The only answer I got was that the system behind their data sources is too complex and diverse to trace it to any one source. The process for removing my information from the directory was also very tedious.
    On the flip side, this problem has created opportunities for startups to fight back. Companies like have created business models around protecting people’s privacy. now has almost $70M in funding primarily from KPCB, which suggests the problem is very real and people are paying for the service. Nonetheless, it’s still a bit saddening that we now have to pay for something that used to be free – our right to privacy.

  2. This topic of privacy is an interesting and ever-evolving one. Just today, Facebook announced new privacy settings…once again. They were responding to rising criticism among privacy advocates about how personal information is being disclosed.

    Among the changes, Facebook makes it easier for users to track who can see their posts and photos. Part of the goal of the changes is to allow users more ease of control in navigating and changing their privacy settings. For example, “The company is modifying its main toolbar to include shortcuts to key privacy settings and also providing more controls in the ‘activity log,’ where users can see how they appear in different places on the social network. Other changes include more explicit requests for permission to share certain information, as well as explanations of where information may appear on the network.” [1] The hope is to bring clarity to a system that’s notorious for being complicated and confusing.

    On the other hand, another change disables the function that previously allowed users to block others from seeing their Timeline page when searching for their name. In this case, people thought they were afforded a certain level of security, but in reality, they were not. I wonder how many other loopholes like this exist.

    It is interesting that privacy seems to be an issue that’s top of mind for many people; however, only 1% of Facebook’s users voted on their Statement of Rights and Responsiblities (SRR) and Data Use Policy. You could argue that the proposed changes were not meaningful enough for people to act. You could argue that the policy documentation is simply too time-consuming and / or confusing for most people to follow. You could argue that people were pre-occupied. I wonder if it’s more likely that we live in a reactive society where people do not take action unless a problem is at their doorstep. And, I wonder when these privacy concerns will reach the doorstep of the masses.

    [1] Brandon Bailey, “Facebook privacy controls changed, social network adds and subtracts features”, Mercury News, December 12, 2012,

  3. Henry Lin

    I also believe online reputation and privacy concerns will become bigger issues as we all begin to have digital footprints on the web. My recommendation is to start early. Instead of hiring a professional to eliminate the bad stuff, don’t put it online in the first place. It’s a lot hard to put the genie back into the bottle.

    Similar to what professors have mentioned in several of my classes, “don’t do anything you don’t mind being posted on the Wall Street Journal.” This includes your online aliases – that stuff will get back to you. Maybe I’m paranoid about posting my photos on Facebook – I even give them a wrong birthday. While I’m probably on one the extreme end of the spectrum, it’ll be interesting how personal data plays out in the hands of corporations.

  4. Katie Peek

    I think one interesting element of this piece is the line "There is also a push towards using big data for more data oriented decision within organizations with the hope that it will result in better business outcomes."

    It's clear that, as detailed above, the universal push right now is towards data-driven everything which, frankly, I find somewhat unsettling (can you tell I'm a humanities major?) I worry that too much data, too much certainty, can help stamp out spontaneity and creativity; that if all the data suggests a new venture or new idea is likely to fail, its creator won't take the risk to actually pursue it and find out. (I will say that the horrendous odds of startup success doesn't seem to be scaring my classmates away from founding their own, but my personal view is this is more an information and recall bias; it's much easier for us to imagine ourselves wildly successful than as abject failures, and I think we hear and recall stories of our peers' success much more than we do stories of the opposite.) I'm not an expert on innovation, and certainly we've never had the kind of data available to guide decision-making that we have today, but I have to wonder on the top innovations of the last 200 years, which big data would have suggested would work out, and which it would have signaled as a "don't even try." Not to reduce this to its simplest construction, but consider the old adage about Edison failing a thousand times in his pursuit of the lightbulb; up until time one thousand and one (for argument's sake) all the data indicated it was unlikely to work.

    Furthermore, as someone who now has 1.5 years (way!) of business school under her belt, and who's gotten to consider how many extraordinarily different ways there are to be effective in business and an effective leader, I do think that decision-making in organizations should always continue to be informed by human judgment, experience, sense of the people involved, etc. In short data has its place, but so does our humanity.

  5. Alex Blankfein

    "Big Data" is not just about better targeted advertising but about better targeting discounts and customer incentive programs to the people who actually care/want the products.

    This is actually not that new. A company in the UK called Nectar has been selling loyalty reward programs for several years. Customers sign up for a "Nectar" card that they use with their purchases at the supermarket, gas station, and department store. The customers can redeem these points (more valuable if they were just standalone loyalty programs), and the participating stores get more a complete profile of its customers (across different spend categories). In this example, sign up is voluntary and everyone benefits.

    When it comes to the online realm and big data, consumers are understandably weary. There is no quid pro quo. Websites get to collect information and bundle this data but consumers seemingly get nothing in return. There needs to be better balance.