The Cloud & Big Data Collide
Analysts in the cloud sphere think that the next few years will see an explosion of Big Data enterprise services. Due to intense competition, Hadoop development has accelerated to the point that Apache recently gave it the “enterprise ready” stamp – though it has been in production with some of the world’s largest companies for years. Combined with high market interest (a influx of new capital), many analysts expect to see a start-up Big Data explosion.
Public cloud vendors (in particular) have advantages supporting Big Data offerings, because it’s simply more efficient to process Big Data where it lives than to process it centrally. That’s why the cloud is an ideal environment for Big Data filtering because much of the unstructured data that enterprises want already lives in public clouds.
Verticals that benefit from Hadoop-based Big Data mining include retail, healthcare & financial services. Each benefits from exploiting Big Data to gain insight into enterprise end user trends. This data intelligence is driving interest in cloud-based Hadoop deployments & start-up Big-Data-as-a-Service (BDaaS) offerings, but exploiting Big Data brings up some very important challenges.
First, there are big privacy perception issues. As BDaaS makes its way into mainstream IT conversations, the mainstream public will also be talking about it, since Big Data is (to a large extent) personal data. 2011 saw one massive data security breach after another that were (in their own right) Big Data backlashes. Beyond security, corporations with access to a person’s public (& private) life will need to be held responsible for how they exploit that information.
Second, one of the biggest obstacles standing between enterprises & Big Data adoption is the lack of skilled & Big Data engineers & developers with Hadoop experience, as well as Data Scientists who can analyze & exploit Big Data intel. So, enterprises that lack the engineering muscle to exploit Hadoop will increasingly take advantage of Big Data mining from professional BDaaS vendors.
- » Redis Labs further changes licensing terms – to make developers happy and keep big cloud vendors at bay
- » Red Hat: On bridging between the first wave of cloud and next generation platforms
- » How ideal DevOps recruitment requires a mix of soft and technical skills
- » Cloudera looks to being a true multi-cloud home and calls out Amazon as primary competitor
- » CloudBees, Google and Linux Foundation launch Continuous Delivery Foundation