The Cloud & Big Data Collide
Analysts in the cloud sphere think that the next few years will see an explosion of Big Data enterprise services. Due to intense competition, Hadoop development has accelerated to the point that Apache recently gave it the “enterprise ready” stamp – though it has been in production with some of the world’s largest companies for years. Combined with high market interest (a influx of new capital), many analysts expect to see a start-up Big Data explosion.
Public cloud vendors (in particular) have advantages supporting Big Data offerings, because it’s simply more efficient to process Big Data where it lives than to process it centrally. That’s why the cloud is an ideal environment for Big Data filtering because much of the unstructured data that enterprises want already lives in public clouds.
Verticals that benefit from Hadoop-based Big Data mining include retail, healthcare & financial services. Each benefits from exploiting Big Data to gain insight into enterprise end user trends. This data intelligence is driving interest in cloud-based Hadoop deployments & start-up Big-Data-as-a-Service (BDaaS) offerings, but exploiting Big Data brings up some very important challenges.
First, there are big privacy perception issues. As BDaaS makes its way into mainstream IT conversations, the mainstream public will also be talking about it, since Big Data is (to a large extent) personal data. 2011 saw one massive data security breach after another that were (in their own right) Big Data backlashes. Beyond security, corporations with access to a person’s public (& private) life will need to be held responsible for how they exploit that information.
Second, one of the biggest obstacles standing between enterprises & Big Data adoption is the lack of skilled & Big Data engineers & developers with Hadoop experience, as well as Data Scientists who can analyze & exploit Big Data intel. So, enterprises that lack the engineering muscle to exploit Hadoop will increasingly take advantage of Big Data mining from professional BDaaS vendors.
- » Which AWS container orchestration platform is best for your organisation? A guide
- » CircleCI aims to further break down the ‘hornet’s nest’ of continuous delivery with EMEA expansion
- » 10 years of DevOps: With the hype cycle moving on – what’s next?
- » Databricks raises $400m in series F funding and tops $6bn valuation
- » Why businesses fail to maximise the value of data visualisation