Hadoop is dead thanks to EMC, long live to Hadoop

Today EMC announced the launch of Pivotal HD, their new version of Greenplum HD. Most of the underlying detail are not know and neither the pricing, but it’s really a game changer. Technologically it’s just Greenplum DB on top of HDFS. In fact we can say it’s just Greenplum DB. It feature an awesome integration with HDFS but that’s it. I believe Greenplum DB will be faster than Pivotal HD on a traditional data warehouse workload. Nothing new here then. For Hadoop, it’s a new day.

And what a day. Hadoop, with the map reduce part, was build on the idea to throw a lot of cheap hardware to solve a problem. Smart was never associated with Hadoop. EMC did just put a brain in the hadoop body. In the presentation there was a window function example. A WINDOW FUNCTION !!! It has more expressiveness than all the MySQL based data store or even SQL Server. More expressiveness mean more efficient data scientist and more business results.

The sad part will be the price. The being for Hadoop was to have a cheap software running on cheap hardware. We don’t know the price of Pivotal HD but it will not be cheap. It will not be cheaper than Greenplum DB (it’s basically Greenplum DB + Greenplum HD in the same box). Last price tag for Greenplum is old but let’s say we can expect at least 10-20k$/TB of user data. It’s not really super cheap anymore when software is more expensive than hardware.