Azure HDInsight

Azure HDInsight is Microsoft‘s distribution of Hadoop. The Azure HDInsight ecosystem includes the following features/components: Pig, Hive, Hbase, Sqoop, Oozie, Ambari, Microsoft Avro Library, YARN, Cluster Dashboard and Tez.

Apart from the above listed features/components, there are a few other components which enable reporting and analytics on top of data present in Azure HDInsight. These components include the following:

• Microsoft Power Query Add-in for Excel
• Microsoft Hive ODBC Connector

More information: http://azure.microsoft.com/en-us/documentation/articles/hdinsight-introduction

Here are few highlights of Azure HDInsight:

• Azure HDInsight is based on Hortonworks Data Platform.
• Azure HDInsight enables Apache Hadoop as a service in Microsoft Azure cloud thereby leveraging all the benefits of cloud computing.
• Azure HDInsight offers strong support for PowerShell via HDInsight PowerShell Cmdlets.
• Windows Azure and HDInsight PowerShell Cmdlets can be used to perform various activities including uploading, downloading, movement of data to and from Azure Blob Storage and On-Premise file systems, configuring/executing/post-processing jobs on HDInsight, and other related activities.
• Azure HDInsight being a Hadoop service in the cloud, one can provision a cluster, process the data, and destroy the cluster and pay for only the resources used.
• Microsoft also offers an HDInsight Emulator which allows developers to explore HDInsight on premise without requiring an Azure Account.

Links & Additional Information

• HDInsight Versions
• Windows Azure Price Calculator
• HDInsight Training
• HDInsight Forum

Getting Started

• Getting started with HDInsight
• Sign-up for a free trial account and get started with Azure HDInsight
• Download HDInsight Emulator and get started on-premise

Cloudera

Cloudera was the first company to be formed to build enterprise solutions based on Hadoop. Cloudera has a Hadoop distribution known as Cloudera‘s Distribution for Hadoop (CDH). Here is a simplified representation of Cloudera‘s Hadoop Ecosystem.

Source: http://www.cloudera.com/content/cloudera/en/products-and-services/cdh.html

Cloudera‘s Hadoop Ecosystem includes the following features/components: Apache Avro, Apache Crunch, Apache DataFu, Apache Flume, Apache Hadoop, Apache Hbase, Apache Hive, Hue, Cloudera Impala, Kite SDK (formerly CDK), LLAMA, Apache Mahout, Apache Oozie, Parquet, Apache Pig, Cloudera Search, Apache Sentry, Apache Spark, Apache Sqoop and Apache ZooKeeper.

More Information: http://www.cloudera.com/content/dev-center/en/home/developer-admin-resources/cdh-components.html

Here are few highlights of CDH:

• CDH can be deployed on-premise as well as in the cloud.
• Cloudera manager simplifies the deployment and management of Hadoop and other components in Cloudera‘s Hadoop Ecosystem.
• Cloudera has an Enterprise edition - Cloudera Enterprise, and is proprietary. There three variations of this - Basic, Flex, and Data Hub.
• Express edition is available via a free download.
• Cloudera Enterprise Data Hub edition is supported on AWS cloud.

Links & Additional Information

• Cloudera Product Comparison
• Cloudera Versions
• Cloudera Training
• Cloudera Forum

Getting Started

• Cloudera Training/Learning Resources
• Try Cloudera Live
• Download Cloudera QuickStart VM

Hortonworks

Hortonworks has a Hadoop distribution known as Hortonworks Data Platform (HDP). Here is a simplified representation of Hortonworks Data Platform.

Source: http://hortonworks.com/hdp/

Hortonworks Data Platform includes the following features/components: Apache Hadoop, Apache Pig, Apache Hive, Apache Hbase, Apache ZooKeeper, Apache Oozie, Apache Sqoop, Apache Flume, Apache Ambari, Hue, Apache Mahout, Apache Knox, Apache Storm, Apache Tez, Apache Phoenix, Apache Accumulo and Apache Falcon.

More Information: http://hortonworks.com/hadoop/

Here are few highlights of Hortonworks Data Platform:

• Can be deployed on-premise as well as in the cloud.
• Supports deploying on Linux as well Windows platforms.
• HDP is built in open through Apache Projects.

Links & Additional Information

• Hortonworks Documentation
• HDP as part of a modern data architecture
• Hortonworks Training
• Hortonworks Forums

Getting Started

• Getting started with Hortonworks Data Platform
• Hortonworks Tutorials - Learn Hadoop on Sandbox
• Download ready to use VMs

Amazon Elastic Map Reduce (EMR)

Amazon Web Services (AWS) Elastic MapReduce (EMR) was among the first Hadoop offerings available in the market. Here is a high-level architecture/job flow of Amazon EMR.

Source: http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-what-is-emr.html

Amazon EMR contains most of the popular features/components like Hive, Pig, HBase, DistCp, Ganglia, etc. integrated into it.

Here are few highlights of Amazon EMR:

• EMR is a Hadoop distribution in the Cloud.
• Leverages AWS‘s Elastic Compute Cloud (EC2) for computation.
• Leverages AWS‘s Simple Storage Service (S3) for storage.
• Is tightly integrated with other AWS services.
• Deployment and Management is simplified using AWS Management Console and AWS Toolkit.

Links & Additional Information

• Amazon‘s Elastic MapReduce (EMR)
• AWS Simple Monthly Calculator
• Amazon EMR Articles/Tutorials
• Amazon EMR Forum

Getting Started

• Get Started with EMR
• EMR Step by Step Tutorial

MapR

MapR is another major distribution available in the market. Below is a simplified architecture of MapR Data Platform.

Source: http://www.mapr.com/products/product-overview/overview

Here are few highlights of MapR:

• MapR is available in the cloud through some of the leading cloud providers - Amazon Web Services (AWS), Google Compute Engine, CenturyLink Technology Solutions, and OpenStack.
• MapR integrates/supports more than 20 open source projects.
• MapR supports multiple versions of various individual projects it integrates into its data platform. This gives the users flexibility to migrate to the subsequent/latest versions at their own pace.

Links & Additional Information

• MapR Editions
• MapR Cloud Solutions
• MapR Academy/Trainings
• MapR Forums

Getting Started

• MapR Videos
• MapR Sandbox - Fastest On-Ramp to Hadoop

Apart from the distributions listed above, there are various other distributions available in the market from leading providers like Intel, Oracle, HP, and many others.