Here are the recommended specifications for DataNode/TaskTrackers in a balanced Hadoop cluster:

1. 12-24 1-4TB hard disks in a JBOD (Just a Bunch Of Disks) configuration

2. 2 quad-/hex-/octo-core CPUs, running at least 2-2.5GHz

3. 64-512GB of RAM

4. Bonded Gigabit Ethernet or 10Gigabit Ethernet (the more storage density, the higher the network throughput needed)



Here are the recommended specifications for NameNode/JobTracker/Standby NameNode nodes. The drive count will fluctuate depending on the amount of redundancy:

1. 4–6 1TB hard disks in a JBOD configuration (1 for the OS, 2 for the FS image [RAID 1], 1 for Apache ZooKeeper, and 1 for Journal node)

2. 2 quad-/hex-/octo-core CPUs, running at least 2-2.5GHz

3. 64-128GB of RAM

4. Bonded Gigabit Ethernet or 10Gigabit Ethernet

Below is a list of various hardware configurations for different workloads, including our original “balanced” recommendation:

1. Light Processing Configuration (1U/machine): Two hex-core CPUs, 24-64GB memory, and 8 disk drives (1TB or 2TB)

2. Balanced Compute Configuration (1U/machine): Two hex-core CPUs, 48-128GB memory, and 12 – 16 disk drives (1TB or 2TB) directly attached using the motherboard controller. These are often available as twins with two motherboards and 24 drives in a single 2U cabinet.

3. Storage Heavy Configuration (2U/machine): Two hex-core CPUs, 48-96GB memory, and 16-24 disk drives (2TB – 4TB). This configuration will cause high network traffic in case of multiple node/rack failures.

4. Compute Intensive Configuration (2U/machine): Two hex-core CPUs, 64-512GB memory, and 4-8 disk drives (1TB or 2TB)

本文出自 “Ilovecat(个人笔记)” 博客，请务必保留此出处http://hj192837.blog.51cto.com/655995/1567351

hadoop hardware recommendation