首页 > 代码库 > hadoop-1.2.1 伪分布配置

hadoop-1.2.1 伪分布配置

首先JDK安装及相关环境变量配置

# Java environment settingJAVA_HOME=/usr/java/defaultCLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jarPATH=$JAVA_HOME/bin:$PATHJRE_HOME=$JAVA_HOME/jreexport JAVA_HOME CLASSPATH PATH JRE_HOME# Maven configurationM2_HOME=/usr/local/apache-maven/defaultPATH=$PATH:$M2_HOME/bin# hadoop settingHADOOP_HOME=/home/asn/opt/hadoop-1.2.1PATH=$PATH:$HADOOP_HOME/bin

 

1. 修改主机名

[asn@vm-centos6-asn opt]$ cat /etc/sysconfig/networkNETWORKING=yesHOSTNAME=vm-centos6-asn

  在/etc/hosts文件中,添加主机名映射

[root@vm-centos6-asn ~]# cat /etc/hosts127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 vm-centos6-asn::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

 

2. hadoop配置

1)core-site.xml

 1 <?xml version="1.0"?> 2 <?xml-stylesheet type="text/xsl" href="http://www.mamicode.com/configuration.xsl"?> 3  4 <!-- Put site-specific property overrides in this file. --> 5  6 <configuration> 7     <property> 8         <name>fs.default.name</name> 9         <value>hdfs://192.168.56.101:9000</value>10     </property>11 12     <property>13         <name>hadoop.tmp.dir</name>14         <value>/home/asn/opt/hadoop-tmp</value>15     </property>16 </configuration>

2) hdfs-site.xml

 1 <?xml version="1.0"?> 2 <?xml-stylesheet type="text/xsl" href="http://www.mamicode.com/configuration.xsl"?> 3  4 <!-- Put site-specific property overrides in this file. --> 5  6 <configuration> 7     <property> 8         <name>dfs.data.dir</name> 9         <value>/home/asn/opt/hadoop-data</value>10     </property>11 12     <property>13         <name>dfs.replication</name>14         <value>1</value>15     </property>16 </configuration>

3) mapred-site.xml

 1 <?xml version="1.0"?> 2 <?xml-stylesheet type="text/xsl" href="http://www.mamicode.com/configuration.xsl"?> 3  4 <!-- Put site-specific property overrides in this file. --> 5  6 <configuration> 7  8     <property> 9         <name>mapred.job.tracker</name>10         <value>192.168.56.101:9001</value>11     </property>12 </configuration>

4)  设置hadoop-env.sh文件中的JAVA_HOME环境变量

[asn@vm-centos6-asn conf]$ cat hadoop-env.sh# Set Hadoop-specific environment variables here.
JAVA_HOME环境变量必须配置,其他环境变量可选。 当运行一个分布式配置时,最好配置该文件中的JAVA_HOME环境变量,以便于在远程节点上被正确定义。# The only required environment variable is JAVA_HOME. All others are optional. # When running a distributed configuration it is best to# set JAVA_HOME
in this file, so that it is correctly defined on remote nodes.# The java implementation to use. Required.export JAVA_HOME=/usr/java/default# Extra Java CLASSPATH elements. Optional.# export HADOOP_CLASSPATH=# The maximum amount of heap to use, in MB. Default is 1000.# export HADOOP_HEAPSIZE=2000# Extra Java runtime options. Empty by default.# export HADOOP_OPTS=-server

 

3. 免密码登陆ssh配置

[asn@vm-centos6-asn ~]$ ssh-keygenGenerating public/private rsa key pair.Enter file in which to save the key (/home/asn/.ssh/id_rsa):   私钥存放文件Created directory /home/asn/.ssh.Enter passphrase (empty for no passphrase):    直接回车Enter same passphrase again:  直接回车Your identification has been saved in /home/asn/.ssh/id_rsa. Your public key has been saved in /home/asn/.ssh/id_rsa.pub.   公钥存放文件The key fingerprint is:6e:78:e0:30:10:b5:6b:fb:11:55:55:46:1e:b2:8f:57 asn@vm-centos6-asnThe keys randomart image is:+--[ RSA 2048]----+|  ...     ..oo=  ||   . .   .   = . ||  . .   .   . . E||   . . .     o . ||    = o S   . o  ||   . = =     .   ||    . + +        ||     . +         ||      .          |+-----------------+[asn@vm-centos6-asn ~]$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys   把公钥拷贝到认证文件中

[asn@vm
-centos6-asn ~]$ ssh localhostThe authenticity of host localhost (::1) cant be established.RSA key fingerprint is bd:c8:f0:20:87:be:ee:2f:05:6d:01:14:94:d7:c0:11.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added localhost (RSA) to the list of known hosts. 首次连接提示是否把localhost添加到已知的主机列表中Last login: Tue Dec 23 14:14:43 2014 from localhost[asn@vm-centos6-asn ~]$ exitlogoutConnection to localhost closed.

[asn@vm
-centos6-asn ~]$ ssh localhost 直接免密码登录成功Last login: Tue Dec 23 14:16:14 2014 from localhost[asn@vm-centos6-asn ~]$

 

4. hadoop  namenode -format  格式化名字节点

 

5. start-all.sh 启动

[asn@vm-centos6-asn ~]$ start-all.shstarting namenode, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-namenode-vm-centos6-asn.outlocalhost: starting datanode, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-datanode-vm-centos6-asn.outlocalhost: starting secondarynamenode, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-secondarynamenode-vm-centos6-asn.outstarting jobtracker, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-jobtracker-vm-centos6-asn.outlocalhost: starting tasktracker, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-tasktracker-vm-centos6-asn.out
[asn@vm
-centos6-asn ~]$ jps14406 Jps13823 NameNode14101 SecondaryNameNode14193 JobTracker14319 TaskTracker13942 DataNode

 

hadoop-1.2.1 伪分布配置