首页 > 代码库 > hadoop-1.2.1 伪分布配置
hadoop-1.2.1 伪分布配置
首先JDK安装及相关环境变量配置
# Java environment settingJAVA_HOME=/usr/java/defaultCLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jarPATH=$JAVA_HOME/bin:$PATHJRE_HOME=$JAVA_HOME/jreexport JAVA_HOME CLASSPATH PATH JRE_HOME# Maven configurationM2_HOME=/usr/local/apache-maven/defaultPATH=$PATH:$M2_HOME/bin# hadoop settingHADOOP_HOME=/home/asn/opt/hadoop-1.2.1PATH=$PATH:$HADOOP_HOME/bin
1. 修改主机名
[asn@vm-centos6-asn opt]$ cat /etc/sysconfig/networkNETWORKING=yesHOSTNAME=vm-centos6-asn
在/etc/hosts文件中,添加主机名映射
[root@vm-centos6-asn ~]# cat /etc/hosts127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 vm-centos6-asn::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
2. hadoop配置
1)core-site.xml
1 <?xml version="1.0"?> 2 <?xml-stylesheet type="text/xsl" href="http://www.mamicode.com/configuration.xsl"?> 3 4 <!-- Put site-specific property overrides in this file. --> 5 6 <configuration> 7 <property> 8 <name>fs.default.name</name> 9 <value>hdfs://192.168.56.101:9000</value>10 </property>11 12 <property>13 <name>hadoop.tmp.dir</name>14 <value>/home/asn/opt/hadoop-tmp</value>15 </property>16 </configuration>
2) hdfs-site.xml
1 <?xml version="1.0"?> 2 <?xml-stylesheet type="text/xsl" href="http://www.mamicode.com/configuration.xsl"?> 3 4 <!-- Put site-specific property overrides in this file. --> 5 6 <configuration> 7 <property> 8 <name>dfs.data.dir</name> 9 <value>/home/asn/opt/hadoop-data</value>10 </property>11 12 <property>13 <name>dfs.replication</name>14 <value>1</value>15 </property>16 </configuration>
3) mapred-site.xml
1 <?xml version="1.0"?> 2 <?xml-stylesheet type="text/xsl" href="http://www.mamicode.com/configuration.xsl"?> 3 4 <!-- Put site-specific property overrides in this file. --> 5 6 <configuration> 7 8 <property> 9 <name>mapred.job.tracker</name>10 <value>192.168.56.101:9001</value>11 </property>12 </configuration>
4) 设置hadoop-env.sh文件中的JAVA_HOME环境变量
[asn@vm-centos6-asn conf]$ cat hadoop-env.sh# Set Hadoop-specific environment variables here.
JAVA_HOME环境变量必须配置,其他环境变量可选。 当运行一个分布式配置时,最好配置该文件中的JAVA_HOME环境变量,以便于在远程节点上被正确定义。# The only required environment variable is JAVA_HOME. All others are optional. # When running a distributed configuration it is best to# set JAVA_HOME in this file, so that it is correctly defined on remote nodes.# The java implementation to use. Required.export JAVA_HOME=/usr/java/default# Extra Java CLASSPATH elements. Optional.# export HADOOP_CLASSPATH=# The maximum amount of heap to use, in MB. Default is 1000.# export HADOOP_HEAPSIZE=2000# Extra Java runtime options. Empty by default.# export HADOOP_OPTS=-server
3. 免密码登陆ssh配置
[asn@vm-centos6-asn ~]$ ssh-keygenGenerating public/private rsa key pair.Enter file in which to save the key (/home/asn/.ssh/id_rsa): 私钥存放文件Created directory ‘/home/asn/.ssh‘.Enter passphrase (empty for no passphrase): 直接回车Enter same passphrase again: 直接回车Your identification has been saved in /home/asn/.ssh/id_rsa. Your public key has been saved in /home/asn/.ssh/id_rsa.pub. 公钥存放文件The key fingerprint is:6e:78:e0:30:10:b5:6b:fb:11:55:55:46:1e:b2:8f:57 asn@vm-centos6-asnThe key‘s randomart image is:+--[ RSA 2048]----+| ... ..oo= || . . . = . || . . . . . E|| . . . o . || = o S . o || . = = . || . + + || . + || . |+-----------------+[asn@vm-centos6-asn ~]$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys 把公钥拷贝到认证文件中
[asn@vm-centos6-asn ~]$ ssh localhostThe authenticity of host ‘localhost (::1)‘ can‘t be established.RSA key fingerprint is bd:c8:f0:20:87:be:ee:2f:05:6d:01:14:94:d7:c0:11.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added ‘localhost‘ (RSA) to the list of known hosts. 首次连接提示是否把localhost添加到已知的主机列表中Last login: Tue Dec 23 14:14:43 2014 from localhost[asn@vm-centos6-asn ~]$ exitlogoutConnection to localhost closed.
[asn@vm-centos6-asn ~]$ ssh localhost 直接免密码登录成功Last login: Tue Dec 23 14:16:14 2014 from localhost[asn@vm-centos6-asn ~]$
4. hadoop namenode -format 格式化名字节点
5. start-all.sh 启动
[asn@vm-centos6-asn ~]$ start-all.shstarting namenode, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-namenode-vm-centos6-asn.outlocalhost: starting datanode, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-datanode-vm-centos6-asn.outlocalhost: starting secondarynamenode, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-secondarynamenode-vm-centos6-asn.outstarting jobtracker, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-jobtracker-vm-centos6-asn.outlocalhost: starting tasktracker, logging to /home/asn/opt/hadoop-1.2.1/libexec/../logs/hadoop-asn-tasktracker-vm-centos6-asn.out
[asn@vm-centos6-asn ~]$ jps14406 Jps13823 NameNode14101 SecondaryNameNode14193 JobTracker14319 TaskTracker13942 DataNode
hadoop-1.2.1 伪分布配置
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。