首页 > 代码库 > centOS 64位下编译hadoop2.4.1

centOS 64位下编译hadoop2.4.1

2024-07-18 11:11:34   226人阅读

1. 安装JDK


a.      解压jdk

[bigdata001@Yuxi software]$ tar -xvzf jdk-7u60-linux-x64.tar.gz

b.      设置环境变量

PATH=$PATH:$HOME/bin
export JAVA_HOME=/home/bigdata001/BigDataPlatform/jdk1.7.0_60
export JAVA_BIN=$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin
export PATH

c.      使之生效

[bigdata001@Yuxi ~]$ source .bash_profile

d.      验证

[bigdata001@Yuxi ~]$ java -version
java version "1.7.0_60"
Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)

2.   安装Maven

a.      解压

[bigdata001@Yuxi software]$ tar -xvzf apache-maven-3.2.3-bin.tar.gz

b.      配置环境变量

修改系统环境变量有两种方式，修改/etc/profile，或者在/etc/profile.d/下添加定制的shell文件，鉴于profile文件的重要性，尽量不要在profile文件里添加内容，官方建议采用第二种，以保证profile文件的绝对安全。
下面采用第二种方式：

[root@Yuxi etc]# cd /etc/profile.d/
[root@Yuxi profile.d]# touch maven.sh

    向maven.sh中添加如下内容

export MAVEN_HOME='/home/bigdata001/BigDataPlatform/apache-maven-3.2.3'
export PATH=$MAVEN_HOME/bin:$PATH

c.      使之生效

[root@Yuxi profile.d]# source /etc/profile

d.      验证安装

Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-12T04:58:10+08:00)
Maven home: /home/bigdata001/BigDataPlatform/apache-maven-3.2.3
Java version: 1.7.0_60, vendor: Oracle Corporation
Java home: /home/bigdata001/BigDataPlatform/jdk1.7.0_60/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-431.el6.x86_64", arch: "amd64", family: "unix"

3.   安装Protobuf


a.      安装依赖软件

[root@Yuxi ~]# yum install gcc*

b.      解压缩（注意压缩文件后缀， maven安装包是—gzip文件，解压时需加–z）

[bigdata001@Yuxi software]$ tar -xvf protobuf-2.5.0.tar.bz2

c.      安装

[bigdata001@Yuxi BigDataPlatform]$ cd protobuf-2.5.0/
[bigdata001@Yuxi protobuf-2.5.0]$ ./configure
[bigdata001@Yuxi protobuf-2.5.0]$ make
[bigdata001@Yuxi protobuf-2.5.0]$ make check
[root@Yuxi protobuf-2.5.0]# make install


d.      验证安装

[root@Yuxi protobuf-2.5.0]# protoc --version
libprotoc 2.5.0

4.    编译Hadoop

a.      解压缩

[bigdata001@Yuxi software]$ tar -xvzf hadoop-2.4.1-src.tar.gz

b.      修改Maven配置文件

    切换为root权限并修改文件

[root@Yuxi BigDataPlatform]# gedit apache-maven-3.2.3/conf/settings.xml


     增加profile内容

<profile>
      <id>jdk-1.7</id>
      <activation>
        <jdk>1.7</jdk>
      </activation>
      <repositories>
        <repository>
          <id>nexus</id>
          <name>local private nexus</name>
          <url>http://maven.oschina.net/content/groups/public/</url>
          <releases>
            <enabled>true</enabled>
          </releases>
          <snapshots>
            <enabled>false</enabled>
          </snapshots>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>nexus</id>
          <name>local private nexus</name>
          <url>http://maven.oschina.net/content/groups/public/</url>
          <releases>
            <enabled>true</enabled>
          </releases>
          <snapshots>
            <enabled>false</enabled>
          </snapshots>
        </pluginRepository>
      </pluginRepositories>
    </profile>

c.      安装依赖包

[root@Yuxi hadoop-2.4.1-src]# yum install ncurses-devel
[root@Yuxi hadoop-2.4.1-src]# yum install openssl-devel
[root@Yuxi hadoop-2.4.1-src]# yum install cmake

d.      由于编译依赖的组件已经安装完成，所以直接执行编译命令即可


[root@Yuxi hadoop-2.4.1-src]# mvn package -Pdist,native -DskipTests -Dtar
    经过的漫长等待，hadoop编译结束，效果如图




e.      验证编译

    编译后的文件存在于
/home/bigdata001/BigDataPlatform/hadoop-2.4.1-src/hadoop-dist/target

f.      验证版本号

[bigdata001@Yuxi hadoop-2.4.1]$ bin/hadoop version

Hadoop 2.4.1
Subversion Unknown -r Unknown
Compiled by root on 2014-08-18T10:51Z
Compiled with protoc 2.5.0
From source with checksum bb7ac0a3c73dc131f4844b873c74b630
This command was run using /home/bigdata001/BigDataPlatform/hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1/share/hadoop/common/hadoop-common-2.4.1.jar

g.      验证hadoop lib 的位数

[bigdata001@Yuxi hadoop-2.4.1]$ file lib//native/*

lib//native/libhadoop.a:        current ar archive
lib//native/libhadooppipes.a:   current ar archive
lib//native/libhadoop.so:       symbolic link to `libhadoop.so.1.0.0'
lib//native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
lib//native/libhadooputils.a:   current ar archive
lib//native/libhdfs.a:          current ar archive
lib//native/libhdfs.so:         symbolic link to `libhdfs.so.0.0.0'
lib//native/libhdfs.so.0.0.0:   ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped








联系
我们