首页 > 代码库 > building hadoop2.4.1 on centos7[在centos7上面构建hadoop2.4.1]
building hadoop2.4.1 on centos7[在centos7上面构建hadoop2.4.1]
本文介绍在centos7上面通过hadoop2.4.1源码构建hadoop distribution 版本,即hadoop的运行版本。
为何要自己building,而不用Apache的distribution 【bin】版本,因为hadoop涉及到Linux系统的底层实现,如:
hadoop fs -ls /
通过 java Native 实现,故应为 native的相关lib。【个人认为的一点】
相关文件下载链接:
文件目录解释:
- hadoop
- hadoop-2.4.1.tar.gz 本人通过在centos7上构建的distribution版本,可以直接下载并配置,然后运行即可,参考文章:
- hadoop-2.4.1-src.tar.gz hadoop的源代码
- relative_tools
- maven 【只需要配置 MAVEN_HADOOP 的环境变量】
- protocolbuf [即:google protocolBUffer],需要自己构建,并检查protocolbuf是否已配置环境变量
- findbugs 需要自己构建,并检查protocolbuf是否已配置环境变量
- Cmake 自己搜索,并构建配置环境变量
Requirements:* Unix System* JDK 1.6+* Maven 3.0 or later* Findbugs 1.3.9 (if running findbugs)* ProtocolBuffer 2.5.0* CMake 2.6 or newer (if compiling native code)* Internet connection for first build (to fetch all Maven and Hadoop dependencies)----------------------------------------------------------------------------------Maven build goals: * Clean : mvn clean * Compile : mvn compile [-Pnative] * Run tests : mvn test [-Pnative] * Create JAR : mvn package * Run findbugs : mvn compile findbugs:findbugs * Run checkstyle : mvn compile checkstyle:checkstyle * Install JAR in M2 cache : mvn install * Deploy JAR to Maven repo : mvn deploy * Run clover : mvn test -Pclover [-DcloverLicenseLocation=${user.name}/.clover.license] * Run Rat : mvn apache-rat:check * Build javadocs : mvn javadoc:javadoc * Build distribution : mvn package [-Pdist][-Pdocs][-Psrc][-Pnative][-Dtar] * Change Hadoop version : mvn versions:set -DnewVersion=NEWVERSION Build options: * Use -Pnative to compile/bundle native code * Use -Pdocs to generate & bundle the documentation in the distribution (using -Pdist) * Use -Psrc to create a project source TAR.GZ * Use -Dtar to create a TAR with the distribution (using -Pdist)----------------------------------------------------------------------------------Importing projects to eclipseWhen you import the project to eclipse, install hadoop-maven-plugins at first. $ cd hadoop-maven-plugins $ mvn installThen, generate eclipse project files. $ mvn eclipse:eclipse -DskipTestsAt last, import to eclipse by specifying the root directory of the project via[File] > [Import] > [Existing Projects into Workspace].----------------------------------------------------------------------------------Building distributions:Create binary distribution without native code and without documentation: $ mvn package -Pdist -DskipTests -DtarCreate binary distribution with native code and with documentation: $ mvn package -Pdist,native,docs -DskipTests -DtarCreate source distribution: $ mvn package -Psrc -DskipTestsCreate source and binary distributions with native code and documentation: $ mvn package -Pdist,native,docs,src -DskipTests -DtarCreate a local staging version of the website (in /tmp/hadoop-site) $ mvn clean site; mvn site:stage -DstagingDirectory=/tmp/hadoop-site
只要上述的Linux系统、cmake、maven、findbuds、protocolbuf、网络连通、java都已构建/安装成功后等条件满足后,就在hadoop源代码目录运行代码区的着色区域
文件下载链接:http://pan.baidu.com/s/1pJJQkPl
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。