首页 > 代码库 > docker安装hive笔记

docker安装hive笔记

前两篇文章介绍了docker的基本命令如何安装hadoop

那么大家会比较了解docker的基本语法的安装过程。那么咱们今天来一起安装一下hive。

安装

1、下载gitHub,地址:https://github.com/prasanthj/docker-hive-on-tez。如果背墙了,可以选择下载zip。进入目录之后就能看见如下内容:

@~/git/github/docker-hive-on-tez-master $ lsDockerfile        datagen.py        hive-log4j.properties    store_sales.sqlLICENSE            hive-0.14        hive-site.xml        store_sales.txtREADME.md        hive-bootstrap.sh    postgresql.conf

2、安装:

docker build --no-cache=true -t local-hive-on-tez .

这是一个漫长的过程,喝一杯咖啡,该干嘛干嘛,几个小时之后回来......

3、进入系统

docker --tls run -i -t -P local-hive-on-tez /etc/hive-bootstrap.sh -bashStarting postgresql server.../2014-12-15 23:12:56 GMT LOG:  database system was interrupted; last known up at 2014-12-15 23:10:11 GMT2014-12-15 23:12:56 GMT LOG:  database system was not properly shut down; automatic recovery in progress2014-12-15 23:12:56 GMT LOG:  redo starts at 0/1782A58

4、查看hive

root@2c1282c522bf:/# hive -f /opt/files/store_sales.sqlLogging initialized using configuration in file:/usr/local/hive-dist/apache-hive-0.15.0-SNAPSHOT-bin/conf/hive-log4j.propertiesSLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.5.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

备注:

为什么下载github?

  因为需要获取他的Dockerfile,好让docker知道它依赖docker-tez,然后在虚拟机执行下载和安装hive,内容如下:

FROM prasanthj/docker-tez:tez-0.5.2  #这是说明依赖什么,下面是安装命令MAINTAINER Prasanth Jayachandran# to configure postgres as hive metastore backendRUN apt-get updateRUN apt-get -yq install vim postgresql-9.3 libpostgresql-jdbc-java# having ADD commands will invalidate the cache forcing hive build from trunk everytime# copy config, sql, data files to /opt/filesRUN mkdir /opt/filesADD hive-site.xml /opt/files/ADD hive-log4j.properties /opt/files/ADD store_sales.* /opt/files/ADD datagen.py /opt/files/# clone and compile hiveENV HIVE_VERSION 0.15.0-SNAPSHOTRUN cd /usr/local && git clone https://github.com/apache/hive.git #在天朝这可能被墙,所以速度非常慢RUN cd /usr/local/hive && /usr/local/maven/bin/mvn clean install -DskipTests -Phadoop-2,distRUN mkdir /usr/local/hive-dist && tar -xf /usr/local/hive/packaging/target/apache-hive-${HIVE_VERSION}-bin.tar.gz -C /usr/local/hive-dist# set hive environmentENV HIVE_HOME /usr/local/hive-dist/apache-hive-${HIVE_VERSION}-binENV HIVE_CONF $HIVE_HOME/confENV PATH $HIVE_HOME/bin:$PATHADD hive-site.xml $HIVE_CONF/hive-site.xmlADD hive-log4j.properties $HIVE_CONF/hive-log4j.properties# zookeeper pulls jline 0.9.94 and hive pulls jline2. This workaround is from HIVE-8609RUN rm $HADOOP_PREFIX/share/hadoop/yarn/lib/jline-0.9.94.jar# add postgresql jdbc jar to classpathRUN ln -s /usr/share/java/postgresql-jdbc4.jar $HIVE_HOME/lib/postgresql-jdbc4.jar# set permissions for hive bootstrap fileADD hive-bootstrap.sh /etc/hive-bootstrap.shRUN chown root:root /etc/hive-bootstrap.shRUN chmod 700 /etc/hive-bootstrap.sh# to avoid psql asking password, set PGPASSWORDENV PGPASSWORD hive# 下面是安装postgresql,这个在国外很流行# To overcome the bug in AUFS that denies postgres permission to read /etc/ssl/private/ssl-cert-snakeoil.key file.# https://github.com/Painted-Fox/docker-postgresql/issues/30# https://github.com/docker/docker/issues/783# To avoid this issue lets disable ssl in postgres.conf. If we really need ssl to encrypt postgres connections we have to fix permissions to /etc/ssl/private directory everytime until AUFS fixes the issueENV POSTGRESQL_MAIN /var/lib/postgresql/9.3/main/ENV POSTGRESQL_CONFIG_FILE $POSTGRESQL_MAIN/postgresql.confENV POSTGRESQL_BIN /usr/lib/postgresql/9.3/bin/postgresADD postgresql.conf $POSTGRESQL_MAINRUN chown postgres:postgres $POSTGRESQL_CONFIG_FILEUSER postgres# create metastore db, hive user and assign privilegesRUN /etc/init.d/postgresql start &&     psql --command "CREATE DATABASE metastore;" &&     psql --command "CREATE USER hive WITH PASSWORD ‘hive‘;" &&      psql --command "ALTER USER hive WITH SUPERUSER;" &&      psql --command "GRANT ALL PRIVILEGES ON DATABASE metastore TO hive;" &&      cd $HIVE_HOME/scripts/metastore/upgrade/postgres/ &&        psql -h localhost -U hive -d metastore -f hive-schema-0.15.0.postgres.sql# revert back to root userUSER root

 

参考:

https://github.com/prasanthj/docker-hive-on-tez

https://github.com/prasanthj/docker-hadoop

https://github.com/prasanthj/docker-tez

docker安装hive笔记