首页 > 代码库 > [sqoop] sqoop命令
[sqoop] sqoop命令
官网文档:http://sqoop.apache.org/docs/1.99.7/user/CommandLineClient.html#delete-link-function
一、了解sqoop数据导入的几个重要概念
(1)connector:sqoop2中预定了各种链接,这些链接是一些配置模板。#查看预定制的连接器sqoop:000> show connector+------------------------+---------+------------------------------------------------------------+----------------------+| Name | Version | Class | Supported Directions |+------------------------+---------+------------------------------------------------------------+----------------------+| oracle-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO || sftp-connector | 1.99.7 | org.apache.sqoop.connector.sftp.SftpConnector | TO || kafka-connector | 1.99.7 | org.apache.sqoop.connector.kafka.KafkaConnector | TO || kite-connector | 1.99.7 | org.apache.sqoop.connector.kite.KiteConnector | FROM/TO || ftp-connector | 1.99.7 | org.apache.sqoop.connector.ftp.FtpConnector | TO || hdfs-connector | 1.99.7 | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO || generic-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |+------------------------+---------+------------------------------------------------------------+----------------------+其中,最基本的是generic-jdbc-connector,是mysql等关系型数据库的连接器。支持数据的从关系型数据库的导入导出。除此之外,支持导入导出的还有:hdfs-connector、kite-connector和oracle-jdbc-connector。仅支持数据导入的连接器有: sftp-connector 、 ftp-connector 以及kafka-connector 。通过这些模板,可以创建出对应数据源的link,比如我们链接mysql,就是使用JDBC的方式进行链接,这时候就从这个generic-jdbc-connector模板继承出一个link。那如果我们需要连接hdfs,则需要使用hdfs-connector模版。(2)link:从connector继承出的对象,用于指定的对数据源的链接。(3)job:指定一个导入导出作业,必须指定数据源和数据目的地,并配置各项job参数,用于提交给mapreduce。
二、几个常用的重要命令
#设置set [server|option|truststore]#查看show [server|version|connector|driver|link|job|submission|option|role|principal|privilege]#创建create [link|job|role]#删除delete [link|job|role]#更新update [link|job]#克隆clone [link|job]#执行任务start [job]#结束任务stop [job]#查看任务状态status [job]#启用enable [link|job]#禁用disable [link|job]#授权grant [role|privilege]#回收授权revoke [role|privilege]
set option --name verbose --value trueset option --name poll-timeout --value 20000
三、查看
查看服务器
sqoop:000> show server -allServer host: localhostServer port: 12000Server webapp: sqoop
查看版本
sqoop:000> show version -allclient version: Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016server version: Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016API versions: [v1]
查看内置连接器
sqoop:000> show connector +------------------------+---------+------------------------------------------------------------+----------------------+| Name | Version | Class | Supported Directions |+------------------------+---------+------------------------------------------------------------+----------------------+| oracle-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO || sftp-connector | 1.99.7 | org.apache.sqoop.connector.sftp.SftpConnector | TO || kafka-connector | 1.99.7 | org.apache.sqoop.connector.kafka.KafkaConnector | TO || kite-connector | 1.99.7 | org.apache.sqoop.connector.kite.KiteConnector | FROM/TO || ftp-connector | 1.99.7 | org.apache.sqoop.connector.ftp.FtpConnector | TO || hdfs-connector | 1.99.7 | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO || generic-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |+------------------------+---------+------------------------------------------------------------+----------------------+
sqoop:000> show link0 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable+------+----------------+---------+| Name | Connector Name | Enabled |+------+----------------+---------++------+----------------+---------+
其他查看命令
show server --allshow option --name verboseshow version --allshow connector --all or show connectorshow drivershow link --allshow link --name linkNameshow job --all show job --name jobNameshow submissionshow submission -j jobNameshow submission --job jobName --detail
四、创建
创建link
create link --connector connectorName
create link -c connectorName
例如:
create link -connector generic-jdbc-connector
create link -connector hdfs-connector
创建job
create job --from fromLinkName --to toLinkName create job -f fromLinkName -t toLinkName
五、更新
update link --name linkNameupdate job --name jobName
六、删除
delete link --name linkNamedelete job --name jobName
七、克隆
clone link --name linkNameclone job --name jobName
八、启动
start job --name jobNamestart job --name jobName --synchronous
九、停止
stop job --name jobName
十、查看任务执行状态
status job --name jobName
[sqoop] sqoop命令
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。