首页 > 代码库 > [sqoop] sqoop命令

[sqoop] sqoop命令

 

官网文档:http://sqoop.apache.org/docs/1.99.7/user/CommandLineClient.html#delete-link-function

 

一、了解sqoop数据导入的几个重要概念

(1)connector:sqoop2中预定了各种链接,这些链接是一些配置模板。#查看预定制的连接器sqoop:000> show connector+------------------------+---------+------------------------------------------------------------+----------------------+|          Name          | Version |                           Class                            | Supported Directions |+------------------------+---------+------------------------------------------------------------+----------------------+| oracle-jdbc-connector  | 1.99.7  | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO              || sftp-connector         | 1.99.7  | org.apache.sqoop.connector.sftp.SftpConnector              | TO                   || kafka-connector        | 1.99.7  | org.apache.sqoop.connector.kafka.KafkaConnector            | TO                   || kite-connector         | 1.99.7  | org.apache.sqoop.connector.kite.KiteConnector              | FROM/TO              || ftp-connector          | 1.99.7  | org.apache.sqoop.connector.ftp.FtpConnector                | TO                   || hdfs-connector         | 1.99.7  | org.apache.sqoop.connector.hdfs.HdfsConnector              | FROM/TO              || generic-jdbc-connector | 1.99.7  | org.apache.sqoop.connector.jdbc.GenericJdbcConnector       | FROM/TO              |+------------------------+---------+------------------------------------------------------------+----------------------+其中,最基本的是generic-jdbc-connector,是mysql等关系型数据库的连接器。支持数据的从关系型数据库的导入导出。除此之外,支持导入导出的还有:hdfs-connector、kite-connector和oracle-jdbc-connector。仅支持数据导入的连接器有: sftp-connector 、 ftp-connector 以及kafka-connector 。通过这些模板,可以创建出对应数据源的link,比如我们链接mysql,就是使用JDBC的方式进行链接,这时候就从这个generic-jdbc-connector模板继承出一个link。那如果我们需要连接hdfs,则需要使用hdfs-connector模版。(2)link:从connector继承出的对象,用于指定的对数据源的链接。(3)job:指定一个导入导出作业,必须指定数据源和数据目的地,并配置各项job参数,用于提交给mapreduce。

 

二、几个常用的重要命令

#设置set [server|option|truststore]#查看show [server|version|connector|driver|link|job|submission|option|role|principal|privilege]#创建create [link|job|role]#删除delete [link|job|role]#更新update [link|job]#克隆clone [link|job]#执行任务start [job]#结束任务stop [job]#查看任务状态status [job]#启用enable [link|job]#禁用disable [link|job]#授权grant [role|privilege]#回收授权revoke [role|privilege]

 

set option --name verbose --value trueset option --name poll-timeout --value 20000

 

三、查看

 

查看服务器

sqoop:000> show server -allServer host: localhostServer port: 12000Server webapp: sqoop

 

查看版本

sqoop:000> show version -allclient version:  Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb   Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016server version:  Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb   Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016API versions:  [v1]

 

查看内置连接器

sqoop:000> show connector +------------------------+---------+------------------------------------------------------------+----------------------+|          Name          | Version |                           Class                            | Supported Directions |+------------------------+---------+------------------------------------------------------------+----------------------+| oracle-jdbc-connector  | 1.99.7  | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO              || sftp-connector         | 1.99.7  | org.apache.sqoop.connector.sftp.SftpConnector              | TO                   || kafka-connector        | 1.99.7  | org.apache.sqoop.connector.kafka.KafkaConnector            | TO                   || kite-connector         | 1.99.7  | org.apache.sqoop.connector.kite.KiteConnector              | FROM/TO              || ftp-connector          | 1.99.7  | org.apache.sqoop.connector.ftp.FtpConnector                | TO                   || hdfs-connector         | 1.99.7  | org.apache.sqoop.connector.hdfs.HdfsConnector              | FROM/TO              || generic-jdbc-connector | 1.99.7  | org.apache.sqoop.connector.jdbc.GenericJdbcConnector       | FROM/TO              |+------------------------+---------+------------------------------------------------------------+----------------------+

 

sqoop:000> show link0    [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable+------+----------------+---------+| Name | Connector Name | Enabled |+------+----------------+---------++------+----------------+---------+

 

其他查看命令

show server --allshow option --name verboseshow version --allshow connector --all or show connectorshow drivershow link --allshow link --name linkNameshow job --all show job --name jobNameshow submissionshow submission -j jobNameshow submission --job jobName --detail

 

四、创建

创建link

create link --connector connectorName
create link -c connectorName
例如:
create link -connector generic-jdbc-connector
create link -connector hdfs-connector

 

创建job

create job --from fromLinkName --to toLinkName create job -f fromLinkName -t toLinkName

 

五、更新

update link --name linkNameupdate job --name jobName

 

六、删除

delete link --name linkNamedelete job --name jobName

 

七、克隆

clone link --name linkNameclone job --name jobName

 

八、启动

start job --name jobNamestart job --name jobName --synchronous

 

九、停止

stop job --name jobName

 

十、查看任务执行状态

status job --name jobName

 

[sqoop] sqoop命令