首页 > 代码库 > PostgreSQL导出一张表到MySQL

PostgreSQL导出一张表到MySQL

1. 查看PostgreSQL表结构,数据量,是否有特殊字段值

region_il=# select count(*) from result_basic;
  count  
---------
 2611527
(1 row)
region_il=# \d result_basic
             Table "noco_il_16q1.result_basic"
        Column         |         Type          | Modifiers 
-----------------------+-----------------------+-----------
 result_id             | integer               | not null
 ppid                  | character varying(24) | 
 title                 | character varying     | 
 categories            | integer[]             | 
 phone                 | character varying(24) | 
 brand_code            | character varying(24) | 
 street_address        | character varying     | 
 zip_code              | character varying(24) | 
 city                  | character varying     | 
 state                 | character varying(50) | 
 country               | character varying(50) | 
 lat                   | double precision      | 
 lon                   | double precision      | 
 vendor_code           | character varying(24) | 
 vendor_poi_id         | character varying(32) | 
 actual_street_address | character varying     | 
 actual_zip_code       | character varying(24) | 
 extender              | json                  | 
 titles                | json                  | 
Indexes:
    "result_basic_pkey" PRIMARY KEY, btree (result_id)
    "result_basic_vendor_poi_id_idx" btree (vendor_poi_id)
region_il=# \x
Expanded display is on.
region_il=# select * from result_basic where result_id = 2;
-[ RECORD 1 ]---------+---------------------------------------------
result_id             | 2
ppid                  | SUK5QD9B7C16D826B26E5
title                 | ????? ??????? ????????
categories            | {791}
phone                 | 
brand_code            | 
street_address        | ???? ?????? ???? ?? ?????????
zip_code              | 42314
city                  | ???????
state                 | ???????
country               | SAU
lat                   | 24.47893
lon                   | 39.65331
vendor_code           | NOCO
vendor_poi_id         | 1040248605
actual_street_address | 
actual_zip_code       | 
extender              | {"sideOfStreet":"+","mapLinkId":"571732670"}
titles                | 

 2. 导出PostgreSQL表结构

$ pg_dump --verbose --schema-only --table=noco_il_16q1.result_basic region_il -f /usr/local/pgsql/dba/exp/result_basic_schema.sql

 3. 导出PostgreSQL表的所有字段

region_il=# COPY result_basic TO /usr/local/pgsql/dba/exp/result_basic.csv WITH (DELIMITER ^,FORMAT csv,FORCE_QUOTE *);
COPY 2611527

 

 4. 将备份集传到MySQL数据库服务器

$ scp /usr/local/pgsql/dba/exp/result_basic* root@172.16.101.66:/usr/local/mysql/dba/imp/

 

5. 编辑表结构文件修改字段类型并创建MySQl表结构

region_il>source /usr/local/mysql/dba/imp/result_basic_schema.sql;
region_il>desc result_basic;
+-----------------------+-------------+------+-----+---------+-------+
| Field                 | Type        | Null | Key | Default | Extra |
+-----------------------+-------------+------+-----+---------+-------+
| result_id             | int(11)     | NO   | PRI | NULL    |       |
| ppid                  | varchar(24) | YES  |     | NULL    |       |
| title                 | text        | YES  |     | NULL    |       |
| categories            | text        | YES  |     | NULL    |       |
| phone                 | varchar(24) | YES  |     | NULL    |       |
| brand_code            | varchar(24) | YES  |     | NULL    |       |
| street_address        | text        | YES  |     | NULL    |       |
| zip_code              | varchar(24) | YES  |     | NULL    |       |
| city                  | text        | YES  |     | NULL    |       |
| state                 | varchar(50) | YES  |     | NULL    |       |
| country               | varchar(50) | YES  |     | NULL    |       |
| lat                   | double      | YES  |     | NULL    |       |
| lon                   | double      | YES  |     | NULL    |       |
| vendor_code           | varchar(24) | YES  |     | NULL    |       |
| vendor_poi_id         | varchar(32) | YES  | MUL | NULL    |       |
| actual_street_address | text        | YES  |     | NULL    |       |
| actual_zip_code       | varchar(24) | YES  |     | NULL    |       |
| extender              | text        | YES  |     | NULL    |       |
| titles                | text        | YES  |     | NULL    |       |
+-----------------------+-------------+------+-----+---------+-------+
19 rows in set (0.01 sec)

 

 

6. 加载数据到MySQL表中

$ script /dev/null
$ screen -S restore_data
enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 33
Server version: 5.6.23-log MySQL Community Server (GPL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type help; or \h for help. Type \c to clear the current input statement.
region_il>LOAD DATA INFILE /usr/local/mysql/dba/imp/result_basic.csv INTO TABLE result_basic FIELDS TERMINATED BY ^ OPTIONALLY ENCLOSED BY " LINES TERMINATED BY \n;
Query OK, 2611527 rows affected (50 min 3.08 sec)
Records: 2611527  Deleted: 0  Skipped: 0  Warnings: 0

 

 

 

7. 验证导入的数据

region_il>select count(*) from result_basic;
+----------+
| count(*) |
+----------+
|  2611527 |
+----------+
1 row in set (0.94 sec)
region_il>select * from result_basic where result_id = 2\G
*************************** 1. row ***************************
            result_id: 2
                 ppid: SUK5QD9B7C16D826B26E5
                title: ????? ??????? ????????
           categories: {791}
                phone: 
           brand_code: 
       street_address: ???? ?????? ???? ?? ?????????
             zip_code: 42314
                 city: ???????
                state: ???????
              country: SAU
                  lat: 24.47893
                  lon: 39.65331
          vendor_code: NOCO
        vendor_poi_id: 1040248605
actual_street_address: 
      actual_zip_code: 
             extender: {"sideOfStreet":"+","mapLinkId":"571732670"}
               titles: 
1 row in set (0.05 sec)
region_il>show variables like %char%;
+--------------------------+----------------------------------+
| Variable_name            | Value                            |
+--------------------------+----------------------------------+
| character_set_client     | latin1                           |
| character_set_connection | latin1                           |
| character_set_database   | utf8                             |
| character_set_filesystem | binary                           |
| character_set_results    | latin1                           |
| character_set_server     | latin1                           |
| character_set_system     | utf8                             |
| character_sets_dir       | /usr/local/mysql/share/charsets/ |
+--------------------------+----------------------------------+
8 rows in set (0.03 sec)

region_il>set character_set_results = utf8;
Query OK, 0 rows affected (0.02 sec)

region_il>select * from result_basic where result_id = 2\G
*************************** 1. row ***************************
            result_id: 2
                 ppid: SUK5QD9B7C16D826B26E5
                title: ????? ??????? ????????
           categories: {791}
                phone: 
           brand_code: 
       street_address: ???? ?????? ???? ?? ?????????
             zip_code: 42314
                 city: ???????
                state: ???????
              country: SAU
                  lat: 24.47893
                  lon: 39.65331
          vendor_code: NOCO
        vendor_poi_id: 1040248605
actual_street_address: 
      actual_zip_code: 
             extender: {"sideOfStreet":"+","mapLinkId":"571732670"}
               titles: 
1 row in set (0.00 sec)

 

PostgreSQL导出一张表到MySQL