首页 > 代码库 > 排查Hive报错:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start of Array expected

排查Hive报错:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start of Array expected

CREATE TABLE json_nested_test (    count string,    usage string,    pkg map<string,string>,    languages array<string>,    store map<string,array<map<string,string>>>)ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDeSTORED AS TEXTFILE;

以上述sql创建表json_nested_test后,查询时发现报错:Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start of Array expected

于是进行如下测试,发现是org.apache.hive.hcatalog.data.JsonSerDe 对复杂类型支持不足造成,例如map<string,array<string>>,这个例子中就是不支持array作为map的value.
CREATE TABLE json_nested_test_openx (    count string,    usage string,    pkg map<string,string>,    languages array<string>,    store map<string,array<map<string,string>>>)ROW FORMAT SERDE org.openx.data.jsonserde.JsonSerDeSTORED AS TEXTFILE;CREATE TABLE s1 (    count string,    usage string)ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDeSTORED AS TEXTFILE;load data local inpath /home/work/s1.txt overwrite into table s1;select * from s1;CREATE TABLE s2 (    count string,    usage string,    pkg map<string,string>)ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDeSTORED AS TEXTFILE;load data local inpath /home/work/s2.txt overwrite into table s2;select * from s2;CREATE TABLE s3 (    count string,    usage string,    pkg map<string,string>,    languages array<string>)ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDeSTORED AS TEXTFILE;load data local inpath /home/work/s3.txt overwrite into table s3;select * from s3;CREATE TABLE s4 (    count string,    usage string,    pkg map<string,string>,    languages array<string>,    store map<string,array<map<string,string>>>)ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDeSTORED AS TEXTFILE;load data local inpath /home/work/s4.txt overwrite into table s4;select * from s4;CREATE TABLE s5 (    store map<string,array<map<string,string>>>)ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDeSTORED AS TEXTFILE;load data local inpath /home/work/s5.txt overwrite into table s5;select * from s5;CREATE TABLE s6 (    store map<string,array<string>>)ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDeSTORED AS TEXTFILE;load data local inpath /home/work/s6.txt overwrite into table s6;select * from s6;

这个Serde hive自带,路径为$HIVE_HOME/hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1.jar,它也存在其它问题:不支持数据文件中的空行

排查Hive报错:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start of Array expected