使用HDFS上的json格式数据建立hive表

数据库  /  管理员 发布于 5年前   628

使用HDFS上的json格式数据建立hive表

1 下载对应版本的 jar包 

http://mvnrepository.com/artifact/org.apache.hive.hcatalog/hive-hcatalog-core

add jar  /home/xxx/hive-hcatalog-core-2.1.0.jar

2.创建表

CREATE TABLE json_table(id bigint, sourceType int)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;

create external table flume(
id string,
id_user string,
id_other string,
city string,
page string,
temporary_id string,
behavior string,
time string,
origin string,
value string,
bankid string,
cardid string,
card_name string,
loan_id string,
loan_name string
)ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;

如果报错:

FAILED: 
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.apache.hive.hcatalog.data.JsonSerDe

将目录:

/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive-hcatalog/share/hcatalog

下的jar包:

hive-hcatalog-core.jar

通过命令:

add jar /opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar

添加到hive的class path

加载数据

load data local inpath '/home/xxx/json.txt' into table json_table;

add jar /tmp/hive-hcatalog-core-2.3.0.jar