Dynamic Mapping Hive issue

Elastic Stack Elasticsearch
1

Index in elasticsearch have dynamic feilds
For example I have following mapping
"Person_ID": {
"type": "string"
},
"First_name": {
"type": "string"
},
"last_name": {
"type": "string"
}

Some documents have last_name , Some documents doesn't have last name
When I query from Hive last_name is always returned as null for every document

Hive external table defination is
CREATE EXTERNAL TABLE elasticsearch ( Person_ID STRING, First_name String , last_name String)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.nodes' = '123.456.789.111', 'es.port' = '9200', 'es.resource' = 'person/data','es.query' = '?q=*');

Any help will be appreciated

2

Could you provide some trace level logging for the job workers over a limited dataset? This should highlight the actual responses the connector is getting from the framework, and help debug the issue.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.