Hive - es.update.script property value more than 4000 Chars

In one of the use cases, Length of Hive TBLPROPERTIES - es.update.script property value is more than 4000 characters, so hive query is failing with the below error message in the initial table creation step.

As a workaround, we had to bump up hive metastore table - TABLE_PARAMS column size PARAM_VALUE to 10000. Is there a permanent solution for permanent solution for this issue ?
Options like loading es.update.script file from HDFS ?

Hive Create Table Snippet

STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.update.script':''

Error Msg

INSERT INTO "TABLE_PARAMS" ("PARAM_VALUE","TBL_ID","PARAM_KEY") VALUES (?,?,?) [10:17]
Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(4000

Coming in 6.0 we are introducing support for file-based and stored scripts. This should definitely clear up your issue. As a note, the 6.0 version of the connector will be backwards compatible with previous ES versions, so you should be able to upgrade ES-Hadoop without much disruption to take advantage of this.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.