Indexing from hdfs to elasticsearch using Pig

Elastic Stack Elasticsearch
1

A= LOAD 'hdfs://10.70.16.91:8020/travel_stats/travel/stats-1453945200000/part-00001' USING PigStorage('\t');

STORE A INTO 'logs/travels' USING org.elasticsearch.hadoop.pig.EsStorage('es.nodes=xx.xx.xx.xx,xx.xx.xx.xx,xx.xx.xx.xx','es.port=9200','es.index.auto.create = true' );

java.lang.Exception: java.lang.StringIndexOutOfBoundsException: String index out of range: -1

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Please help.

2

You are using ES 2.x with ES-Hadoop 2.0 or 2.1. For ES 2.x, you need to use ES-Hadoop 2.2 (currently rc1) as well.

3

I am using ES 2.1.1 with ES-Hadoop 2.1.1.

Thanks it worked with ES-Hadoop 2.2.0 rc1

4

Cheers. ES-Hadoop 2.1.3 will provide a better exception message.

1 Like