my env:
cloudera hadoop 6.3
es 7.10
spark 2.4.0
elasticsearch-spark-20_2.11-7.10.0.jar
my code very simple:
import org.elasticsearch.spark.rdd.api.java.JavaEsSpark;
JavaEsSpark.saveToEs(rdd_tJava_1, "nyc_yellow_taxi/docs");
the output is as expected, the record count is the same. but when inspecting the records, there are problems, some columns got nulled. all of the records from that colums.
so columns:
pulocationid, vendorid, dolocationid, dolocationid
got nulled.
example here a record in hadoop:
and here the output in elastic:
any idea guys?