Hello All,
We have been trying to push the data to Elasticsearch cluster by using the following method in spark "JavaEsSpark.saveToEs"
I have defined the "rd_innovation_config1" index mappings and settings in ES cluster and below is the code snipet of JavaRDD to save the document in Elasticsearch
JavaRDD<InnovationConfig> iConfigRDD = initParquetDf.javaRDD().map(y -> InnovationConfig.builder()
.isoCntryCode(y.getString(0))
.chrID(y.getString(1))
.chrValID(y.getString(2))
.processType(y.getString(4))
.activeID(y.getString(5))
.releaseInd(String.valueOf(y.getLong(6)))
.build());
JavaEsSpark.saveToEs(iConfigRDD, "rd_innovation_config1/innovation_config");
Since InnovationConfig.builder() has all the pojo's and in which each has set @jsonproperty as like below,
package com.ogrds.datamodel.es.rdinnovationconfig;
import java.io.Serializable;
import com.fasterxml.jackson.annotation.JsonClassDescription;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.modeliosoft.modelio.javadesigner.annotations.objid;
import lombok.Builder;
import lombok.Getter;
@objid ("0dcc4051-9213-435e-88da-50c7d6433b8f")
@JsonClassDescription("rd_innovation_config")
@Builder
@Getter
public class InnovationConfig implements Serializable {
@objid ("99e5975b-401e-4aa6-8736-26e21348e4b5")
@JsonProperty("ACTIVE_ID")
public String activeID;
@objid ("b71d26fd-4203-4ffc-a93a-ceef95aaf91e")
@JsonProperty("CHR_ID")
public String chrID;
@objid ("2fba5712-ff54-4375-a3a2-e94a8a9dbf48")
@JsonProperty("CHR_VAL_ID")
public String chrValID;
@objid ("ef742f1e-ff29-42a1-afca-172049e87b9e")
@JsonProperty("ISO_COUNTRY_CODE")
public String isoCntryCode;
@objid ("f402da18-5123-43c2-b93f-86da44cd787a")
@JsonProperty("PROCESS_TYPE")
public String processType;
@objid ("e2b7beb8-119c-4663-a98d-2055a62c70e7")
@JsonProperty("RELEASE_IND")
public String releaseInd;
}
But after saveToES operation done the values are not saved to the right field in the pre-defined mapping and seems @jsonproperty are not able to serialize properly???
Please correct me if I am doing anything wrong and let me know your thoughts..
Thanks,
Ganeshbabu R