Spring Data Elasticsearch with ES 7.2.1 | GeoPoint mapping failure while indexing

Elastic Stack Elasticsearch
1

Hi,

I am using ES 7.2.1 to store large amount of location based data and querying for near-by locations.
For location coordinates, I am using GeoPoint fields from my java codebase.

ES: 7.2.1
Spring Data Elasticsearch: 4.0.0.DATAES-690-SNAPSHOT
org.elasticsearch: 7.2.1

Template:

    curl -X PUT "localhost:9200/_template/store_locator_template?pretty" -H 'Content-Type: application/json' -d'
    {
      "order": 1,
      "index_patterns": [
        "store_locator_*"
      ],
      "settings": {
      },
      "mappings": {
          "properties": {
          "esId": {
            "type": "keyword"
          },
          "geoPoint": {
            "type": "geo_point"
          },
          "storeName": {
            "type": "keyword"
          }
        }
      }
    }

Entity class:

@Getter
@Setter
@ToString
@EqualsAndHashCode(of = "esId", callSuper = false)
@NoArgsConstructor
@Document(indexName = "store_locator_index", replicas = 0, createIndex = false)
public class EsEntity {

  @Id
  @Field(type = FieldType.Text)
  private String esId;

  @GeoPointField
  private GeoPoint geoPoint;

  @Field(type = FieldType.Text)
  private String storeName;
}

And the code to PUT mapping from Spring:

//clazz -> entity class with @Document annotation

boolean indexCreated = false;
if (!elasticsearchOperations.indexExists(clazz)) {
    indexCreated = elasticsearchOperations.createIndex(clazz);
}
if (indexCreated) {
    elasticsearchOperations.refresh(clazz);
    elasticsearchOperations.putMapping(clazz);   --> Does the MAGIC
}

When trying to insert data via bulkIndex(), I am getting this error:

org.springframework.data.elasticsearch.ElasticsearchException: Bulk indexing has failures. Use ElasticsearchException.getFailedDocuments() for detailed messages [{QObQeXEBqxAg6uMFyeNZ=ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=**mapper [geoPoint] of different type, current_type [geo_point], merged_type [ObjectMapper]]]**}]

Also.......
Everything seems to be working for:
ES 6.4.3
Spring Data Elasticsearch 3.1.X

I am able to put mapping (via template) and insert document with GeoPoint.
The index is generated automatically when doc is inserted via code.
Here's my template:

    curl -X PUT "localhost:9200/_template/store_locator_template?pretty" -H 'Content-Type: application/json' -d'
    {
      "order": 1,
      "index_patterns": [
        "store_locator_*"
      ],
      "settings": {
      },
      "mappings": {
        "store_locator_index": {
          "properties": {
          "esId": {
            "type": "keyword"
          },
          "geoPoint": {
            "type": "geo_point"
          },
          "storeName": {
            "type": "keyword"
          }
        }
       }
      }
    }

Here's the mapping:

    {
      "mapping": {
        "properties": {
          "esId": {
            "type": "keyword"
          },
          "geoPoint": {
            "type": "geo_point"
          }
        }
      }
    }
2

May be print / debug the exact json document which is sent to elasticsearch and share it here?

Note that when using springboot with elasticsearch, you need to be explicit with some transitive dependencies as SpringBoot declares a version 6.4...

Basically you can put this in your pom.xml:

<properties>
  <elasticsearch.version>7.6.2<elasticsearch.version>
</properties>

See documentation here: https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-customize-dependency-versions

3

I have not upgraded to 7.6.2 and the cluster nodes I am using are 7.2.1

Also, here's the mapping generated from Spring Data code:

{
   "esentity":{
      "properties":{
         "esId":{
            "type":"keyword",
            "index":true
         },
         "geoPoint":{
            "type":"geo_point"
         }
      }
   }
}
4

My answer still applies.

Note that when using springboot with elasticsearch, you need to be explicit with some transitive dependencies as SpringBoot declares a version 6.4...

It would be safe to run mvn dependency:tree to check it.

5

I was more asking for the JSON documents that you are indexing. You Javabeans basically but as they are generated as JSON by Spring Data.

6

I was debugging the code, and this is what I suspect is being sent from Spring:

{
   "_class":"jp.co.rakuten.elastic.elasticservice.entity.EsEntity",
   "esId":"XbGJgXEBnOE6CQTS9vZM",
   "geoPoint":{
      "lat":33.93468543777032,
      "lon":131.8429772766326
   }
}

I think there are converters working in place which convert from a GeoPoint type (when set from Java) into a Map type with 2 keys (lat & lon).

7

Could you GET the mapping from elasticsearch? Not the one which is generated by Spring Data but the real one. Something like:

GET store_locator_*/_mapping
8

My answer still applies.

I do get a warning in my Java codebase:

Version mismatch in between Elasticsearch Client and Cluster: 7.6.2 - 7.2.1

Is this okay to live with?

9

This is what is generated from Spring Data side and what I can see in the GET mapping API:

{
  "store_locator_index": {
    "mappings": {
      "properties": {
        "_class": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "esId": {
          "type": "keyword"
        },
        "geoPoint": {
          "type": "geo_point"
        }
      }
    }
  }
}
10

So from elasticsearch point of view, if all what you shared is exact, there is nothing wrong.

DELETE store_locator_index
PUT store_locator_index
{
  "mappings": {
    "properties": {
      "_class": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "esId": {
        "type": "keyword"
      },
      "geoPoint": {
        "type": "geo_point"
      }
    }
  }
}
POST store_locator_index/_doc
{
   "_class":"jp.co.rakuten.elastic.elasticservice.entity.EsEntity",
   "esId":"XbGJgXEBnOE6CQTS9vZM",
   "geoPoint":{
      "lat":33.93468543777032,
      "lon":131.8429772766326
   }
}

Gives:

{
  "_index" : "store_locator_index",
  "_type" : "_doc",
  "_id" : "IiQoiXEBJjDNjw2Cw5no",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

Tested on 7.6.1.

So you need to check if what you shared is really the actual mapping and data.
If it is, you need to ask spring data elasticsearch project probably.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.