Geo_shape and unsupported validation_method : IGNORE_MALFORMED

Elastic Stack Elasticsearch
1

Dear All,

I'm using ES 5.2

I would need to specify a field where the BBOX could be wrong, e.g. I could have -180.0000001, ideally wrong BBOX could not be indexed.

Reading https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-bounding-box-query.html
I tried to add
'bbox': {'type': 'geo_shape', 'precision': '10km', 'tree': 'quadtree', 'validation_method': 'IGNORE_MALFORMED'},

But I got:
CommandError:***
'mapper_parsing_exception', 'Mapping definition for [bbox] has unsupported parameters: [validation_method : IGNORE_MALFORMED]')

Ideas?

Thanks,

D

2

On a phone so I can't look at the code. Can you try with the same option but lowercased?

3

Thanks David for replying so quicky!

I also tried:

PUT test1
{
    "settings" : {
        "number_of_shards" : 1
    },
    "mappings" : {
        "type1" : {
            "properties" : {
                "field1" : { "type" : "text" },
                "bbox": {
                  "type": "geo_shape",
                  "precision": "10km",
                  "tree": "quadtree",
                  "validation_method": "ignore_malformed"
                  }
            }
        }
    }
}

an I got:

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Mapping definition for [bbox] has unsupported parameters:  [validation_method : ignore_malformed]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Failed to parse mapping [type1]: Mapping definition for [bbox] has unsupported parameters:  [validation_method : ignore_malformed]",
    "caused_by": {
      "type": "mapper_parsing_exception",
      "reason": "Mapping definition for [bbox] has unsupported parameters:  [validation_method : ignore_malformed]"
    }
  },
  "status": 400
}

The same using:
"validation_method": "COERCE" or "validation_method": "coerce":
"reason": "Mapping definition for [bbox1] has unsupported parameters: [validation_method : COERCE]"

Or the deprecated (5.0.0) ignore_malformed

Removing
"validation_method": "ignore_malformed"

Of course it works, but while indexing I have:

'error': {'type': 'mapper_parsing_exception', 'caused_by': {'type': 'invalid_shape_exception', 'reason': 'Bad X value 180.083333 is not in boundary Rect(minX=-180.0,maxX=180.0,minY=-90.0,maxY=90.0)'}, 'reason': 'failed to parse [bbox]'}, '_type': 'layer'}}, {'index': {'status': 400, '_index': 'myindex', '_id': '10877', 'error': {'type': 'mapper_parsing_exception', 'caused_by': {'type': 'invalid_shape_exception', 'reason': 'Bad X value 180.083333 is not in boundary Rect(minX=-180.0,maxX=180.0,minY=-90.0,maxY=90.0)'}, 'reason': 'failed to parse [bbox]'}, '_type': 'layer'}}])

Because some BBOX could be wrong and I simply don't want to index that field only.

D

4

This option is only at query time. It's not part of the mapping.

For mapping read: https://www.elastic.co/guide/en/elasticsearch/reference/5.2/geo-point.html

5

Thanks David.
I still don't get how it's possibile to query malformed BBOX if I cannot insert them.
Isn't it more useful to be able to skip only the malformed field instead of skipping the entire record?

6

May be open a feature request on GitHub about your needs?

Ignore malformed still exist in mapping: https://www.elastic.co/guide/en/elasticsearch/reference/current/ignore-malformed.html

7

I just created it: https://github.com/elastic/elasticsearch/issues/23747
Let's see what the team will decide.
Thanks!

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.