Mapping with allow_malformed doesn't seem to be applied

Elastic Stack Elasticsearch
1

Hi,

The mapping on my index doesn't seem to be respected. One of the properties, bytes_read is defined as an integer in the mapping. But I have also enabled allow_malformed in the index settings, in the hope of salvaging data that doesn't fit the mapping.

This is (a part of) the index effective mapping, as reported in kibana (Settings > Index Management > Mapping)

{
  "mapping": {
    "properties": {
      "bytes_read": {
        "type": "integer"
      }
    }
  }
}

And (a part of) the index settings:

{
  "settings": {
    "index": {
      "mapping": {
        "ignore_malformed": "true"
      }
    }
  }
}

A new index is created everyday. The mapping is consistent throughout the indices, but the value of bytes_read is still mapped as a string, according to kibana:

14

Also, I can't search or use it like a number anyway. Older indices didn't have the correct mapping, and probably assigned the type string/keyword to bytes_read, but it should now be fixed.

tldr: Why is my bytes_read not stored as an integer?

2

If you want string delimited numeric values to be indexed as numerics, try using the coerce setting.

3

What are you referring to by "string delimited numeric values"?

I've just checked in our development cluster, with the same mapping and same "allow_malformed" settings, bytes_read is properly parsed as an integer, even though I don't have that coerce setting set. Surely it must be something else I overlooked.

4

It turns out I had to refresh the index pattern in kibana for it to notice the new data type. I'm left with data type conflicts now, which renders search unusable. Apparently reindexing can solve the issue.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.