Elastic Search Unclear Mapper Parsing Exception

Elastic Stack Elasticsearch
2

The issue you're encountering is not a bug in Elasticsearch or Graylog, but rather a mismatch between the data type defined in your Elasticsearch mapping and the actual data you're trying to index.

In Elasticsearch, a field can indeed hold multiple values, but the entire array needs to be of the same data type. In your case, the fields file_attribute_ids and connection_src_port are defined as long in your mapping, but you're trying to index arrays of longs (e.g., [5, 12] and 63318, 63320), which is causing the mapper parsing exception.

To fix this issue, you need to ensure that the data you're indexing matches the data type defined in your Elasticsearch mapping. If you expect these fields to contain arrays of longs, you should adjust your log pipeline to transform these fields into arrays of longs before indexing the data into Elasticsearch.

Here's a general approach to fix this issue:

  1. Adjust your log pipeline: Modify your log pipeline to transform the file_attribute_ids and connection_src_port fields into arrays of longs before forwarding the logs to Elasticsearch. This could involve parsing the fields as strings, splitting the strings into arrays, and then converting each element in the arrays to a long.

  2. Update your Elasticsearch mapping: If you have control over the Elasticsearch mapping, you could also consider updating the mapping to expect arrays of longs for these fields. However, this would require reindexing your existing data.

  3. Use a script to transform the data: If you can't adjust your log pipeline or update your Elasticsearch mapping, you could consider using an ingest node with a script processor in Elasticsearch to transform the data as it's being indexed. The script would need to parse the fields as strings, split the strings into arrays, and then convert each element in the arrays to a long.

Please note that the specific steps to implement these solutions would depend on the details of your log pipeline and your Elasticsearch setup. If you need more detailed instructions, please provide more information about your log pipeline and your Elasticsearch mapping.

OpsGPT helped with part of this answer :grinning: