Mapper_parsing_exception error

Elastic Stack Elasticsearch
1

Hi all,

I create indexes on a daily basis using fluentd in Elasticsearch. I don't do any mapping on elasticsearch side. After a while, the related index could not be created in Elasticsearch and I got the following error. There is a template for ilm policy. I deleted the previously created indexes, template and ilm policy. I restarted Elasticsearch and Kibana. When I create an index with an index name that matches the old index pattern and send data to Kibana, I still get the same error. But when I give the index name different, there is no problem. Is this a bug? How can I solve it?

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "object mapping for [Direction] tried to parse field [Direction] as object, but found a concrete value"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "object mapping for [Direction] tried to parse field [Direction] as object, but found a concrete value"
  },
  "status" : 400
}
2

It means that some data in your index have been first created as:

{
  "Direction": {
    "foo": "bar"
  }
}

But new documents are coming as:

{
  "Direction": "bar"
}

Which can not work.

You need to fix your source of data or disable this field so it's not indexed (and not searchable as a consequence).

3

Here is a little further explanation of what is happening

And Here is a sample code how you might fix something like this with an ingest pipeline.

4

Yes, I understand the problem. But there is a situation. My problematic indexes are as follows: test-<date_format>

I deleted all indexes like this: DELETE test-*
NOTE: I am not using any mapping.

I send request from Kibana ui like this, but I still get the same error:

POST test-20230529/_doc
{
   "Direction" : "rq"
}

When I name the index completely different and run the above command, I don't get any errors.

5

What you described in the previous post means that you have a index template that is creating a mapping that matches the index name...so check your templates

Default mapping would not reject that post command... If that is the first document posted

Back to the larger issue ...There is no fix without some choices/ work.

I showed you a sample ingest pipeline that could address the problem, something like that is prob your best approach

Otherwise the the docs that cannot be mapped will continue to be dropped.

Exactly what error do you get when you post that?

6

Yes! It worked. Thanks.
Do pipelines increase resource usage a lot?

7

It depends how complex.
For something like that not much.

You can get some stats

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.