Logs drop on field conflict

[slavik57 (Slava Shpitalny)]

I am using ELK 5.5.0
When I send a log containing a field with conflicted type it doesn't show up in Kibana but I cannot find any error in the logs.

Example:

curl -H "content-type: application/json" -XPOST 'http://localhost:9191' -d '{
"message": "This is a log line",
"extrafield": 1
}'

curl -H "content-type: application/json" -XPOST 'http://localhost:9191' -d '{
"message": "This is a log line",
"extrafield": "no i am string"
}'

The first log sets the extrafield type to be a number.
The second log is simply ignored without any errors.

Am I missing a place where the errors can be written?

[Christian_Dahlqvist]

It seems like you are missing the index and type in your request. If I run the following:

curl -H "content-type: application/json" -XPOST 'http://localhost:9200/test/doc?pretty' -d '{
  "message": "This is a log line",
  "extrafield": 1
}'

curl -H "content-type: application/json" -XPOST 'http://localhost:9200/test/doc?pretty' -d '{
  "message": "This is a log line",
  "extrafield": "no i am string"
}'

I get an error in the response for the second request:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "failed to parse [extrafield]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "failed to parse [extrafield]",
    "caused_by" : {
      "type" : "illegal_argument_exception",
      "reason" : "For input string: \"no i am string\""
    }
  },
  "status" : 400
}

[slavik57 (Slava Shpitalny)]

Changed to:

curl -H "content-type: application/json" -XPOST 'http://localhost:9191/logs' -d '{
"message": "This is a log line",
"extrafield": 1
}'

curl -H "content-type: application/json" -XPOST 'http://localhost:9191/logs' -d '{
"message": "This is a log line",
"extrafield": "no i am string"
}'

Still same behaviour.

[A_B]

This should show up in Elasticsearch logs as an error.

To get around this you have to set index.mapping.ignore_malformed=true for extrafield

[Christian_Dahlqvist]

You are still not specifying the type, which is causing the requests to fail. Please try running the example I provided.

[slavik57 (Slava Shpitalny)]

I am using logstash so it hid the message for me.
When i sent the request directly to elastic it showed the error

[A_B]

One more thing. If you can avoid having different types of data for the same key, do it. I don't have much control of what format logs are sent to our Elastic Stack and over time this has become a serious pain.

[Christian_Dahlqvist]

If you are using Logstash, you may want to enable the dead letter queue as I believe it would catch this scenario.