Blank message field

Trying to create a simple, non-filtered pipeline, but Kibana shows the message field as blank
image

Pipeline conf is

input {
  tcp {
    port => 9600
    codec => json
    mode => server
  }
}
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "myindex"
  }
}

When starting Logstash the following message is displayed

[WARN ] 2020-08-10 16:33:02.698 [[main]>worker0] elasticsearch - Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"myindex", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0xbba272>], :response=>{"index"=>{"_index"=>"myindex", "_type"=>"_doc", "_id"=>"oJs32XMBjyfMsZpZ2EjQ", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [message] tried to parse field [message] as object, but found a concrete value"}}}}

Other fields are being mapped, but I would like to see the full message as wel...

Does this help?

This is the mapping of the message field:

      },
        "message" : {
          "properties" : {
            "someField" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            }
          }
        },

Tried to change/add this mapping with

{
  "properties": {
    "message": {
      "type": "object"
    }
  }
}

My point is - I am missing some information on the fields ES is detecting, so I would like to see the full message. Am I in the right direction?

Thank you

If I am reading that correctly then elasticsearch expects the field

[message][someField]

to exist. That makes [message] an object, not a string. You could try

if ![message][someField] { mutate { rename => { "message" => "originalMessage" } } }

Thanks @Badger after applying the filter you suggested, the messages are being displayed on the doc.

However, every line of this message generates a new doc, instead of showing the full message in the same doc. Is there any way we can group these lines into a single message?

** Edit: These docs share a common field, i.e. execution.execid - Is there a way to aggregate all messages into the same doc, based on this ID?

Thanks

I have no idea why that could be happening, so no suggestions on how to fix it.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.