Ignore unwanted data from message field multiline logstash

Elastic Stack Logstash
1

I am using multiline input codec in logstash with currently no filter.
Below is a sample of how my output looks.

"@timestamp" => "2017-01-27T12:03:46.189Z",
   "message" => "{\"@timestamp\":\"2017-01-27T11:35:55.477Z\",\"beat\":{\"hostname\":\"ip-10-1-2-9\",\"name\":\"ip-10-1-2-9\",\"version\":\"5.0.0\"},\"input_type\":\"log\",\"message\":\"Jan 13 11:12:13 ip-10-0-0-23 abc [LOGBACK] 00:00:15.767 [INFO ] Outgoing Message\",\"offset\":120923590,\"source\":\"/rescue_109/dataLogging/10.130.2.114/newfile.log\",\"type\":\"core_demo_001\"}\n{\"@timestamp\":\"2017-01-27T11:35:55.477Z\",\"beat\":{\"hostname\":\"ip-10-44-55-9\",\"name\":\"ip-10-1-2-9\",\"version\":\"5.0.0\"},\"input_type\":\"log\",\"message\":\"Jan 13 11:12:13 ip-10-02-33-23 abc ---------------------------\",\"offset\":120923651,\"source\":\"/rescue_109/dataLogging/10.10.20.14/newfile.log\",\"type\":\"core_demo_001\"}\n{\"@timestamp\":\"2017-01-27T11:35:55.477Z\",\"beat\":{\"hostname\":\"ip-10-1-2-9\",\"name\":\"ip-10-1-2-9\",\"version\":\"5.0.0\"},\"input_type\":\"log\",\"message\":\"Jan 13 11:12:13 ip-10-2-3-23 abc ID: 574245\",\"offset\":120923695,\"source\":\"/rescue_109/dataLogging/10.1.40.14/newfile.log\",\"type\":\"core_demo_001\"}\n{\"@timestamp\":\"2017-01-27T11:35:55.477Z\",\"beat\":{\"hostname\":\"ip-10-2-3-9\",\"name\":\"ip-10-3-4-9\",\"version\":\"5.0.0\"},\"input_type\":\"log\",\"message\":\"Jan 13 11:12:13 ip-10-0-0-23 abc Encoding: UTF-8\",\"offset\":120923744,\"source\":\"/rescue_109/dataLogging/10.10.20.14/newfile.log\",\"type\":\"core_demo_001\"}107]\",\"offset\":120924874,\"source\":\"/rescue_109/dataLogging/10.10.20.14/newfile.log\",\"type\":\"core_demo_001\"}",
  "@version" => "1",
      "tags" => [
    [0] "multiline",
    [1] "groked"
]

}

As it is visible that inside my message, there are multiple tags of timestamp, message,beat, hostname etc.
I want to only take the all the hostname field and ignore rest of the unwanted data from main message.

What filter should I use. Should I use mutate or any other filter.

Any help would be appreciated.

2

Why are you using the multiline input? It doesn't look like you need it.

Use a json filter to deserialize the JSON string in the `message' field. If possible, use a json or json_lines codec in your input configuration.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.