# Grok on logstash not working

**URL:** <https://discuss.elastic.co/t/grok-on-logstash-not-working/334172>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [May 24, 2023, 6:17am UTC](https://discuss.elastic.co/t/grok-on-logstash-not-working/334172 "2023-05-24T06:17:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![nitisha](https://avatars.discourse-cdn.com/v4/letter/n/6de8d8/32.png) [@nitisha](https://discuss.elastic.co/u/nitisha)\
**Post date:** [May 24, 2023, 6:17am UTC](https://discuss.elastic.co/t/grok-on-logstash-not-working/334172/1 "2023-05-24T06:17:13Z")

</div>

Hi,

I am monitoring CPU metric threshold of containers in our infra using elasticsearch using Connector as "Server Log" which defaults to kibana.log.

I have created the following logstash configuration file to intercept the messages from kibana.log.

```auto
input {
    file {
        type => "json"
        codec => "json"
        path => "/var/log/kibana/kibana.log"
        start_position => "beginning"
    }
}

filter {
   grok{
      match => [
         "message", "Server log: CP1 containers CPU metric threshold - %{WORD:container_id} is in a state of %{DATA:sev_level} %{DATA:container_message} for %{GREEDYDATA:logMessage}"
        ]
   }
}

output {
  elasticsearch { hosts => ["xxxxx"]
  index => "docker-logs-%{+xxxx.ww}"
 }
  stdout { codec => rubydebug }
}

```

When I debug the logstash .conf file, I see the logs getting ingested as expected :

```auto
{
           "host" => {
        "name" => "infra-elasticvm-02"
    },
    "transaction" => {
        "id" => "177d25f28df3bd5c"
    },
           "type" => "json",
     "logMessage" => "Server log: CP1 containers CPU metric threshold - 52e5c413c5eba2b161ade239aa124d2041d5bdca76deb5c5260b6023ac608464 is in a state of ALERT CPU usage is 17% in the last 1 min for 52e5c413c5eba2b161ade239aa124d2041d5bdca76deb5c5260b6023ac608464. Alert when > 10%.;",
          "trace" => {
        "id" => "cf9f5500d60596ec76700cf3f8a53b56"
    },
     "@timestamp" => 2023-05-24T06:11:28.414Z,
       "@version" => "1",
            "log" => {
        "logger" => "plugins.actions",
         "level" => "ERROR",
          "file" => {
            "path" => "/var/log/kibana/kibana.log"
        }
    },
            "ecs" => {
        "version" => "8.0.0"
    },
          "event" => {
        "original" => "{\"ecs\":{\"version\":\"8.0.0\"},\"@timestamp\":\"2023-05-24T02:11:28.414-04:00\",\"message\":\"Server log: CP1 containers CPU metric threshold - 52e5c413c5eba2b161ade239aa124d2041d5bdca76deb5c5260b6023ac608464 is in a state of ALERT CPU usage is 17% in the last 1 min for 52e5c413c5eba2b161ade239aa124d2041d5bdca76deb5c5260b6023ac608464. Alert when > 10%.;\",\"log\":{\"level\":\"ERROR\",\"logger\":\"plugins.actions\"},\"process\":{\"pid\":366220},\"trace\":{\"id\":\"cf9f5500d60596ec76700cf3f8a53b56\"},\"transaction\":{\"id\":\"177d25f28df3bd5c\"}}"
    },
        "process" => {
        "pid" => 366220
    },
        "message" => "Server log: CP1 containers CPU metric threshold - 52e5c413c5eba2b161ade239aa124d2041d5bdca76deb5c5260b6023ac608464 is in a state of ALERT CPU usage is 17% in the last 1 min for 52e5c413c5eba2b161ade239aa124d2041d5bdca76deb5c5260b6023ac608464. Alert when > 10%.;"
}

```

However, I don't see the "container\_id" or "container\_message" fields neither in stdout nor in kibana dashboard. Surprisingly, the "Discover" tab is not even showing \_grokparsefailure, rather it is able to parse the message and just displaying the "logMessage" field replicating the complete message available in "message" field.

I would like to know if I am doing something wrong here.

Thanks,  
Nitish

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 22, 2023, 11:21am UTC](https://discuss.elastic.co/t/grok-on-logstash-not-working/334172/3 "2023-06-22T11:21:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.