# Grok syslog-ng stats

**URL:** <https://discuss.elastic.co/t/grok-syslog-ng-stats/121685>\
**Category:** Logstash\
**Created:** [February 27, 2018, 3:19pm UTC](https://discuss.elastic.co/t/grok-syslog-ng-stats/121685 "2018-02-27T15:19:59Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![pdna](https://avatars.discourse-cdn.com/v4/letter/p/3bc359/32.png) [@pdna](https://discuss.elastic.co/u/pdna)\
**Post date:** [February 27, 2018, 3:20pm UTC](https://discuss.elastic.co/t/grok-syslog-ng-stats/121685/1 "2018-02-27T15:20:00Z")

</div>

```
Feb 20 00:30:00 host syslog-ng[12336]: Log statistics; queued='global(scratch_buffers_count)=0', processed='destination(ntpd_clients)=59', processed='destination(allusers)=0', processed='center(received)=372450', processed='destination(named_clients)=32', processed='source(local)=103306', processed='destination(periodic)=1230', processed='destination(periodic_clients)=3028', processed='destination(cron)=6829',......

```

I have syslog-ng stats in the above format and would like to parse each stat. For example, `queued='global(scratch_buffers_count)=0'` should be grokked as

```
statistic_type: queued
object_type: global
object_name: scratch_buffers_count
total_messages: 0

```

and so on for each stat.

I can only think of kv filter but that only do `queued: 'global(scratch_buffers_count)=0'` and there will be duplicate keys. How do I approach this?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 27, 2018, 3:57pm UTC](https://discuss.elastic.co/t/grok-syslog-ng-stats/121685/2 "2018-02-27T15:57:19Z")

</div>

> [@pdna](#):
>
> and so on for each stat

It is unclear what you mean by that. Do you want to end up with an array of objects each of which has those 4 properties?

I would be tempted to use grok or dissect to split at the semicolon, then mutate/split and process each array entry with a ruby filter to do this.

---

<div class="post-metadata">

**Author:** ![pdna](https://avatars.discourse-cdn.com/v4/letter/p/3bc359/32.png) [@pdna](https://discuss.elastic.co/u/pdna)\
**Post date:** [February 27, 2018, 5:54pm UTC](https://discuss.elastic.co/t/grok-syslog-ng-stats/121685/3 "2018-02-27T17:54:37Z")

</div>

Thanks a lot. I was able to get the stats into an array using a filter like this. How do I process each array entry with a ruby filter? Do I have to loop over the items and extract fields or is there a grok filter for that?

```
filter {
  grok {
    match => { "message" => ["%{SYSLOGTIMESTAMP:[syslog_timestamp]} %{SYSLOGHOST:[syslog_hostname]} %{DATA:[syslog_program]}(?:\[%{POSINT:[pid]}\])?: Log statistics; %{GREEDYDATA:[statistics]}",
                             "%{SYSLOGTIMESTAMP:[syslog_timestamp]} %{SYSLOGHOST:[syslog_hostname]} %{DATA:[syslog_program]}(?:\[%{POSINT:[pid]}\])?: %{GREEDYDATA:[syslog_message]}"] }
      }
    date {
        match => ["[syslog_timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
      }
  mutate {
    split => { "statistics" => "," }
    }

}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 27, 2018, 7:18pm UTC](https://discuss.elastic.co/t/grok-syslog-ng-stats/121685/4 "2018-02-27T19:18:19Z")

</div>

> [@pdna](#):
>
> How do I process each array entry with a ruby filter? Do I have to loop over the items and extract fields

Yes, something like this...

```auto
  mutate { split => { "stats" => "," } }
  ruby {
    code => '
      stats = Array.new
      event.get("stats").each { |x|
        x = x.strip
        x = x.split(/[=\'\)\(]+/)
        h = { "statistic_type" => x[0], "object_type" => x[1], "object_name" => x[2], "total_messages" => x[3] }
        stats.push(h)
      }
      event.set("statistics", stats)
    '
  }
# mutate { remove_field => ["stats"] }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2018, 7:18pm UTC](https://discuss.elastic.co/t/grok-syslog-ng-stats/121685/5 "2018-03-27T19:18:24Z")

</div>

Closed on March 27, 2018, 7:18pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.