# Indexing sysstat data from sar and sadf

**URL:** <https://discuss.elastic.co/t/indexing-sysstat-data-from-sar-and-sadf/42044>\
**Category:** Logstash\
**Created:** [February 17, 2016, 4:04pm UTC](https://discuss.elastic.co/t/indexing-sysstat-data-from-sar-and-sadf/42044 "2016-02-17T16:04:14Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![robertws8](https://avatars.discourse-cdn.com/v4/letter/r/8e8cbc/32.png) [@robertws8](https://discuss.elastic.co/u/robertws8)\
**Post date:** [February 17, 2016, 4:04pm UTC](https://discuss.elastic.co/t/indexing-sysstat-data-from-sar-and-sadf/42044/1 "2016-02-17T16:04:14Z")

</div>

I am already collecting syslog data via logstash into elasticsearch. My customer now wants sysstat date as well. So, I created a new index and types (using for example field names and types from sar -d -p 1 1)

PUT /monitor/\_mapping/disk  
{ "disk": {  
"properties": {  
"event\_time": {"type": "date"},  
"logsource": {"type": "string"},  
"DEV": {"type": "string"},  
"avgqu-sz": {"type": "float"},  
"avgrq-sz": {"type": "float"},  
"await": {"type": "float"},  
"%util": {"type": "float"},  
"rd\_sec/s": {"type": "float"},  
"svctm": {"type": "float"},  
"tps": {"type": "float"},  
"wr\_sec/s": {"type": "float"}  
}}}

Populating data with curl directly to elasticsearch works...  
curl -XPOST [http://127.0.0.1:9200/monitor/disk](http://127.0.0.1:9200/monitor/disk) -d '{"logsource":"intense","event\_time":"1455723240000","DEV":"sda","tps":"25.82","rd\_sec/s":"119.09","wr\_sec/s":"4836.90","avgrq-sz":"191.98","avgqu-sz":"1.08","await":"41.82","svctm":"4.14","%util":"10.70"}'

I have a script that runs periodically that parses the output of sadf into json.

I am also forwarding the data via syslog so the message part of the event is the above json prepended with the sar data type and the program field is the index name (monitor)...

monitor disk '{"logsource":"intense","event\_time":"1455723240000","DEV":"sda","tps":"25.82","rd\_sec/s":"119.09","wr\_sec/s":"4836.90","avgrq-sz":"191.98","avgqu-sz":"1.08","await":"41.82","svctm":"4.14","%util":"10.70"}'

I also have a very plain vanilla logstash config file for syslog...

input {  
syslog {  
type =\> syslog  
port =\> 5544  
facility\_labels =\> ["kernel", "user-level", "mail", "system", "sec/auth", "syslogd", "line printer", "network news", "UUCP", "clock", "sec/auth", "FTP", "NTP", "log audit", "log alert", "clock", "local0", "local1", "local2", "local3", "local4", "local5", "local6", "local7"]  
}  
}

output {  
if [type] == "syslog" and "\_grokparsefailure\_sysloginput" in [tags] {  
file { path =\> "/var/log/logstash/syslog\_failures-%{+YYYY-MM-dd}" }}  
else {  
elasticsearch { index =\> "logstash-%{+YYYY.MM.dd.HH}"}}  
}

What I want to do (and don't know how) is to add another conditional to the logstash config that will check for program = monitor and if true do the right thing with the json from the syslog message field. Any help would be greatly appreciated.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 20, 2016, 7:37pm UTC](https://discuss.elastic.co/t/indexing-sysstat-data-from-sar-and-sadf/42044/2 "2016-02-20T19:37:52Z")

</div>

```auto
filter {
  if [program] == "monitor" {
    json {
      source => "name of syslog field"
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![robertws8](https://avatars.discourse-cdn.com/v4/letter/r/8e8cbc/32.png) [@robertws8](https://discuss.elastic.co/u/robertws8)\
**Post date:** [February 21, 2016, 4:04pm UTC](https://discuss.elastic.co/t/indexing-sysstat-data-from-sar-and-sadf/42044/3 "2016-02-21T16:04:38Z")

</div>

Thanks a lot. I think I misunderstood the whole filter concept. After I saw your repsonse I redid my solution using the kv filter which also works fine. This makes it possible to encapsulate just about any kind of data within syslog.

Rob

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:10am UTC](https://discuss.elastic.co/t/indexing-sysstat-data-from-sar-and-sadf/42044/4 "2017-07-06T05:10:25Z")

</div>