Grokparsefailure when filtering a multiline json value

Due to memory limitation I cannot deploy Beats on my device. Therefore I pull metric data with a remote Http_poller into Logstash.
The response data of the poll-request looks like this:

{
	"memory": {
		"raw": "              total        used        free      shared  buff/cache   available\r\nMem:        6876856     4778664     1338016        4084      760176     2009568\r\nSwap:       2097148       24332     2072816"
	}
}

As you can see it is the plain output of the free command (Linux). If I do not use a filter and just print it to stdout with rubydebug it looks like this:

{
    "http_poller_metadata" => {
        ...truncated...
    },
    "@timestamp" => 2019-10-23T06:52:02.034Z,
    "@version" => "1",
    "memory" => {
        "raw" => "              total        used        free      shared  buff/cache   available\r\nMem:        6876856     4778664     1338016        4084      760176     2009568\r\nSwap:       2097148       24332     2072816"
    }
}

Now I want to put those numbers into fields using grok.
My Logstash config approach is this:

input {
  http_poller {
    urls => {
      test1 => "http://10.10.1.XXX/raw_metrics.json"
    }
    request_timeout => 60
    schedule => { "every" => "1m" }
    codec => "json"
    metadata_target => "http_poller_metadata"
  }
}
filter {
        grok {
                match => { '[memory][raw]' => '              total        used        free      shared  buff/cache   available\\r\\nMem:        %{NUMBER:total_memory}     %{NUMBER:used_memory}     %{NUMBER:free_memory}        %{NUMBER:shared_memory}      %{NUMBER:buffered_memory}     %{NUMBER:available_memory}\\r\\nSwap:       %{NUMBER:total_swap_memory}       %{NUMBER:used_swap_memory}     %{NUMBER:free_swap_memory}' }
        }
}
output {
        stdout { codec => rubydebug }
}

But this produces a grokparsefailure:

"tags" => [
        [0] "_grokparsefailure"
]

Do you have any hint for me? The grok debugger in Kabana showed no issues when using only the raw data against the match pattern.

Thank you!
Andreas

The string probably does not literally contain \r and \n, it probably contains a carriage return and newline. Try matching against two patterns, one for each line.

My issue was indeed a newline issue. I now split it now first with mutate->split and it works for me now :slight_smile:

It seems that the match just affected one line all the time.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.