Dissect Filter - Catching a sub field

Elastic Stack Logstash
1

Hi,
Following is my logstash configuration,

filter {
  dissect {
    mapping => { '[exec.stdout]' => "%{host} %{up} %{users} %{load} %{IP} %{Memoryfree} %{MemoryTotal} %{SwapTotal} %{SwapFree} %{Psql_active} %{psql_established} %{Java_Thread} %{chargepercent} %{chargestate}" }
  }

mutate {
 convert =>     {"up" => "float"}
 convert =>     {"users" => "float"}
 convert =>     {"load" => "float"}
 convert =>     {"Memoryfree" => "float"}
 convert =>     {"MemoryTotal" => "float"}
 convert =>     {"SwapTotal" => "float"}
 convert =>     {"SwapFree" => "float"}
 convert =>     {"Psql_active" => "float"}
 convert =>     {"psql_established" => "float"}
 convert =>     {"Java_Thread" => "float"}
 convert =>     {"chargepercent" => "float"}
 convert =>     {"chargestate" => "string"}
}

The following is my elastic search mapping,

{
  "_index": "logstash-2018.11.05",
  "_type": "doc",
  "_id": "pdy74mYBwKRwUDhIN_kD",
  "_version": 1,
  "_score": null,
  "_source": {
    "host": "masterserver",
    "tags": [
      "beats_input_raw_event",
      "_dissectfailure"
    ],
    "@version": "1",
    "@timestamp": "2018-11-05T07:16:18.000Z",
    "beat": {
      "name": "masterserver",
      "hostname": "masterserver",
      "version": "3.3.0"
    },
    "type": "execbeat",
    "exec": {
      "exitCode": 0,
      "stdout": "masterserver 1:05, 2 10.25.20.39 4798260 7870328 15626236 15626236 0 50 79 charging\n",
      "command": "sh"
    }
  },
  "fields": {
    "@timestamp": [
      "2018-11-05T07:16:18.000Z"
    ]
  },
  "sort": [
    1541402178000
  ]
}

I am not able to pull the exec.stdout field from the execbeat output in logstash.

The following dissect parse error is getting in the logtsash O/P log,

Dissector mapping, field not found in event {"field"=>"[exec.stdout]", "event"=>{"host"=>"masterserver", "@version"=>"1", "tags"=>["beats_input_raw_event"], "type"=>"execbeat", "exec"=>{"command"=>"sh", "stdout"=>"masterserver 2:11, 2 10.25.20.39 4799780 7870328 15626236 15626236 0 50 79 charging\n", "exitCode"=>0}, "@timestamp"=>2018-11-05T08:22:31.000Z, "beat"=>{"name"=>"masterserver", "version"=>"3.3.0", "hostname"=>"masterserver"}}}

Request to advise , how to map a sub field in the logtsash.

Thanks in advance.

2

You need to use the Field Reference syntax:

[exec][stdout]
3

Thanks!
But its still getting error while dissecting!

`

> Dissector mapping, pattern not found {"field"=>"[exec][stdout]", "pattern"=>"%{host} %{up} %{users} %{load} %{IP} %{Memoryfree} %{MemoryTotal} %{SwapTotal} %{SwapFree} %{Psql_active} %{psql_established} %{Java_Thread} %{chargepercent} %{chargestate}", "event"=>{"beat"=>{"version"=>"3.3.0", "name"=>"localhost.localdomain", "hostname"=>"localhost.localdomain"}, "type"=>"execbeat", "exec"=>{"stdout"=>"localhost.localdomain 32 3 192.168.43.219 6013812 7869320 104857596 104857596 79 discharging 8 1 1\n", "exitCode"=>0, "command"=>"sh"}, "@timestamp"=>2018-11-13T11:38:36.000Z, "tags"=>["beats_input_raw_event", "_dissectfailure"], "@version"=>"1", "host"=>"localhost.localdomain"}}

Request to advice! Really stuck!

4

The [exec][stdout] field in the event shown in the last error message seems to have 13 space-separated components, while your pattern specifies 14 fields to be extracted. I would not be surprised if that is what is causing the problem.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.