Logstash grok to match the metricbeat type and add field

Elastic Stack Logstash
1

Hi Logstash Experts,

I have metricbeat to logstash to Kibana pipeline. In the logstash, I have the below grok filter to parse java process. But it is not adding the field for the matched pattern. secondly, it is additionally matching the filebeat logs. The expectation is to match only the metricbeat java process and add the field for the matching pattern.

filter {
if [type] == "metricsets" {
grok {
match => { "system.process.cmdline" => ["(?java -jar)\s(?([-]?Env[=]?[prdeval]+[23]?))\s(?([-]?Int[=]?[a-zA-Z0-9-]+))\s%{GREEDYDATA:rest}"] }
add_field => { "service_status" => "running" }
}
}
}

From Kibana, json output
{
"_index": "metric-2018.03.02",
"_type": "metricsets",
"_id": "AWHnPFRuOJtEtYrh-QW7",
"_score": null,
"_source": {
"@timestamp": "2018-03-02T15:02:25.158Z",
"system": {
"process": {
"memory": {
"rss": {
"pct": 0.0065,
"bytes": 439881728
},
"size": 3891261440,
"share": 14336000
},
"cmdline": "java -jar -Env=test2 -Int=3-0 -Dprotocol=https -Dssl_enabled=true -Xms128m -Xmx256m /var/log",

please help here to fix this issue.

2

Hi,

Metricbeat should deliver as json already. At least it does if you let metricbeat write to file and ship this via filebeat.
Then you can use json filter and the parsing is done automatically. Later you can add your own fields bases on the other fields.

You did not show the full json from kibana. Is there a tag like _grokparsefailure? If grok is failing, i dont know if the fiel is added. I miss the rest field in your json. So I think thatnis thr issue you are facing.

3

I am getting the filebeat data where i am supposed to get only metricbeat data.
here is my code, what i am missing or what are the ways i can debug and find the issue ?

metricbeat.yml:

metricbeat.modules:

  • module: system
    metricsets:
    • cpu
    • load
    • core
    • diskio
    • filesystem
    • fsstat
    • memory
    • network
    • process
      enabled: true
      period: 10s
      processes: ['.*']
      output.logstash:
      hosts: ["logstash-server.domain:5044"]
      logging.level: warning

logstash-metric.conf:

input {
  beats {
    port => 5044
  }
}

filter {
    if [@metadata][beat] == "metricbeat" {
        grok {
            match => { "[system][process][cmdline]" => "[(?<projectdate>java -jar\s)(?<projectenv>([-]?Denv[=]?[prdeval]+[23]?\s))(?<projecttype>([-]?Dtype[=]?[a-zA-Z0-9-]+\s))(?<projectapp>([-]?Dapp[=]?[project23]+\s))(?<prof>([a-zA-Z0-9=-]+\s))(?<springprofile>([.a-zA-Z0-9=-]+\s))(?<servicename>([a-zA-Z0-9=-]+\s))(?<appserver>([.a-zA-Z0-9=-]+\s))(%{GREEDYDATA:rest})]" }
        add_field => { "service_status" => "running" }
        }
    }
}
output {
  elasticsearch {
    hosts => ["elasticsearch-server.domain:9200"]
    manage_template => false
    index => "metric-microservice-%{+YYYY.MM.dd}"
  }
}

I need to match only similar java processes and other processes only and not Filebeat data:

{
  "_index": "metric-microservice-2018.03.05",
  "_type": "metricsets",
  "_id": "AWH2k5q0OJtEtYrhjgpL",
  "_score": null,
  "_source": {
    "system": {
      "process": {
        "memory": {
          "rss": {
            "pct": 0.011,
            "bytes": 742027264
          },
          "size": 4006264832,
          "share": 11497472
        },
        "cmdline": "java -jar -Denv=ver -Dtype=3-0 -Dapp=project3 -Dprof=ver-3-0 -Dspring.profiles.active=env-3-0 -Dsvcname=project-component-3-0 -Dhostname=my-build-server.project.adml -Dsecname=my-build-server.fullname -Dapachsvr_port=8090 -Dprotocol=https -Dssl_enabled=true -Xms128m -Xmx256m -DLOG_PATH=/var/opt/logs -Dlog_path=/var/opt/logs /var/opt/pivotal/svcs/project-component.jar /tmp",
        "pgid": 18546,
        "name": "java",
        "cpu": {
          "start_time": "2018-02-20T16:53:40.000Z",
          "total": {
            "pct": 0
          }
        },
        "pid": 19615,
        "state": "sleeping",
        "fd": {
          "limit": {
            "hard": 4096,
            "soft": 4096
          },
          "open": 46
        },
        "ppid": 1,
        "username": "root"
      }
    },
    "@timestamp": "2018-03-05T14:32:03.089Z",
    "beat": {
      "hostname": "my-build-server",
      "name": "my-build-server",
      "version": "5.0.1"
    },
    "@version": "1",
    "host": "my-build-server",
    "service_status": "running",
    "metricset": {
      "rtt": 84209,
      "module": "system",
      "name": "process"
    },
    "type": "metricsets",
    "tags": [
      "beats_input_raw_event"
    ]
  },
  "fields": {
    "@timestamp": [
      1520260323089
    ]
  },
  "sort": [
    1520260323089
  ]
}
4

can you please format your json with the code button grafik?

Please try to change system.process.cmdline to [system][process][cmdline]. Thats the correct syntax to access nested fields.

Haven't checked your regex / grok.

5

Yes, did changed to [_source][system][process][cmdline] but still facing the same issue.

Below is the sample filebeat json data that is matched from this logstash configuration.

{
  "_index": "metric-microservice-2018.03.05",
  "_type": "apache_access",
  "_id": "AWH2qL8GOJtEtYrhkXtV",
  "_score": null,
  "_source": {
    "request": "/",
    "offset": 2508254,
    "auth": "-",
    "ident": "-",
    "input_type": "log",
    "verb": "GET",
    "source": "/usr/apache/logs/access_log.2018-03-05-00_00_00",
    "message": "172.26.35.2 - - [05/Mar/2018:15:55:07 +0100] \"GET / HTTP/1.0\" 200 25626 \"-\" \"HTTP-Monitor/1.1\" 1816",
    "type": "apache_access",
    "tags": [
      "dev",
      "beats_input_codec_plain_applied"
    ],
    "@timestamp": "2018-03-05T14:55:08.268Z",
    "response": "200",
    "bytes": "25626",
    "clientip": "192.56.35.2",
    "@version": "1",
    "beat": {
      "hostname": "my-webserver",
      "name": "my-webserver",
      "version": "5.0.1"
    },
    "host": "my-webserver",
    "httpversion": "1.0",
    "timestamp": "05/Mar/2018:15:55:07 +0100"
  },
  "fields": {
    "@timestamp": [
      1520261708268
    ]
  },
  "sort": [
    1520261708268
  ]
}
6

@asp - how to format using code button? Is there any tool by which code button can be applied ?

7

just set an empty line before and after your code. Then mark the text and hit the code button in the menu:grafik

then you get sth like that

{
  "bla" : "blabla",
  "blub": {
    "test" : 1
    }
}

grafik
grafik1430×206 7.58 KB

otherwise spaces which are used for formating will be lost an saving.

8

thanks, formatted the code.

9

@asp - thanks. done formatting as code.
if [type] in ["metricsets", "metricbeat] solved the issue. Thanks a lot.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.