Is it possible to use the elasticsearch template to create a filter template in logstash


#1

I'm using auditbeat to send syslog messages to logstash and then onto elasticsearch. I have deployed the template from auditbeat to elasticsearch and the messages being sent through are being correctly parsed and show nicely in Kibana.

I need to filter out some messages in logstash before they get to elasticsearch and the impression is that I will need to use grok in the filter to parse it to select the right information. (Running on separate systems and more sense to have logstash filter out than require the privileges etc on the server being monitored to keep playing with auditbeat rules)

For elasticsearch to manage these correctly then it appears that the templates must have the correct information in to parse and so I shouldn't need to create them from scratch in logstash.

Any idea how I can get them to effectively copy and paste in to the logstash filter?

Many thanks!


#2

I'm guessing the auditbeat.json file in the index-pattern directory, but can a json definition be used in a logstash filter or does it have to be translated?

This is what is in it (Fields cut down as too large for post):

{
  "objects": [
    {
      "attributes": {
        "fieldFormatMap": "{\"@timestamp\":{\"id\":\"date\"}}",
        "fields": "[{\"aggregatable\":true,\"analyzed\":false,\"count\":0,\"doc_values\":true,\"indexed\":true,\"name\":\"beat.name\",\"scripted\":false,\"searchable\":true,\"type\":\"string\"}, etc etc etc}]",
        "timeFieldName": "@timestamp",
        "title": "auditbeat-*"
      },
      "id": "auditbeat-*",
      "type": "index-pattern",
      "version": 1
    }
  ],
  "version": "6.2.1"
}

#3

OK, so found out that auditbeat is sending the entries as a json format. So I'm guessing I need to filter with something like:

filter {
  json {
    source => "message"
    if [user.name_map.auid] == "nagios" {
      drop { }
    }
  }
}

Trying to lose the ones created by nagios. Cheers


#4

Sheesh it is easy once you know

filter {
  if [user][name_map][auid] == "nagios" {
    drop { }
  }
}

(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.