Issue sending logs with FileBeat Getting parsed at Logstash

Elastic Stack Beats
1

I am able to get some of my logs sent from FileBeat to Logstash, but I seem to be having an issue with a 2 of them.

Do you guys have any words of wisdom?

I see logs in the folders that FileBeat is supposed to read from, but I am getting no dice on getting them to be sent back.

Part of the Filebeat Yml

# Mailoney
     -
        paths:
        - /data/mailoney/log/commands.log
        input_type: log
        document_type: Mailoney

        fields:
        fields_under_root: true
        json.keys_under_root: false
        json.add_error_key: true


# Conpot
     -
        paths:
        - /data/conpot/log/*.json"
        input_type: log
        document_type: Conpot

        fields:
        fields_under_root: true
        json.keys_under_root: false
        json.add_error_key: true


# Heralding
     -
        paths:
        - /data/heralding/log/auth.csv"
        document_type: Heralding

        fields:
        fields_under_root: true
        json.keys_under_root: false
        json.add_error_key: true

Logstash conf

# Heralding
  if [type] == "Heralding" {
    csv {
      columns => ["timestamp","auth_id","session_id","src_ip","src_port","dest_ip","dest_port","proto","username","password"] separator => ","
    }
    date {
      match => [ "timestamp", "yyyy-MM-dd HH:mm:ss.SSSSSS" ]
      remove_field => ["timestamp"]
    }
  }



# Conpot
  if [type] == "Conpot" {
    date {
      match => [ "timestamp", "ISO8601" ]
    }
    mutate { 
      rename => { 
        "dst_port" => "dest_port" 
        "dst_ip" => "dest_ip" 
      } 
    } 
  }


# Mailoney
  if [type] == "Mailoney" {
    grok {
      match => [ "message", "\A%{NAGIOSTIME}\[%{IPV4:src_ip}:%{INT:src_port:integer}] %{GREEDYDATA:smtp_input}" ]
    }
    mutate {
      add_field => {
        "dest_port" => "25"
      }
    }
    date {
      match => [ "nagios_epoch", "UNIX" ]
      remove_field => ["nagios_epoch"]
    }
  }
2

When you say you're having trouble with two of them, do you mean one of these three type configurations is working? Which ones are / aren't getting sent to Logstash?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.