How to filter Filebeat input in Logstash from different log?

So, I have a VPS and have 2 json log from 2 different services. I used Filebeat to forward my log to ELK Stack Server. This is my filebeat.yml:

filebeat.inputs:
- type: log
  paths:
    - /home/cowrie/cowrie/log/cowrie.json*
  encoding: plain
  fields:
    document_type: cowrie

- type: log
  paths:
    - /opt/dionaea/var/lib/dionaea/dionaea.json*
  encoding: plain
  fields:
    document_type: dionaea

output.logstash:
  hosts: ["10.33.109.64:5044"]

logging.to_files: true
logging.files:
  path: /var/log/filebeat/
  name: mybeat-vpn
  rotateeverybytes: 10485760 # = 10MB
  keepfiles: 7

I want to filter each log( cowrie and dionaea) with different rule on Logstash. This my logstash config:

input {
   beats {
       port => 5044
   }
}
filter{
 if [type] == "cowrie" {
.......some filter..........
}
 if [type] == "dionaea" {
.......some filter..........
}
}

output {
   elasticsearch {   
    hosts => ["10.33.109.24:9200"]
	}
}

I just wondering, what is the condition for this -->if [type] == "cowrie" because i can not declare type field in input. There is only beat input in this configuration. TL;DR I want to separate this 2 logs with different filter rule.

I just wondering, what is the condition for this -->if [type] == "cowrie" because i can not declare type field in input.

Yes you can. You can define any fields you like in the Filebeat configuration.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.