How to filter Filebeat input in Logstash from different log?

Elastic Stack Logstash
1

So, I have a VPS and have 2 json log from 2 different services. I used Filebeat to forward my log to ELK Stack Server. This is my filebeat.yml:

filebeat.inputs:
- type: log
  paths:
    - /home/cowrie/cowrie/log/cowrie.json*
  encoding: plain
  fields:
    document_type: cowrie

- type: log
  paths:
    - /opt/dionaea/var/lib/dionaea/dionaea.json*
  encoding: plain
  fields:
    document_type: dionaea

output.logstash:
  hosts: ["10.33.109.64:5044"]

logging.to_files: true
logging.files:
  path: /var/log/filebeat/
  name: mybeat-vpn
  rotateeverybytes: 10485760 # = 10MB
  keepfiles: 7

I want to filter each log( cowrie and dionaea) with different rule on Logstash. This my logstash config:

input {
   beats {
       port => 5044
   }
}
filter{
 if [type] == "cowrie" {
.......some filter..........
}
 if [type] == "dionaea" {
.......some filter..........
}
}

output {
   elasticsearch {   
    hosts => ["10.33.109.24:9200"]
	}
}

I just wondering, what is the condition for this -->if [type] == "cowrie" because i can not declare type field in input. There is only beat input in this configuration. TL;DR I want to separate this 2 logs with different filter rule.

2

I just wondering, what is the condition for this -->if [type] == "cowrie" because i can not declare type field in input.

Yes you can. You can define any fields you like in the Filebeat configuration.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.