Filebeat to send logs to Logstash and ES

Bit late to respond, but... anyway...

myindexname is just a name you come up with. If you collect system logs, you can call your index syslog. If you collect all kind of logs, you can call it logs or logstash.

Tiny tutorial

You need to set up Filebeat and Logstash. These are by default configured in /etc/filebeat
/filebeat.yml and /etc/logstash/logstash.yml.

For starters, delete everything from these files, because 95% of these are comments anyway. If you need the default contents later (e.g. the explanations), it's all out there on GitHub.

Now in Filebeat's yml (still /etc/filebeat/filebeat.yml :slight_smile: ) you need to specify an input and an output, e.g. Logstash if you fancy that:

filebeat.inputs:
- type: log
  - /var/log/messages
  fields:
    index: syslog

output.logstash
  hosts: ["mylogstash.example.com:5044"]

That's all for a very very basic FB setup.

In Logstash's yml (at /etc/logstash/logstash.yml) specify the data and log directories:

path.data: /var/lib/logstash
path.log: /var/log/logstash

Create a config file for your inputs/outputs, e.g.

input {
  beats {
    port => "5044"
  }
}
output {
  elasticsearch {
    hosts => [
      "elasticsearch01.example.com:9200",
      "elasticsearch02.example.com:9200" ]
    index => "%{[index]}"

  }
}

Again, this is a quite basic approach, but should work as soon as you restart Filebeat and Logstash. Your server's /var/log/messages will end up in a syslog index.

Be aware that a single syslog index will not be sufficient if you start collecting logs from many servers. There are techniques to handle this (e.g daily/weekly/monthly indexes, aliasing/rollover, etc.), you just need to read the Elastic documentation to learn about them.