Take input from more than 5 servers

[shrikantgulia (Shrikant)]

I am new to ELK
Pls Help
As i have to observe 5 different Servers(5 metricbeat & 5 Filebeat) on ec2 instance . Than do i need to run different logstash config. file or i can take input in a single logstash file and run it?
and also do i need to create different cluster for each server?

Thanks & Regards
Shrikant

[leandrojmp (Leandro Pereira)]

You can have only one logstash instance for receive the data from the 5 different servers.

On your logstash you will have an input like this.

input {
  beats {
    port => 5044
  }
}

And in each one of your servers you will configure your beats to output to the same logstash server.

output.logstash:
  hosts: ["your-logstash-server:5044"]

[shrikantgulia (Shrikant)]

Greetings thankyou for your reply
Than how will we be able to create different index for all the 5 Servers.
As I want to create different dashboard for each server.
What should be the output of logstash for this?

[leandrojmp (Leandro Pereira)]

For each one of your servers you can define a type in the beats configuration, than in your logstash pipeline, in the filter and output blocks, you will need to use conditionals to direct each input to its own output.

For example, in a filebeat configuration you will have something like this:

filebeat:
 prospectors:
  - input_type: log
    paths:
     - C:\inetpub\logs\LogFiles\W3SVC1\*.log
    fields:
      document_type: server01-type

Then in your logstash pipeline you will have something like this:

filter {
    if [type] == "server01-type" {
        your filters
    }
}
output {
    if [type] == "server01-type" {
        elasticsearch {
        hosts           => ["localhost:9200"]
        index           => "server01-%{+YYYY.MM.dd}"
        document_type       => "server01-type"
        template_name       => "server01-template"
       }
    }
}

You can read more about conditionals here

[magnusbaeck (Magnus Bäck)]

Instead of using conditionals, you should get a host field with the name of the host that collected the event, in which case you can reference that field in your single elasticsearch output:

elasticsearch {
  hosts           => ["localhost:9200"]
  index           => "%{host}-%{+YYYY.MM.dd}"
  document_type   => "%{host}-type"
  template_name   => "%{host}-template"
}