Specifying index name from filebeat to logstash

Elastic Stack Beats
1

I am trying to setup my ELK pipeline. I have filebeat monitoring on one VM and my logstash and elasticsearch instances running on another. When I make an update to the file which specified in filebeat.yml, I see that change reflected in logstash (logstash,conf also prints to stdout). In my filebeat.yml under the logstash section I specify the index name as "demo". When I try to search for that index in elasticsearch, I get an error response.

So, was the change not added to elasticsearch, or am I specifying the index name incorrectly? I think it's the latter because I was following a tutorial to set this up, and ES is checked to contain the right information by using a different index name, but I don't know how that index name was specified.

2

Providing your beats config would be helpful.

3

These are all the uncommented lines in my beat config

filebeat:

  prospectors:
     - 
        paths:
            - /path/to/mylog.log
        input_type: log

logstash: 

    hosts:
        - my.logstash_instance.com:5044

    index: demo

shipper:

group

logging:
4

Ok that looks normal.
What about LS?

5

what's this group in your config file is about? It doesn't seem to belong anywhere.

6

Here

 input {
  beats {
    port => 5044
  }
}

filter {
    grok {
        match => {"message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
}

output {
  elasticsearch {

  }
  stdout { }
}

Where is the index supposed to be specified?

7

I don't know, but by default it's uncommented

8

  1. that's weird. should not be uncommented.

  2. check out logstash elasticsearch output docs. The index option is normally used to configure the index. You can use any fields in your event to construct your index.

9

that works. thanks!