Time Based Indices

Elastic Stack Elasticsearch
1

Hi, I was trying to create an index based on time, but with a different name. For example if I want to create an index called "nginx- *", at the time of consulting will be nginx-2018.11.05, but I need to geolocate, and reading articles, can only be "logstash" the name of the index to create the geoip.

So in what way could I create an index based on time with geolocation ?, and how do I call it in logstash ?.

This is my current .conf.

input {

    file {
            path => "/var/log/nginx"
            start_position => "beginning"
            stat_interval => 1
            discover_interval => 15
            sincedb_path => "/dev/null"
            sincedb_write_interval => 15
         }

  }


filter {

grok {
   patterns_dir => "/etc/logstash/patterns/"
   match => { "message" => "%{NGINXDATE:logtimestamp}" }
}

date { match => [ "logtimestamp", "MMM dd HH:mm:ss", "MMM  d HH:mm:ss" ] }

grok {

    patterns_dir => "/etc/logstash/patterns"
    match => { "message" => "%{NGINX_ACCESS}" }
    remove_tag => ["_grokparsefailure"]
    add_tag => ["nginx_access"]
}

grok {
    match => { "agent" => "[(]%{DATA:OS}; %{DATA:version}; %{DATA:device}[)]" }
    match => { "agent" => "[(]%{DATA:OS}[)]" }
}

geoip {
           source => "clientip"
}



mutate {
remove_field => [ "type", "path", "host"]
}
}

output {

elasticsearch {
            hosts => ["192.168.1.105:9200"]
            index => "logstash-nginx"
               }
    }

I'm new to this, sorry.

Regards,

2

This can be set up for any index, but Logstash comes with a default index template for the logstash-* index pattern. You can retrieve this index template, update it to match your index pattern and then store it back in Elasticsearch under an appropriate name.

3

your index is pre with logstash not nginx
can with *nginx * in kibana create parrten.

5

Thanks for support.

But creating * nginx *, will create a time based Index?, for e.g. nginx-2018.05.14?.

Regards,

6

Hi, I think I can give an alternative solution to this, thinking that creating a new template is a bit difficult, and solving my doubt to create time-based indexes, configure this.

Logstash

output {

elasticsearch {
    hosts => ["192.168.1.105:9200"]
    index => "logstash-nginx-%{+ YYYY.MM.dd}"
}
}

Kibana

logstash-nginx-*

Now, when I run "get _cat / shards", I get the following.

logstash-2018.05.14
logstash-nginx-2018.05.14

Can you tell me why you create two indexes, if I only define logstash-nginx?

regards,

7

delete the indices and try again? see which create.
logstash maybe other create.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.