Questions about the timezone

[qin (zhenjie)]

I want to let logstash output to elasticsearch, My configuration is

input {
    tcp {
        host => "0.0.0.0" 
        port => 4560
        mode => "server"
        codec => json_lines  
      }
}

output {
    stdout{codec =>rubydebug}
    elasticsearch {
        hosts => ["192.168.31.41:9200","192.168.31.42:9200"]  
        flush_size => 10000
        idle_flush_time => 2
        index=>"logstash-task-%{taskId}-%{+YYYYMMddHH}"
     }
}

I want let index name is logstash-task-001-2017040120, but index name is logstash-task-001-2017040112, This may be the time zone problem, how to set ?

[qin (zhenjie)]

please help me

[Christian_Dahlqvist]

Each index and shard comes with overhead in terms of system resources. Having a large number of very small shards is very inefficient. You seem to want to create an hourly index per task id, which is likely to result in a very large number of small shards unless you have massive volumes of data coming in. What is the expected shard size with this configuration? Why such a small time period per index?

[qin (zhenjie)]

Thank you for your advice, this is done to achieve production requirements, Even if I create an daily index per task id， the problem of timezone is also there， Because differ for eight hours. Can you understand me? I'm not good at English.

[Christian_Dahlqvist]

I am not aware of any way to force the Elasticsearch output plugin to use a non-UTC timezone for the index name. It looks like this issue is tracked in this GitHub issue. You may also be able to get around it by creating the correct date string at the filter stage and use this to form the index name, but I do not have an example of how to do this.