### i delete one index of yesterday,but 8 hours data of today is gone! help me pls

[nwpu.yangqing]

【replay angin】

1、first，check my indices before i do any detele operations

% curl -XGET '172.18.102.5:9200/_cat/indices/migu_csms_debug*'
green open migu_csms_debug-2017-04-20 SlGTE1A0TWSDerSuwDs0rA 5 1 273033 0 236.3mb 118.2mb
green open migu_csms_debug-2017-04-21 gxl3wmLORdiISZXq16PNHA 5 1 26845 0 28.5mb 14.2mb

2、then i delete the index migu_csms_debug-2017-04-20
[kibana@MGHJ-YW-mg-web05 ~]$ curl -XDELETE 'http://172.18.102.5:9200/migu_csms_debug-2017-04-20?pretty'
{
"acknowledged" : true
}

3、look into kibana again

As you can see,some (exactly 8 hours) data of today is gone.
i don't know why,and doubt sth is wrong elasticsearch node

pls help me!!
thk you very much from China!

[Christian_Dahlqvist]

Time-based indices generated by Logstash are based on UTC time. Each index will therefore not necessarily contain a full days of data in the time zone you are in. If you create an index pattern just matching a single index, you would be able to see this in Kibana.

[nwpu.yangqing]

I wrote a shell to delete historical indices automatically,as follws:

[kibana@MGHJ-YW-mg-web05 shell]$ more del_index.sh
#!/bin/bash
source ~/.bash_profile
del_date=`date -d "4 days ago" +%Y-%m-%d`
curl -XDELETE ''http://172.18.102.5:9200/migu_csms_debug-${del_date}?pretty''

so ,how could i modify it to delete a precise day's data?not +8(-8) hours?

[Christian_Dahlqvist]

Retention is generally managed by deleting entire indices, as this is much more efficient than deleting records individually from an index. You could keep an extra days worth of indices, which would ensure the full time period is covered or possibly modify the index naming in Logstash and override the default to get a date based on your time zone.

[nwpu.yangqing]

thk you for your replies.

if i add config belows into my logstash_config file,would it help?
date {
timezone => "Asia/Shanghai"
}

[Christian_Dahlqvist]

Kibana and Elasticsearch require/assume that the timestamp is is UTC, so changing this to something else will cause problems. A better way may be to parse the raw event date and use this to build the index name for the elastic search output.

[nwpu.yangqing]

Thank you for all your assistance.

As you'd advised,i decide to keep it a bit longer time.