# TTL set and enabled but logs not expiring ...please help **URL:** \ **Category:** Elasticsearch\ **Created:** [May 15, 2013, 1:17pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974 "2013-05-15T13:17:06Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 15, 2013, 1:17pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/1 "2013-05-15T13:17:06Z")
I have TTL enabled, but the logs are not removed when I view them from Kibana. They still show up, even after their expiration time is over. Here is the mapping that I get from the API, which clearly shows that the TTL is enabled with a default value of 1 day: {"logstash-2013.05.14":{"_default_":{"\_ttl":{"enabled":true,"default":86400000},"properties":{}},"filetype":{"\_ttl":{"enabled":true,"default":86400000},"properties":{"@fields":{"properties":{"\_second":{"type":"string"},"fiber1":{"type":"string"},"fiber2":{"type":"string"},"hour":{"type":"string"},"message":{"type":"string"},"minute":{"type":"string"},"monthday":{"type":"string"},"monthnum":{"type":"string"},"second":{"type":"string"},"slave":{"type":"string"},"ts":{"type":"string"},"type1":{"type":"string"},"type2":{"type":"string"},"year":{"type":"string"}}},"@message":{"type":"string"},"@source":{"type":"string"},"@source\_host":{"type":"string"},"@source\_path":{"type":"string"},"@tags":{"type":"string"},"@timestamp":{"type":"date","format":"dateOptionalTime"},"@type":{"type":"string"}}}},"logstash-2013.05.09":{"_default_":{"\_ttl":{"enabled":true,"default":86400000},"properties":{}},"filetype":{"\_ttl":{"enabled":true,"default":86400000},"properties":{"@fields":{"properties":{"\_second":{"type":"string"},"fiber1":{"type":"string"},"fiber2":{"type":"string"},"hour":{"type":"string"},"message":{"type":"string"},"minute":{"type":"string"},"monthday":{"type":"string"},"monthnum":{"type":"string"},"second":{"type":"string"},"slave":{"type":"string"},"ts":{"type":"string"},"type1":{"type":"string"},"type2":{"type":"string"},"year":{"type":"string"}}},"@message":{"type":"string"},"@source":{"type":"string"},"@source\_host":{"type":"string"},"@source\_path":{"type":"string"},"@tags":{"type":"string"},"@timestamp":{"type":"date","format":"dateOptionalTime"},"@type":{"type":"string"}}}}} Where am I going wrong? I am badly stuck here ☹ Please give valuable suggestions and help -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 15, 2013, 1:24pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/2 "2013-05-15T13:24:01Z")
Aakash - please stop sending the same emails repeatedly Send the question, and wait for a response. If you don't get one after a few days, perhaps you need to clarify the question clint On 15 May 2013 15:17, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > I have TTL enabled, but the logs are not removed when I view them from > Kibana. They still show up, even after their expiration time is over. > Here is the mapping that I get from the API, which clearly shows that the > TTL is enabled with a default value of 1 day: > > {"logstash-2013.05.14":{"_\*\*default_":{"_ttl":{"enabled":\*\*true,"default":86400000},"\*\*properties":{}},"filetype":{"_ **ttl":{"enabled":true,"default"** :86400000},"properties":{"@\*\*fields":{"properties":{"_\*\*second":{"type":"string"},"\*\*fiber1":{"type":"string"},"\*\*fiber2":{"type":"string"},"\*\*hour":{"type":"string"},"\*\*message":{"type":"string"},"\*\*minute":{"type":"string"},"\*\*monthday":{"type":"string"},"\*\*monthnum":{"type":"string"},"\*\*second":{"type":"string"}," **slave":{"type":"string"},"ts":** {"type":"string"},"type1":{"\*\*type":"string"},"type2":{" **type":"string"},"year":{"type"** :"string"}}},"@message":{"\*\*type":"string"},"@source":{" **type":"string"},"@source\_host"** :{"type":"string"},"@source_\*\*path":{"type":"string"},"@\*\*tags":{"type":"string"},"@\*\*timestamp":{"type":"date","\*\*format":"dateOptionalTime"},"@\*\*type":{"type":"string"}}}},"\*\*logstash-2013.05.09":{"_\*\*default_":{"_ttl":{"enabled":\*\*true,"default":86400000},"\*\*properties":{}},"filetype":{"_ **ttl":{"enabled":true,"default"** :86400000},"properties":{"@\*\*fields":{"properties":{"_\*\*second":{"type":"string"},"\*\*fiber1":{"type":"string"},"\*\*fiber2":{"type":"string"},"\*\*hour":{"type":"string"},"\*\*message":{"type":"string"},"\*\*minute":{"type":"string"},"\*\*monthday":{"type":"string"},"\*\*monthnum":{"type":"string"},"\*\*second":{"type":"string"}," **slave":{"type":"string"},"ts":** {"type":"string"},"type1":{"\*\*type":"string"},"type2":{" **type":"string"},"year":{"type"** :"string"}}},"@message":{"\*\*type":"string"},"@source":{" **type":"string"},"@source\_host"** :{"type":"string"},"@source_\*\*path":{"type":"string"},"@\*\*tags":{"type":"string"},"@\*\*timestamp":{"type":"date","\*\*format":"dateOptionalTime"},"@\*\*type":{"type":"string"}}}}} > > Where am I going wrong? I am badly stuck here ☹ > > Please give valuable suggestions and help > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 15, 2013, 1:48pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/3 "2013-05-15T13:48:59Z")
It's kind of urgent and there is no reply... ☹ On Wednesday, May 15, 2013, Clinton Gormley wrote: > Aakash - please stop sending the same emails repeatedly > > Send the question, and wait for a response. If you don't get one after a > few days, perhaps you need to clarify the question > > clint > > On 15 May 2013 15:17, Aakash Anuj \<[aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com)\ > > > wrote: > > > I have TTL enabled, but the logs are not removed when I view them from > > Kibana. They still show up, even after their expiration time is over. > > Here is the mapping that I get from the API, which clearly shows that the > > TTL is enabled with a default value of 1 day: > > > > {"logstash-2013.05.14":{"_\*\*default_":{"_ttl":{"enabled":\*\*true,"default":86400000},"\*\*properties":{}},"filetype":{"_ **ttl":{"enabled":true,"default"** :86400000},"properties":{"@\*\*fields":{"properties":{"_\*\*second":{"type":"string"},"\*\*fiber1":{"type":"string"},"\*\*fiber2":{"type":"string"},"\*\*hour":{"type":"string"},"\*\*message":{"type":"string"},"\*\*minute":{"type":"string"},"\*\*monthday":{"type":"string"},"\*\*monthnum":{"type":"string"},"\*\*second":{"type":"string"}," **slave":{"type":"string"},"ts":** {"type":"string"},"type1":{"\*\*type":"string"},"type2":{" **type":"string"},"year":{"type"** :"string"}}},"@message":{"\*\*type":"string"},"@source":{" **type":"string"},"@source\_host"** :{"type":"string"},"@source_\*\*path":{"type":"string"},"@\*\*tags":{"type":"string"},"@\*\*timestamp":{"type":"date","\*\*format":"dateOptionalTime"},"@\*\*type":{"type":"string"}}}},"\*\*logstash-2013.05.09":{"_\*\*default_":{"_ttl":{"enabled":\*\*true,"default":86400000},"\*\*properties":{}},"filetype":{"_ **ttl":{"enabled":true,"default"** :86400000},"properties":{"@\*\*fields":{"properties":{"_\*\*second":{"type":"string"},"\*\*fiber1":{"type":"string"},"\*\*fiber2":{"type":"string"},"\*\*hour":{"type":"string"},"\*\*message":{"type":"string"},"\*\*minute":{"type":"string"},"\*\*monthday":{"type":"string"},"\*\*monthnum":{"type":"string"},"\*\*second":{"type":"string"}," **slave":{"type":"string"},"ts":** {"type":"string"},"type1":{"\*\*type":"string"},"type2":{" **type":"string"},"year":{"type"** :"string"}}},"@message":{"\*\*type":"string"},"@source":{" **type":"string"},"@source\_host"** :{"type":"string"},"@source_\*\*path":{"type":"string"},"@\*\*tags":{"type":"string"},"@\*\*timestamp":{"type":"date","\*\*format":"dateOptionalTime"},"@\*\*type":{"type":"string"}}}}} > > > > Where am I going wrong? I am badly stuck here ☹ > > > > Please give valuable suggestions and help > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \ > 'cvml', 'elasticsearch%2Bunsubscribe@googlegroups.com');\>. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \ 'elasticsearch%2Bunsubscribe@googlegroups.com');\>. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 15, 2013, 1:50pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/4 "2013-05-15T13:50:30Z")
Then you have three choices: 1. Wait for a reply 2. Read the documentation 3. Pay for support But resending emails is considered spam. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 15, 2013, 1:52pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/5 "2013-05-15T13:52:16Z")
I understand. But had the doc been written that well, I would not have been asking this question 😛 On Wednesday, May 15, 2013, Clinton Gormley wrote: > Then you have three choices: > > 1. Wait for a reply > 2. Read the documentation > 3. Pay for support > > But resending emails is considered spam. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \ 'elasticsearch%2Bunsubscribe@googlegroups.com');\>. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 15, 2013, 2:29pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/6 "2013-05-15T14:29:14Z")
I have tested out your settings and they work. Which makes me think that either: 1. you are setting a ttl when you index each doc 2. you have disabled expiry 3. you only enabled default expiry after already indexing docs (it doesn't affect existing docs) Any of these true? But as David pointed out, it is much much more efficient to have an index per day, and drop the old indices when you no longer need them On 15 May 2013 15:52, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > I understand. > But had the doc been written that well, I would not have been asking this > question 😛 > > On Wednesday, May 15, 2013, Clinton Gormley wrote: > > > Then you have three choices: > > > > 1. Wait for a reply > > 2. Read the documentation > > 3. Pay for support > > > > But resending emails is considered spam. > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Regards, > Aakash Anuj, > Junior Undergraduate, > Department of Computer Science and Engineering, > Indian Institute of Technology, Kharagpur. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 15, 2013, 2:34pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/7 "2013-05-15T14:34:16Z")
Hey, All I have done is this: I have created a folder mappings/\_default/_default_.json which contains the ttl enabling with the TTL duration. Since I am shipping my logs with logstash, there will be a index per day. What I know is that mappings/\_default/_default_.json will apply the mapping to all new indices created. Do I need to do anything else? They above would enable the TTL, ryt? On Wed, May 15, 2013 at 7:59 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > I have tested out your settings and they work. Which makes me think that > either: > > 1. you are setting a ttl when you index each doc > 2. you have disabled expiry > 3. you only enabled default expiry after already indexing docs (it doesn't > affect existing docs) > > Any of these true? > > But as David pointed out, it is much much more efficient to have an index > per day, and drop the old indices when you no longer need them > > On 15 May 2013 15:52, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > I understand. > > But had the doc been written that well, I would not have been asking this > > question 😛 > > > > On Wednesday, May 15, 2013, Clinton Gormley wrote: > > > > > Then you have three choices: > > > > > > 1. Wait for a reply > > > 2. Read the documentation > > > 3. Pay for support > > > > > > But resending emails is considered spam. > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > > . > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > Regards, > > Aakash Anuj, > > Junior Undergraduate, > > Department of Computer Science and Engineering, > > Indian Institute of Technology, Kharagpur. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 15, 2013, 2:50pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/8 "2013-05-15T14:50:19Z")
I don't know why this isn't working for you. It certainly works for me. But if you're creating an index per day, then you don't need to worry about ttl. You just drop the old indices. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 15, 2013, 2:53pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/9 "2013-05-15T14:53:20Z")
But how to I drop the indices which are old automatically from the ELASTIC SEARCH database? And please explain me one more thing..... When is the file _defaul_.json loaded, that is, when is the TTL enabled? Is it when Elasticsearch runs? Also, how do I actually know if the indices get deleted? Even if I add logs which have a TTL of 1 day and a timestamp sometime before 1 day, they get added to ES. Why ? I sorry for so many questions, but its really getting on my nerves now ☹ On Wed, May 15, 2013 at 8:20 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > I don't know why this isn't working for you. It certainly works for me. > > But if you're creating an index per day, then you don't need to worry > about ttl. You just drop the old indices. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 15, 2013, 2:53pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/10 "2013-05-15T14:53:35Z")
I am sorry\* On Wed, May 15, 2013 at 8:23 PM, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com)wrote: > But how to I drop the indices which are old automatically from the ELASTIC > SEARCH database? > > And please explain me one more thing..... > When is the file _defaul_.json loaded, that is, when is the TTL enabled? > Is it when Elasticsearch runs? > > Also, how do I actually know if the indices get deleted? > Even if I add logs which have a TTL of 1 day and a timestamp sometime > before 1 day, they get added to ES. Why ? > > I sorry for so many questions, but its really getting on my nerves now ☹ > > On Wed, May 15, 2013 at 8:20 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > > > I don't know why this isn't working for you. It certainly works for me. > > > > But if you're creating an index per day, then you don't need to worry > > about ttl. You just drop the old indices. > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Regards, > Aakash Anuj, > Junior Undergraduate, > Department of Computer Science and Engineering, > Indian Institute of Technology, Kharagpur. -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 15, 2013, 3:25pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/11 "2013-05-15T15:25:24Z")
On 15 May 2013 16:53, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > But how to I drop the indices which are old automatically from the ELASTIC > SEARCH database? you don't. you use a cron daemon to run a job once a day > And please explain me one more thing..... > When is the file _defaul_.json loaded, that is, when is the TTL enabled? > Is it when Elasticsearch runs? > > Only when you restart a node. And its values only apply when you create a > new index > Also, how do I actually know if the indices get deleted? You make your cron job send you an email > Even if I add logs which have a TTL of 1 day and a timestamp sometime > before 1 day, they get added to ES. Why ? The ttl depends on the time you index it, not the timestamp field. This could explain why things weren't working the way you expected. Your expectations were incorrect. Please read this advice about how to ask questions in the best way to get answers: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 15, 2013, 3:32pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/12 "2013-05-15T15:32:02Z")
Oh, I guess the last point was where I was fumbling. I had read on a blog somewhere that the TTL applies relative to the timestamp. So do you mean that the TTL applies relative to the system time of creation? On Wed, May 15, 2013 at 8:55 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > On 15 May 2013 16:53, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > But how to I drop the indices which are old automatically from the > > ELASTIC SEARCH database? > > you don't. you use a cron daemon to run a job once a day > > > And please explain me one more thing..... > > When is the file _defaul_.json loaded, that is, when is the TTL enabled? > > Is it when Elasticsearch runs? > > > > Only when you restart a node. And its values only apply when you create > > a new index > > > Also, how do I actually know if the indices get deleted? > > You make your cron job send you an email > > > Even if I add logs which have a TTL of 1 day and a timestamp sometime > > before 1 day, they get added to ES. Why ? > > The ttl depends on the time you index it, not the timestamp field. > > This could explain why things weren't working the way you expected. Your > expectations were incorrect. > > Please read this advice about how to ask questions in the best way to get > answers: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > clint > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 15, 2013, 3:35pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/13 "2013-05-15T15:35:04Z")
I've just asked, and apparently I'm incorrect. ttl and timestamp ARE related. But: i see a "@timestamp" field in your mapping, but no \_timestamp path mapping to point to that field > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. clint On 15 May 2013 17:32, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > Oh, I guess the last point was where I was fumbling. > > I had read on a blog somewhere that the TTL applies relative to the > timestamp. So do you mean that the TTL applies relative to the system time > of creation? > > On Wed, May 15, 2013 at 8:55 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > > > On 15 May 2013 16:53, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > But how to I drop the indices which are old automatically from the > > > ELASTIC SEARCH database? > > > > you don't. you use a cron daemon to run a job once a day > > > > > And please explain me one more thing..... > > > When is the file _defaul_.json loaded, that is, when is the TTL enabled? > > > Is it when Elasticsearch runs? > > > > > > Only when you restart a node. And its values only apply when you create > > > a new index > > > > > Also, how do I actually know if the indices get deleted? > > > > You make your cron job send you an email > > > > > Even if I add logs which have a TTL of 1 day and a timestamp sometime > > > before 1 day, they get added to ES. Why ? > > > > The ttl depends on the time you index it, not the timestamp field. > > > > This could explain why things weren't working the way you expected. Your > > expectations were incorrect. > > > > Please read this advice about how to ask questions in the best way to get > > answers: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > > > clint > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Regards, > Aakash Anuj, > Junior Undergraduate, > Department of Computer Science and Engineering, > Indian Institute of Technology, Kharagpur. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 15, 2013, 6:17pm UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/14 "2013-05-15T18:17:29Z")
You were right I guess....becoz now my logs are getting deleted after the specified time after the indexing, not the time stamp value. Any idea how to automatically delete the logs relative to the time stamp value? On Wednesday, May 15, 2013, Clinton Gormley wrote: > I've just asked, and apparently I'm incorrect. ttl and timestamp ARE > related. > > But: i see a "@timestamp" field in your mapping, but no \_timestamp path > mapping to point to that field > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field/) > > clint > > On 15 May 2013 17:32, Aakash Anuj \<[aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com)\ > > > wrote: > > > Oh, I guess the last point was where I was fumbling. > > > > I had read on a blog somewhere that the TTL applies relative to the > > timestamp. So do you mean that the TTL applies relative to the system time > > of creation? > > > > On Wed, May 15, 2013 at 8:55 PM, Clinton Gormley \<[clint@traveljury.com](mailto:clint@traveljury.com)\ > > > > > wrote: > > > > > On 15 May 2013 16:53, Aakash Anuj \<[aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com)\ > > > > > > > wrote: > > > > > > > But how to I drop the indices which are old automatically from the > > > > ELASTIC SEARCH database? > > > > > > you don't. you use a cron daemon to run a job once a day > > > > > > > And please explain me one more thing..... > > > > When is the file _defaul_.json loaded, that is, when is the TTL > > > > enabled? Is it when Elasticsearch runs? > > > > > > > > Only when you restart a node. And its values only apply when you > > > > create a new index > > > > > > > Also, how do I actually know if the indices get deleted? > > > > > > You make your cron job send you an email > > > > > > > Even if I add logs which have a TTL of 1 day and a timestamp sometime > > > > before 1 day, they get added to ES. Why ? > > > > > > The ttl depends on the time you index it, not the timestamp field. > > > > > > This could explain why things weren't working the way you expected. > > > Your expectations were incorrect. > > > > > > Please read this advice about how to ask questions in the best way to > > > get answers: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > > > > > clint > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > > . > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \ > > 'elasticsearch%2Bunsubscribe@googlegroups.com');\>. > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > Regards, > > Aakash Anuj, > > Junior Undergraduate, > > Department of Computer Science and Engineering, > > Indian Institute of Technology, Kharagpur. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \ > 'cvml', 'elasticsearch%2Bunsubscribe@googlegroups.com');\>. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \ 'elasticsearch%2Bunsubscribe@googlegroups.com');\>. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 16, 2013, 5:39am UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/15 "2013-05-16T05:39:22Z")
Could you help me with this? [https://groups.google.com/forum/?fromgroups=#!topic/elasticsearch/2jUyZY8B30Q](https://groups.google.com/forum/?fromgroups=#!topic/elasticsearch/2jUyZY8B30Q) On Wed, May 15, 2013 at 11:47 PM, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com)wrote: > You were right I guess....becoz now my logs are getting deleted after the > specified time after the indexing, not the time stamp value. Any idea how > to automatically delete the logs relative to the time stamp value? > > On Wednesday, May 15, 2013, Clinton Gormley wrote: > > > I've just asked, and apparently I'm incorrect. ttl and timestamp ARE > > related. > > > > But: i see a "@timestamp" field in your mapping, but no \_timestamp path > > mapping to point to that field > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field/) > > > > clint > > > > On 15 May 2013 17:32, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > Oh, I guess the last point was where I was fumbling. > > > > > > I had read on a blog somewhere that the TTL applies relative to the > > > timestamp. So do you mean that the TTL applies relative to the system time > > > of creation? > > > > > > On Wed, May 15, 2013 at 8:55 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > > > > > > > On 15 May 2013 16:53, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > > > > > But how to I drop the indices which are old automatically from the > > > > > ELASTIC SEARCH database? > > > > > > > > you don't. you use a cron daemon to run a job once a day > > > > > > > > > And please explain me one more thing..... > > > > > When is the file _defaul_.json loaded, that is, when is the TTL > > > > > enabled? Is it when Elasticsearch runs? > > > > > > > > > > Only when you restart a node. And its values only apply when you > > > > > create a new index > > > > > > > > > Also, how do I actually know if the indices get deleted? > > > > > > > > You make your cron job send you an email > > > > > > > > > Even if I add logs which have a TTL of 1 day and a timestamp sometime > > > > > before 1 day, they get added to ES. Why ? > > > > > > > > The ttl depends on the time you index it, not the timestamp field. > > > > > > > > This could explain why things weren't working the way you expected. > > > > Your expectations were incorrect. > > > > > > > > Please read this advice about how to ask questions in the best way to > > > > get answers: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > > > > > > > clint > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > Regards, > > > Aakash Anuj, > > > Junior Undergraduate, > > > Department of Computer Science and Engineering, > > > Indian Institute of Technology, Kharagpur. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Regards, > Aakash Anuj, > Junior Undergraduate, > Department of Computer Science and Engineering, > Indian Institute of Technology, Kharagpur. -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 16, 2013, 9:27am UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/16 "2013-05-16T09:27:11Z")
Use delete-by-query On 15 May 2013 20:17, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > You were right I guess....becoz now my logs are getting deleted after the > specified time after the indexing, not the time stamp value. Any idea how > to automatically delete the logs relative to the time stamp value? > > On Wednesday, May 15, 2013, Clinton Gormley wrote: > > > I've just asked, and apparently I'm incorrect. ttl and timestamp ARE > > related. > > > > But: i see a "@timestamp" field in your mapping, but no \_timestamp path > > mapping to point to that field > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field/) > > > > clint > > > > On 15 May 2013 17:32, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > Oh, I guess the last point was where I was fumbling. > > > > > > I had read on a blog somewhere that the TTL applies relative to the > > > timestamp. So do you mean that the TTL applies relative to the system time > > > of creation? > > > > > > On Wed, May 15, 2013 at 8:55 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > > > > > > > On 15 May 2013 16:53, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > > > > > But how to I drop the indices which are old automatically from the > > > > > ELASTIC SEARCH database? > > > > > > > > you don't. you use a cron daemon to run a job once a day > > > > > > > > > And please explain me one more thing..... > > > > > When is the file _defaul_.json loaded, that is, when is the TTL > > > > > enabled? Is it when Elasticsearch runs? > > > > > > > > > > Only when you restart a node. And its values only apply when you > > > > > create a new index > > > > > > > > > Also, how do I actually know if the indices get deleted? > > > > > > > > You make your cron job send you an email > > > > > > > > > Even if I add logs which have a TTL of 1 day and a timestamp sometime > > > > > before 1 day, they get added to ES. Why ? > > > > > > > > The ttl depends on the time you index it, not the timestamp field. > > > > > > > > This could explain why things weren't working the way you expected. > > > > Your expectations were incorrect. > > > > > > > > Please read this advice about how to ask questions in the best way to > > > > get answers: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > > > > > > > clint > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > Regards, > > > Aakash Anuj, > > > Junior Undergraduate, > > > Department of Computer Science and Engineering, > > > Indian Institute of Technology, Kharagpur. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Regards, > Aakash Anuj, > Junior Undergraduate, > Department of Computer Science and Engineering, > Indian Institute of Technology, Kharagpur. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![aakashanuj](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@aakashanuj](https://discuss.elastic.co/u/aakashanuj)\ **Post date:** [May 16, 2013, 9:28am UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/17 "2013-05-16T09:28:10Z")
I want it to be automatic . Is it possible? On Thu, May 16, 2013 at 2:57 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > Use delete-by-query > > On 15 May 2013 20:17, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > You were right I guess....becoz now my logs are getting deleted after the > > specified time after the indexing, not the time stamp value. Any idea how > > to automatically delete the logs relative to the time stamp value? > > > > On Wednesday, May 15, 2013, Clinton Gormley wrote: > > > > > I've just asked, and apparently I'm incorrect. ttl and timestamp ARE > > > related. > > > > > > But: i see a "@timestamp" field in your mapping, but no \_timestamp path > > > mapping to point to that field > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field/) > > > > > > clint > > > > > > On 15 May 2013 17:32, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > > > Oh, I guess the last point was where I was fumbling. > > > > > > > > I had read on a blog somewhere that the TTL applies relative to the > > > > timestamp. So do you mean that the TTL applies relative to the system time > > > > of creation? > > > > > > > > On Wed, May 15, 2013 at 8:55 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > > > > > > > > > On 15 May 2013 16:53, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > > > > > > > But how to I drop the indices which are old automatically from the > > > > > > ELASTIC SEARCH database? > > > > > > > > > > you don't. you use a cron daemon to run a job once a day > > > > > > > > > > > And please explain me one more thing..... > > > > > > When is the file _defaul_.json loaded, that is, when is the TTL > > > > > > enabled? Is it when Elasticsearch runs? > > > > > > > > > > > > Only when you restart a node. And its values only apply when you > > > > > > create a new index > > > > > > > > > > > Also, how do I actually know if the indices get deleted? > > > > > > > > > > You make your cron job send you an email > > > > > > > > > > > Even if I add logs which have a TTL of 1 day and a timestamp sometime > > > > > > before 1 day, they get added to ES. Why ? > > > > > > > > > > The ttl depends on the time you index it, not the timestamp field. > > > > > > > > > > This could explain why things weren't working the way you expected. > > > > > Your expectations were incorrect. > > > > > > > > > > Please read this advice about how to ask questions in the best way to > > > > > get answers: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > > > > > > > > > clint > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > Regards, > > > > Aakash Anuj, > > > > Junior Undergraduate, > > > > Department of Computer Science and Engineering, > > > > Indian Institute of Technology, Kharagpur. > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > > . > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > Regards, > > Aakash Anuj, > > Junior Undergraduate, > > Department of Computer Science and Engineering, > > Indian Institute of Technology, Kharagpur. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Regards, Aakash Anuj, Junior Undergraduate, Department of Computer Science and Engineering, Indian Institute of Technology, Kharagpur. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [May 16, 2013, 9:38am UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/18 "2013-05-16T09:38:31Z")
No On 16 May 2013 11:28, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > I want it to be automatic . Is it possible? > > On Thu, May 16, 2013 at 2:57 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > > > Use delete-by-query > > > > On 15 May 2013 20:17, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > You were right I guess....becoz now my logs are getting deleted after > > > the specified time after the indexing, not the time stamp value. Any idea > > > how to automatically delete the logs relative to the time stamp value? > > > > > > On Wednesday, May 15, 2013, Clinton Gormley wrote: > > > > > > > I've just asked, and apparently I'm incorrect. ttl and timestamp ARE > > > > related. > > > > > > > > But: i see a "@timestamp" field in your mapping, but no \_timestamp path > > > > mapping to point to that field > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field/) > > > > > > > > clint > > > > > > > > On 15 May 2013 17:32, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com) wrote: > > > > > > > > > Oh, I guess the last point was where I was fumbling. > > > > > > > > > > I had read on a blog somewhere that the TTL applies relative to the > > > > > timestamp. So do you mean that the TTL applies relative to the system time > > > > > of creation? > > > > > > > > > > On Wed, May 15, 2013 at 8:55 PM, Clinton Gormley \<[clint@traveljury.com](mailto:clint@traveljury.com) > > > > > > > > > > > wrote: > > > > > > > > > > > On 15 May 2013 16:53, Aakash Anuj [aakashanuj.iitkgp@gmail.com](mailto:aakashanuj.iitkgp@gmail.com)wrote: > > > > > > > > > > > > > But how to I drop the indices which are old automatically from the > > > > > > > ELASTIC SEARCH database? > > > > > > > > > > > > you don't. you use a cron daemon to run a job once a day > > > > > > > > > > > > > And please explain me one more thing..... > > > > > > > When is the file _defaul_.json loaded, that is, when is the TTL > > > > > > > enabled? Is it when Elasticsearch runs? > > > > > > > > > > > > > > Only when you restart a node. And its values only apply when you > > > > > > > create a new index > > > > > > > > > > > > > Also, how do I actually know if the indices get deleted? > > > > > > > > > > > > You make your cron job send you an email > > > > > > > > > > > > > Even if I add logs which have a TTL of 1 day and a timestamp > > > > > > > sometime before 1 day, they get added to ES. Why ? > > > > > > > > > > > > The ttl depends on the time you index it, not the timestamp field. > > > > > > > > > > > > This could explain why things weren't working the way you expected. > > > > > > Your expectations were incorrect. > > > > > > > > > > > > Please read this advice about how to ask questions in the best way to > > > > > > get answers: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/help/) > > > > > > > > > > > > clint > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in > > > > > > the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit > > > > > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > > > > > . > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > Regards, > > > > > Aakash Anuj, > > > > > Junior Undergraduate, > > > > > Department of Computer Science and Engineering, > > > > > Indian Institute of Technology, Kharagpur. > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > Regards, > > > Aakash Anuj, > > > Junior Undergraduate, > > > Department of Computer Science and Engineering, > > > Indian Institute of Technology, Kharagpur. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/3j77O3pOCRc/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Regards, > Aakash Anuj, > Junior Undergraduate, > Department of Computer Science and Engineering, > Indian Institute of Technology, Kharagpur. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:36am UTC](https://discuss.elastic.co/t/ttl-set-and-enabled-but-logs-not-expiring-please-help/11974/19 "2017-07-06T02:36:21Z")