TTL in elastic search is not working

I am stuck in implementing the TTL in the case of elastic search.

What I have done is this:

I have created a folder mappings under the folder config.

Under mappings, I have a folder _default , under which I have kept the file
default.json and it contains

{
"default" : {
"_ttl" : { "enabled" : true, "default" : "1d" }
}
}

I am shipping the logs via logstash and I want my docs to have a TTL of 1
day. But this does not work, Why?

Am i missing something? Is there any requirement of the API? Please help.

I am stuck here since long...
Awaiting a reply

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hey,

you can get back the mapping of your logstash index and see if the TTL is
set in the mapping?
Also, enabling TTL in your mapping does not have any impact on your already
indexed data - you need to delete the already indexed data manually (by
query with a date).

--Alex

On Wed, May 15, 2013 at 9:27 AM, Aakash Anuj aakashanuj.iitkgp@gmail.comwrote:

I am stuck in implementing the TTL in the case of elastic search.

What I have done is this:

I have created a folder mappings under the folder config.

Under mappings, I have a folder _default , under which I have kept the
file default.json and it contains

{
"default" : {
"_ttl" : { "enabled" : true, "default" : "1d" }
}
}

I am shipping the logs via logstash and I want my docs to have a TTL of 1
day. But this does not work, Why?

Am i missing something? Is there any requirement of the API? Please help.

I am stuck here since long...
Awaiting a reply

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I add to Alex's comment that you should also think of having rolling indexes.
This is really more efficient.

One index per day with an alias on top of it and just drop the old index everyday.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 15 mai 2013 à 09:43, Alexander Reelsen alr@spinscale.de a écrit :

Hey,

you can get back the mapping of your logstash index and see if the TTL is set in the mapping?
Also, enabling TTL in your mapping does not have any impact on your already indexed data - you need to delete the already indexed data manually (by query with a date).

--Alex

On Wed, May 15, 2013 at 9:27 AM, Aakash Anuj aakashanuj.iitkgp@gmail.com wrote:
I am stuck in implementing the TTL in the case of elastic search.

What I have done is this:

I have created a folder mappings under the folder config.

Under mappings, I have a folder _default , under which I have kept the file default.json and it contains
{
"default" : {
"_ttl" : { "enabled" : true, "default" : "1d" }
}
}
I am shipping the logs via logstash and I want my docs to have a TTL of 1 day. But this does not work, Why?

Am i missing something? Is there any requirement of the API? Please help.

I am stuck here since long...
Awaiting a reply

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Can you tell me how to get back the mapping of the index?

I am using Logstash to export the data to elastic search.

How do I make sure that it is enabled? Is specifying the default mapping as
stated by me in the file default.json not enough?

On Wed, May 15, 2013 at 1:13 PM, Alexander Reelsen alr@spinscale.de wrote:

Hey,

you can get back the mapping of your logstash index and see if the TTL is
set in the mapping?
Also, enabling TTL in your mapping does not have any impact on your
already indexed data - you need to delete the already indexed data manually
(by query with a date).

--Alex

On Wed, May 15, 2013 at 9:27 AM, Aakash Anuj aakashanuj.iitkgp@gmail.comwrote:

I am stuck in implementing the TTL in the case of elastic search.

What I have done is this:

I have created a folder mappings under the folder config.

Under mappings, I have a folder _default , under which I have kept the
file default.json and it contains

{
"default" : {
"_ttl" : { "enabled" : true, "default" : "1d" }
}
}

I am shipping the logs via logstash and I want my docs to have a TTL of 1
day. But this does not work, Why?

Am i missing something? Is there any requirement of the API? Please help.

I am stuck here since long...
Awaiting a reply

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/ol4P2zau3Uc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Regards,
Aakash Anuj,
Junior Undergraduate,
Department of Computer Science and Engineering,
Indian Institute of Technology, Kharagpur.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hey

just point a HTTP client to the _mapping URI of an index, or even to
/_mapping to see the mapping for all indices.
See
http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping/

--Alex

On Wed, May 15, 2013 at 9:46 AM, Aakash Anuj aakashanuj.iitkgp@gmail.comwrote:

Can you tell me how to get back the mapping of the index?

I am using Logstash to export the data to elastic search.

How do I make sure that it is enabled? Is specifying the default mapping
as stated by me in the file default.json not enough?

On Wed, May 15, 2013 at 1:13 PM, Alexander Reelsen alr@spinscale.dewrote:

Hey,

you can get back the mapping of your logstash index and see if the TTL is
set in the mapping?
Also, enabling TTL in your mapping does not have any impact on your
already indexed data - you need to delete the already indexed data manually
(by query with a date).

--Alex

On Wed, May 15, 2013 at 9:27 AM, Aakash Anuj <aakashanuj.iitkgp@gmail.com

wrote:

I am stuck in implementing the TTL in the case of elastic search.

What I have done is this:

I have created a folder mappings under the folder config.

Under mappings, I have a folder _default , under which I have kept the
file default.json and it contains

{
"default" : {
"_ttl" : { "enabled" : true, "default" : "1d" }
}
}

I am shipping the logs via logstash and I want my docs to have a TTL of
1 day. But this does not work, Why?

Am i missing something? Is there any requirement of the API? Please help.

I am stuck here since long...
Awaiting a reply

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/ol4P2zau3Uc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Regards,
Aakash Anuj,
Junior Undergraduate,
Department of Computer Science and Engineering,
Indian Institute of Technology, Kharagpur.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hey, I have TTL enabled, but the logs are not removed when I view them from
Kibana. They still show up.
Here is the mapping:

{"logstash-2013.05.14":{"default":{"_ttl":{"enabled":true,"default":86400000},"properties":{}},"filetype":{"_ttl":{"enabled":true,"default":86400000},"properties":{"@fields":{"properties":{"_second":{"type":"string"},"fiber1":{"type":"string"},"fiber2":{"type":"string"},"hour":{"type":"string"},"message":{"type":"string"},"minute":{"type":"string"},"monthday":{"type":"string"},"monthnum":{"type":"string"},"second":{"type":"string"},"slave":{"type":"string"},"ts":{"type":"string"},"type1":{"type":"string"},"type2":{"type":"string"},"year":{"type":"string"}}},"@message":{"type":"string"},"@source":{"type":"string"},"@source_host":{"type":"string"},"@source_path":{"type":"string"},"@tags":{"type":"string"},"@timestamp":{"type":"date","format":"dateOptionalTime"},"@type":{"type":"string"}}}},"logstash-2013.05.09":{"default":{"_ttl":{"enabled":true,"default":86400000},"properties":{}},"filetype":{"_ttl":{"enabled":true,"default":86400000},"properties":{"@fields":{"properties":{"_second":{"type":"string"},"fiber1":{"type":"string"},"fiber2":{"type":"string"},"hour":{"type":"string"},"message":{"type":"string"},"minute":{"type":"string"},"monthday":{"type":"string"},"monthnum":{"type":"string"},"second":{"type":"string"},"slave":{"type":"string"},"ts":{"type":"string"},"type1":{"type":"string"},"type2":{"type":"string"},"year":{"type":"string"}}},"@message":{"type":"string"},"@source":{"type":"string"},"@source_host":{"type":"string"},"@source_path":{"type":"string"},"@tags":{"type":"string"},"@timestamp":{"type":"date","format":"dateOptionalTime"},"@type":{"type":"string"}}}}}

What do I do?

On Wed, May 15, 2013 at 2:53 PM, Alexander Reelsen alr@spinscale.de wrote:

Hey

just point a HTTP client to the _mapping URI of an index, or even to
/_mapping to see the mapping for all indices.
See
http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping/

--Alex

On Wed, May 15, 2013 at 9:46 AM, Aakash Anuj aakashanuj.iitkgp@gmail.comwrote:

Can you tell me how to get back the mapping of the index?

I am using Logstash to export the data to elastic search.

How do I make sure that it is enabled? Is specifying the default mapping
as stated by me in the file default.json not enough?

On Wed, May 15, 2013 at 1:13 PM, Alexander Reelsen alr@spinscale.dewrote:

Hey,

you can get back the mapping of your logstash index and see if the TTL
is set in the mapping?
Also, enabling TTL in your mapping does not have any impact on your
already indexed data - you need to delete the already indexed data manually
(by query with a date).

--Alex

On Wed, May 15, 2013 at 9:27 AM, Aakash Anuj <
aakashanuj.iitkgp@gmail.com> wrote:

I am stuck in implementing the TTL in the case of elastic search.

What I have done is this:

I have created a folder mappings under the folder config.

Under mappings, I have a folder _default , under which I have kept the
file default.json and it contains

{
"default" : {
"_ttl" : { "enabled" : true, "default" : "1d" }
}
}

I am shipping the logs via logstash and I want my docs to have a TTL of
1 day. But this does not work, Why?

Am i missing something? Is there any requirement of the API? Please
help.

I am stuck here since long...
Awaiting a reply

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/ol4P2zau3Uc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Regards,
Aakash Anuj,
Junior Undergraduate,
Department of Computer Science and Engineering,
Indian Institute of Technology, Kharagpur.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/ol4P2zau3Uc/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Regards,
Aakash Anuj,
Junior Undergraduate,
Department of Computer Science and Engineering,
Indian Institute of Technology, Kharagpur.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.