ES consistently giving not able to parse _ttl exception

Arjit_Gupta · August 10, 2013, 2:43am

Hi,

We are consistently getting exception - I dont know why is it not able to
parse _ttl.

[2013-08-09 01:18:22,929][DEBUG][action.index ] [search5]
[internal_index_hor][1], node[MwQNQpm8Q7GqPBSWaC2rRQ], [P], s[STARTED]:
Failed to execute [index

{[internal_index_hor][internal_index][relationship_namespace],
source[{"refresh_exact":"true","refresh":"true"}]}]

org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[_ttl]

at

org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:396)

at

org.elasticsearch.index.mapper.internal.TTLFieldMapper.postParse(TTLFieldMapper.java:167)

at

org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:524)

at

org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:450)

at

org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:327)

at

org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:203)

at

org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.

java:532)

at

org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:430)

at

java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

    at java.lang.Thread.run(Thread.java:662)

Caused by: org.elasticsearch.index.AlreadyExpiredException: already expired
[internal_index_hor]/[internal_index]/[relationship_namespace] due to
expire at [13

75991302672] and was processed at [1375991302928]

at

org.elasticsearch.index.mapper.internal.TTLFieldMapper.innerParseCreateField(TTLFieldMapper.java:205)

at

org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:167)

at

org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:385)

    ... 10 more

{

state: open
settings: {
- index.analysis.analyzer.my_analyzer.tokenizer: keyword
- index.analysis.analyzer.my_analyzer.filter: lowercase
- index.number_of_shards: 5
- index.number_of_replicas: 2
- index.version.created: 900199
- index.refresh_interval: 1s
}
mappings: {
- internal_index: {
  - _ttl: {
    - enabled: true
    - default: 1000
  - }
  - properties: {
    - index_version_qual: {
      - include_in_all: false
      - store: true
      - analyzer: my_analyzer
      - type: string
    - }
    - refresh_qual: {
      - include_in_all: false
      - store: true
      - analyzer: my_analyzer
      - type: string
    - }
    - entity_version: {
      - include_in_all: false
      - store: true
      - type: long
    - }
    - index_version: {
      - include_in_all: false
      - store: true
      - type: long
    - }
    - refresh_exact: {
      - type: string
    - }
    - entity_version_qual: {
      - include_in_all: false
      - store: true
      - analyzer: my_analyzer
      - type: string
    - }
    - refresh: {
      - include_in_all: false
      - store: true
      - analyzer: my_analyzer
      - type: string
    - }
  - }
- }
}
aliases: [
- internal_index
]

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Benjamin_Deveze · August 10, 2013, 11:06am

Hi Arjit,

from this message:

Caused by: org.elasticsearch.index.AlreadyExpiredException: already expired
[internal_index_hor]/[internal_index]/[relationship_namespace] due to
expire at [13

75991302672] and was processed at [1375991302928]

it seems like you are trying to index an already expired document.

This seems to be explained here by your default ttl value set to "1000"
which is in milliseconds and stands for 1s.

Hope it helps

On Saturday, August 10, 2013 4:43:04 AM UTC+2, Arjit Gupta wrote:

Hi,

We are consistently getting exception - I dont know why is it not able to
parse _ttl.

[2013-08-09 01:18:22,929][DEBUG][action.index ] [search5]
[internal_index_hor][1], node[MwQNQpm8Q7GqPBSWaC2rRQ], [P], s[STARTED]:
Failed to execute [index

{[internal_index_hor][internal_index][relationship_namespace],
source[{"refresh_exact":"true","refresh":"true"}]}]

org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[_ttl]
    at 
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:396)
    at 
org.elasticsearch.index.mapper.internal.TTLFieldMapper.postParse(TTLFieldMapper.java:167)
    at 
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:524)
    at 
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:450)
    at 
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:327)
    at 
org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:203)
    at 
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.

java:532)
    at 
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:430)
    at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: org.elasticsearch.index.AlreadyExpiredException: already
expired [internal_index_hor]/[internal_index]/[relationship_namespace] due
to expire at [13

75991302672] and was processed at [1375991302928]
    at 
org.elasticsearch.index.mapper.internal.TTLFieldMapper.innerParseCreateField(TTLFieldMapper.java:205)
    at 
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:167)
    at 
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:385)
    ... 10 more
{

state: open

settings: {

index.analysis.analyzer.my_analyzer.tokenizer: keyword

index.analysis.analyzer.my_analyzer.filter: lowercase

index.number_of_shards: 5

index.number_of_replicas: 2

index.version.created: 900199

index.refresh_interval: 1s

}

mappings: {

internal_index: {

_ttl: {

enabled: true

default: 1000

}

properties: {

index_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

entity_version: {

include_in_all: false

store: true

type: long

}

index_version: {

include_in_all: false

store: true

type: long

}

refresh_exact: {

type: string

}

entity_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

}

}

}

aliases: [

internal_index

]

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Arjit_Gupta · August 12, 2013, 10:36am

Thanks a lot.
But I cant index an expired document ?
Once a document is expired I cant index the same document ?

Thanks ,
Arjit

On Sat, Aug 10, 2013 at 4:36 PM, Benjamin Devèze
benjamin.deveze@gmail.comwrote:

Hi Arjit,

from this message:

Caused by: org.elasticsearch.index.**AlreadyExpiredException: already
expired [internal_index_hor]/[**internal_index]/[relationship_**namespace]
due to expire at [13

75991302672] and was processed at [1375991302928]

it seems like you are trying to index an already expired document.

This seems to be explained here by your default ttl value set to "1000"
which is in milliseconds and stands for 1s.

Hope it helps

On Saturday, August 10, 2013 4:43:04 AM UTC+2, Arjit Gupta wrote:
Hi,

We are consistently getting exception - I dont know why is it not able to
parse _ttl.

[2013-08-09 01:18:22,929][DEBUG][action.**index ] [search5]
[internal_index_hor][1], node[MwQNQpm8Q7GqPBSWaC2rRQ], [P], s[STARTED]:
Failed to execute [index

{[internal_index_hor][**internal_index][relationship_namespace],
source[{"refresh_exact":"true","refresh":"true"}]}]

org.elasticsearch.index.**mapper.MapperParsingException: failed to parse
[_ttl]
    at org.elasticsearch.index.**mapper.core.**
AbstractFieldMapper.parse(**AbstractFieldMapper.java:396)
    at org.elasticsearch.index.**mapper.internal.**
TTLFieldMapper.postParse(**TTLFieldMapper.java:167)
    at org.elasticsearch.index.**mapper.DocumentMapper.parse(**
DocumentMapper.java:524)
    at org.elasticsearch.index.**mapper.DocumentMapper.parse(**
DocumentMapper.java:450)
    at org.elasticsearch.index.shard.**service.InternalIndexShard.**
prepareIndex(**InternalIndexShard.java:327)
    at org.elasticsearch.action.**index.TransportIndexAction.**
shardOperationOnPrimary(**TransportIndexAction.java:203)
    at org.elasticsearch.action.**support.replication.**
TransportShardReplicationOpera**tionAction$AsyncShardOperationAction.
performOnPrimary(TransportShardReplicationOperationAction.

java:532)
    at org.elasticsearch.action.**support.replication.**
TransportShardReplicationOpera**tionAction$**AsyncShardOperationAction$1.
**run(TransportShardReplicationOperationAction.java:430)
    at java.util.concurrent.**ThreadPoolExecutor$Worker.**
runTask(ThreadPoolExecutor.**java:886)
    at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.**java:662)
Caused by: org.elasticsearch.index.**AlreadyExpiredException: already
expired [internal_index_hor]/[**internal_index]/[relationship_**namespace]
due to expire at [13

75991302672] and was processed at [1375991302928]
    at org.elasticsearch.index.**mapper.internal.**TTLFieldMapper.**
innerParseCreateField(**TTLFieldMapper.java:205)
    at org.elasticsearch.index.**mapper.core.NumberFieldMapper.**
parseCreateField(**NumberFieldMapper.java:167)
    at org.elasticsearch.index.**mapper.core.**
AbstractFieldMapper.parse(**AbstractFieldMapper.java:385)
    ... 10 more
{

state: open

settings: {

index.analysis.analyzer.my_**analyzer.tokenizer: keyword

index.analysis.analyzer.my_**analyzer.filter: lowercase

index.number_of_shards: 5

index.number_of_replicas: 2

index.version.created: 900199

index.refresh_interval: 1s

}

mappings: {

internal_index: {

_ttl: {

enabled: true

default: 1000

}

properties: {

index_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

entity_version: {

include_in_all: false

store: true

type: long

}

index_version: {

include_in_all: false

store: true

type: long

}

refresh_exact: {

type: string

}

entity_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

}

}

}

aliases: [

internal_index

]

}
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Benjamin_Deveze · August 12, 2013, 2:37pm

It makes no sense to index an expired document that is going to be deleted
right away.

You can of course reindex the same document with an expiration date in the
future or update the ttl of an existing document.

On Mon, Aug 12, 2013 at 12:36 PM, Arjit Gupta arjit292@gmail.com wrote:

Thanks a lot.
But I cant index an expired document ?
Once a document is expired I cant index the same document ?

Thanks ,
Arjit

On Sat, Aug 10, 2013 at 4:36 PM, Benjamin Devèze <
benjamin.deveze@gmail.com> wrote:
Hi Arjit,

from this message:

Caused by: org.elasticsearch.index.**AlreadyExpiredException: already
expired [internal_index_hor]/[**internal_index]/[relationship_**namespace]
due to expire at [13

75991302672] and was processed at [1375991302928]

it seems like you are trying to index an already expired document.

This seems to be explained here by your default ttl value set to "1000"
which is in milliseconds and stands for 1s.

Hope it helps

On Saturday, August 10, 2013 4:43:04 AM UTC+2, Arjit Gupta wrote:
Hi,

We are consistently getting exception - I dont know why is it not able
to parse _ttl.

[2013-08-09 01:18:22,929][DEBUG][action.**index ] [search5]
[internal_index_hor][1], node[MwQNQpm8Q7GqPBSWaC2rRQ], [P], s[STARTED]:
Failed to execute [index

{[internal_index_hor][**internal_index][relationship_namespace],
source[{"refresh_exact":"true","refresh":"true"}]}]

org.elasticsearch.index.**mapper.MapperParsingException: failed to
parse [_ttl]
    at org.elasticsearch.index.**mapper.core.**
AbstractFieldMapper.parse(**AbstractFieldMapper.java:396)
    at org.elasticsearch.index.**mapper.internal.**
TTLFieldMapper.postParse(**TTLFieldMapper.java:167)
    at org.elasticsearch.index.**mapper.DocumentMapper.parse(**
DocumentMapper.java:524)
    at org.elasticsearch.index.**mapper.DocumentMapper.parse(**
DocumentMapper.java:450)
    at org.elasticsearch.index.shard.**service.InternalIndexShard.**
prepareIndex(**InternalIndexShard.java:327)
    at org.elasticsearch.action.**index.TransportIndexAction.**
shardOperationOnPrimary(**TransportIndexAction.java:203)
    at org.elasticsearch.action.**support.replication.**
TransportShardReplicationOpera**tionAction$*AsyncShardOperationAction.
*performOnPrimary(TransportShardReplicationOperationAction.

java:532)
    at org.elasticsearch.action.**support.replication.**
TransportShardReplicationOperationAction$
AsyncShardOperationAction$1.**run(TransportShardReplicationOpera
tionAction.java:430)
    at java.util.concurrent.**ThreadPoolExecutor$Worker.**
runTask(ThreadPoolExecutor.**java:886)
    at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.**java:662)
Caused by: org.elasticsearch.index.**AlreadyExpiredException: already
expired [internal_index_hor]/[**internal_index]/[relationship_**namespace]
due to expire at [13

75991302672] and was processed at [1375991302928]
    at org.elasticsearch.index.**mapper.internal.**TTLFieldMapper.**
innerParseCreateField(**TTLFieldMapper.java:205)
    at org.elasticsearch.index.**mapper.core.NumberFieldMapper.**
parseCreateField(**NumberFieldMapper.java:167)
    at org.elasticsearch.index.**mapper.core.**
AbstractFieldMapper.parse(**AbstractFieldMapper.java:385)
    ... 10 more
{

state: open

settings: {

index.analysis.analyzer.my_**analyzer.tokenizer: keyword

index.analysis.analyzer.my_**analyzer.filter: lowercase

index.number_of_shards: 5

index.number_of_replicas: 2

index.version.created: 900199

index.refresh_interval: 1s

}

mappings: {

internal_index: {

_ttl: {

enabled: true

default: 1000

}

properties: {

index_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

entity_version: {

include_in_all: false

store: true

type: long

}

index_version: {

include_in_all: false

store: true

type: long

}

refresh_exact: {

type: string

}

entity_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

}

}

}

aliases: [

internal_index

]

}
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Benjamin DEVEZE

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Arjit_Gupta · August 24, 2013, 11:06am

I am not trying to index anything its Elastic search which is performing
some operation. I am not able to get from the stack trace.

org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[_ttl]
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:396)
at
org.elasticsearch.index.mapper.internal.TTLFieldMapper.postParse(TTLFieldMapper.java:167)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:524)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:450)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:327)
at
org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:203)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:532)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:430)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.elasticsearch.index.AlreadyExpiredException: already expired
[internal_index_hor]/[internal_index]/[relationship_namespace] due to
expire at [1377319486435] and was processed at [1377319527779]
at
org.elasticsearch.index.mapper.internal.TTLFieldMapper.innerParseCreateField(TTLFieldMapper.java:205)
at
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:167)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:385)
... 10 more

Thanks ,
Arjit

On Mon, Aug 12, 2013 at 8:07 PM, Benjamin Devèze
benjamin.deveze@gmail.comwrote:

It makes no sense to index an expired document that is going to be deleted
right away.

You can of course reindex the same document with an expiration date in the
future or update the ttl of an existing document.

On Mon, Aug 12, 2013 at 12:36 PM, Arjit Gupta arjit292@gmail.com wrote:
Thanks a lot.
But I cant index an expired document ?
Once a document is expired I cant index the same document ?

Thanks ,
Arjit

On Sat, Aug 10, 2013 at 4:36 PM, Benjamin Devèze <
benjamin.deveze@gmail.com> wrote:
Hi Arjit,

from this message:

Caused by: org.elasticsearch.index.**AlreadyExpiredException: already
expired [internal_index_hor]/[**internal_index]/[relationship_**namespace]
due to expire at [13

75991302672] and was processed at [1375991302928]

it seems like you are trying to index an already expired document.

This seems to be explained here by your default ttl value set to "1000"
which is in milliseconds and stands for 1s.

Hope it helps

On Saturday, August 10, 2013 4:43:04 AM UTC+2, Arjit Gupta wrote:
Hi,

We are consistently getting exception - I dont know why is it not able
to parse _ttl.

[2013-08-09 01:18:22,929][DEBUG][action.**index ]
[search5] [internal_index_hor][1], node[MwQNQpm8Q7GqPBSWaC2rRQ], [P],
s[STARTED]: Failed to execute [index

{[internal_index_hor][**internal_index][relationship_namespace],
source[{"refresh_exact":"true","refresh":"true"}]}]

org.elasticsearch.index.**mapper.MapperParsingException: failed to
parse [_ttl]
    at org.elasticsearch.index.**mapper.core.**
AbstractFieldMapper.parse(**AbstractFieldMapper.java:396)
    at org.elasticsearch.index.**mapper.internal.**
TTLFieldMapper.postParse(**TTLFieldMapper.java:167)
    at org.elasticsearch.index.**mapper.DocumentMapper.parse(**
DocumentMapper.java:524)
    at org.elasticsearch.index.**mapper.DocumentMapper.parse(**
DocumentMapper.java:450)
    at org.elasticsearch.index.shard.**service.InternalIndexShard.*
*prepareIndex(**InternalIndexShard.java:327)
    at org.elasticsearch.action.**index.TransportIndexAction.**
shardOperationOnPrimary(**TransportIndexAction.java:203)
    at org.elasticsearch.action.**support.replication.**
TransportShardReplicationOpera**tionAction$**AsyncShardOperationAction.
**performOnPrimary(TransportShardReplicationOperationAction.

java:532)
    at org.elasticsearch.action.**support.replication.**
TransportShardReplicationOperationAction$
AsyncShardOperationAction$1.**run(TransportShardReplicationOpera
tionAction.java:430)
    at java.util.concurrent.**ThreadPoolExecutor$Worker.**
runTask(ThreadPoolExecutor.**java:886)
    at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.**java:662)
Caused by: org.elasticsearch.index.**AlreadyExpiredException: already
expired [internal_index_hor]/[**internal_index]/[relationship_**namespace]
due to expire at [13

75991302672] and was processed at [1375991302928]
    at org.elasticsearch.index.**mapper.internal.**TTLFieldMapper.*
*innerParseCreateField(**TTLFieldMapper.java:205)
    at org.elasticsearch.index.**mapper.core.NumberFieldMapper.**
parseCreateField(**NumberFieldMapper.java:167)
    at org.elasticsearch.index.**mapper.core.**
AbstractFieldMapper.parse(**AbstractFieldMapper.java:385)
    ... 10 more
{

state: open

settings: {

index.analysis.analyzer.my_**analyzer.tokenizer: keyword

index.analysis.analyzer.my_**analyzer.filter: lowercase

index.number_of_shards: 5

index.number_of_replicas: 2

index.version.created: 900199

index.refresh_interval: 1s

}

mappings: {

internal_index: {

_ttl: {

enabled: true

default: 1000

}

properties: {

index_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

entity_version: {

include_in_all: false

store: true

type: long

}

index_version: {

include_in_all: false

store: true

type: long

}

refresh_exact: {

type: string

}

entity_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

}

}

}

aliases: [

internal_index

]

}
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
--
Benjamin DEVEZE

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Arjit_Gupta · August 24, 2013, 4:04pm

After digging more into logs I found out we are getting. Do we need to have
all the nodes to ntp synced ?

[2013-08-24 10:13:09,791][WARN ][action.index ] [search5]
Failed to perform index on replica
[internal_index_hor][1]org.elasticsearch.transport.RemoteTransportException:
[search6][inet[/172.65.149.97:9300]][index/replica]

Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to
parse [_ttl]

at

org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:396)

at

org.elasticsearch.index.mapper.internal.TTLFieldMapper.postParse(TTLFieldMapper.java:167)

at

org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:524)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:450)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:327)
at
org.elasticsearch.action.index.TransportIndexAction.shardOperationOnReplica(TransportIndexAction.java:258)

at

org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$ReplicaOperationTransportHandler.messageReceived(TransportShardReplicationOperationAction.java:254)

at

org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$ReplicaOperationTransportHandler.messageReceived(TransportShardReplicationOperationAction.java:240)
at
org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:265)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

    at java.lang.Thread.run(Thread.java:662)

Caused by: org.elasticsearch.index.AlreadyExpiredException: already expired
[internal_index_hor]/[internal_index]/[relationship_namespace] due to
expire at [1377319389247] and was processed at [1377319389781]

at

org.elasticsearch.index.mapper.internal.TTLFieldMapper.innerParseCreateField(TTLFieldMapper.java:205)
at
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:167)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:385)
... 11 more

[2013-08-24 10:13:09,801][WARN ][cluster.action.shard ]
[sp-cms-hoodoo-search5] sending failed shard for [internal_index_hor][1],
node[RV9FntFrS7O8FodB3zTAOA], [R], s[STARTED], reason [Failed to perform
[index] on replica, message
[RemoteTransportException[[search6][inet[/172.65.149.97:9300]][index/replica]];
nested: MapperParsingException[failed to parse [_ttl]]; nested:
AlreadyExpiredException[already expired
[internal_index_hor]/[internal_index]/[fk_sp_cms_listing_tag_relationship_namespace]
due to expire at [1377319389247] and was processed at [1377319389781]]; ]]

Thanks ,
Arjit

On Sat, Aug 24, 2013 at 4:36 PM, Arjit Gupta arjit292@gmail.com wrote:

I am not trying to index anything its Elastic search which is performing
some operation. I am not able to get from the stack trace.

org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[_ttl]
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:396)
at
org.elasticsearch.index.mapper.internal.TTLFieldMapper.postParse(TTLFieldMapper.java:167)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:524)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:450)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:327)
at
org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:203)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:532)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:430)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.elasticsearch.index.AlreadyExpiredException: already
expired [internal_index_hor]/[internal_index]/[relationship_namespace]
due to expire at [1377319486435] and was processed at [1377319527779]
at
org.elasticsearch.index.mapper.internal.TTLFieldMapper.innerParseCreateField(TTLFieldMapper.java:205)
at
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:167)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:385)
... 10 more

Thanks ,
Arjit

On Mon, Aug 12, 2013 at 8:07 PM, Benjamin Devèze <
benjamin.deveze@gmail.com> wrote:
It makes no sense to index an expired document that is going to be
deleted right away.

You can of course reindex the same document with an expiration date in
the future or update the ttl of an existing document.

On Mon, Aug 12, 2013 at 12:36 PM, Arjit Gupta arjit292@gmail.com wrote:
Thanks a lot.
But I cant index an expired document ?
Once a document is expired I cant index the same document ?

Thanks ,
Arjit

On Sat, Aug 10, 2013 at 4:36 PM, Benjamin Devèze <
benjamin.deveze@gmail.com> wrote:
Hi Arjit,

from this message:

Caused by: org.elasticsearch.index.**AlreadyExpiredException: already
expired [internal_index_hor]/[**internal_index]/[relationship_**namespace]
due to expire at [13

75991302672] and was processed at [1375991302928]

it seems like you are trying to index an already expired document.

This seems to be explained here by your default ttl value set to "1000"
which is in milliseconds and stands for 1s.

Hope it helps

On Saturday, August 10, 2013 4:43:04 AM UTC+2, Arjit Gupta wrote:
Hi,

We are consistently getting exception - I dont know why is it not able
to parse _ttl.

[2013-08-09 01:18:22,929][DEBUG][action.**index ]
[search5] [internal_index_hor][1], node[MwQNQpm8Q7GqPBSWaC2rRQ], [P],
s[STARTED]: Failed to execute [index

{[internal_index_hor][**internal_index][relationship_namespace],
source[{"refresh_exact":"true","refresh":"true"}]}]

org.elasticsearch.index.**mapper.MapperParsingException: failed to
parse [_ttl]
    at org.elasticsearch.index.**mapper.core.**
AbstractFieldMapper.parse(**AbstractFieldMapper.java:396)
    at org.elasticsearch.index.**mapper.internal.**
TTLFieldMapper.postParse(**TTLFieldMapper.java:167)
    at org.elasticsearch.index.**mapper.DocumentMapper.parse(**
DocumentMapper.java:524)
    at org.elasticsearch.index.**mapper.DocumentMapper.parse(**
DocumentMapper.java:450)
    at org.elasticsearch.index.shard.**service.InternalIndexShard.
**prepareIndex(**InternalIndexShard.java:327)
    at org.elasticsearch.action.**index.TransportIndexAction.**
shardOperationOnPrimary(**TransportIndexAction.java:203)
    at org.elasticsearch.action.**support.replication.**
TransportShardReplicationOperationAction$
AsyncShardOperationAction.performOnPrimary(
TransportShardReplicationOpera**tionAction.

java:532)
    at org.elasticsearch.action.**support.replication.**
TransportShardReplicationOperationAction$
AsyncShardOperationAction$1.**run(TransportShardReplicationOpera
tionAction.java:430)
    at java.util.concurrent.**ThreadPoolExecutor$Worker.**
runTask(ThreadPoolExecutor.**java:886)
    at java.util.concurrent.**ThreadPoolExecutor$Worker.run(**
ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.**java:662)
Caused by: org.elasticsearch.index.**AlreadyExpiredException: already
expired [internal_index_hor]/[**internal_index]/[relationship_**namespace]
due to expire at [13

75991302672] and was processed at [1375991302928]
    at org.elasticsearch.index.**mapper.internal.**TTLFieldMapper.
**innerParseCreateField(**TTLFieldMapper.java:205)
    at org.elasticsearch.index.**mapper.core.NumberFieldMapper.**
parseCreateField(**NumberFieldMapper.java:167)
    at org.elasticsearch.index.**mapper.core.**
AbstractFieldMapper.parse(**AbstractFieldMapper.java:385)
    ... 10 more
{

state: open

settings: {

index.analysis.analyzer.my_**analyzer.tokenizer: keyword

index.analysis.analyzer.my_**analyzer.filter: lowercase

index.number_of_shards: 5

index.number_of_replicas: 2

index.version.created: 900199

index.refresh_interval: 1s

}

mappings: {

internal_index: {

_ttl: {

enabled: true

default: 1000

}

properties: {

index_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

entity_version: {

include_in_all: false

store: true

type: long

}

index_version: {

include_in_all: false

store: true

type: long

}

refresh_exact: {

type: string

}

entity_version_qual: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

refresh: {

include_in_all: false

store: true

analyzer: my_analyzer

type: string

}

}

}

}

aliases: [

internal_index

]

}
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
--
Benjamin DEVEZE

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · August 26, 2013, 8:11am

If you use ttl, ES assumes that all nodes operate with synchronized clocks.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

brian_yoder · August 26, 2013, 2:10pm

Jörg,

Is there an acceptable limit to the clock skew?

Even with very expensive temperature-stabilized time bases in each data
center (another application, not DB related), we still see some clock skew.
Not much, but it's not perfect either.

Brian

On Monday, August 26, 2013 4:11:37 AM UTC-4, Jörg Prante wrote:

If you use ttl, ES assumes that all nodes operate with synchronized clocks.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · August 26, 2013, 6:01pm

It depends on indices.ttl.interval which is by default set to 1m. Since the
TTL expiration collector is fired according to this interval, I think a
clock skew of ~30s is acceptable. Or you can modify indices.ttl.interval to
a larger interval, but this may give longer lags.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Arjit_Gupta · August 27, 2013, 5:31am

If I want the expired documents to be deleted by default should I set
indices.ttl.interval
= 0 ?

Thanks ,
Arjit

On Mon, Aug 26, 2013 at 11:31 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

It depends on indices.ttl.interval which is by default set to 1m. Since
the TTL expiration collector is fired according to this interval, I think a
clock skew of ~30s is acceptable. Or you can modify indices.ttl.interval to
a larger interval, but this may give longer lags.

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Arjit_Gupta · August 27, 2013, 10:07am

I tried changing the setting
curl -XPUT 'http://localhost:9200/<index_name>/_settings' -d
'{"indices.ttl.interval" : 0}'
first I got exception
ElasticSearchIllegalArgumentException[Can't update non dynamic
settings[[index.indices.ttl.interval]] for open indices
Then I closed the index and tried running the command again. But its still
not honoring it.

Any help will be appreciated.

Thanks ,
Arjit

On Tue, Aug 27, 2013 at 11:01 AM, Arjit Gupta arjit292@gmail.com wrote:

If I want the expired documents to be deleted by default should I set indices.ttl.interval
= 0 ?

Thanks ,
Arjit

On Mon, Aug 26, 2013 at 11:31 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

It depends on indices.ttl.interval which is by default set to 1m. Since
the TTL expiration collector is fired according to this interval, I think a
clock skew of ~30s is acceptable. Or you can modify indices.ttl.interval to
a larger interval, but this may give longer lags.

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Arjit_Gupta · August 27, 2013, 10:08am

Thanks ,
Arjit

On Tue, Aug 27, 2013 at 3:37 PM, Arjit Gupta arjit292@gmail.com wrote:

I tried changing the setting
curl -XPUT 'http://localhost:9200/<index_name>/_settings' -d
'{"indices.ttl.interval" : 0}'
first I got exception
ElasticSearchIllegalArgumentException[Can't update non dynamic
settings[[index.indices.ttl.interval]] for open indices
Then I closed the index and tried running the command again. But its still
not honoring it.

Any help will be appreciated.

Thanks ,
Arjit

On Tue, Aug 27, 2013 at 11:01 AM, Arjit Gupta arjit292@gmail.com wrote:

If I want the expired documents to be deleted by default should I set indices.ttl.interval
= 0 ?

Thanks ,
Arjit

On Mon, Aug 26, 2013 at 11:31 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

It depends on indices.ttl.interval which is by default set to 1m. Since
the TTL expiration collector is fired according to this interval, I think a
clock skew of ~30s is acceptable. Or you can modify indices.ttl.interval to
a larger interval, but this may give longer lags.

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/0So4AdJMT84/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.