Delete by time range with ES 1.1


(Bastien Chong) #1

I'm trying to delete document from an index based on the timestamp.
I'm using ES v1.1, so I have to use the query wrapper.

This return the wanted result :

curl -*XGET *'http://localhost:9200/test/_search?pretty=1' -d
'{"query":{"range":{"@timestamp":{"from":"2014-07-10T00:00:00","to":"2014-07-13T21:00:00"}}}}'

but this doesn't always work, command is hanging :

curl -XDELETE 'http://localhost:9200/test/_query?pretty=1' -d
'{"query":{"range":{"@timestamp":{"from":"2014-07-10T00:00:00","to":"2014-07-13T21:00:00"}}}}'

The cluster's health is good, I worked a few hours ago, and now it's not. I
tried to restart elasticsearch without luck.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/be990246-efb3-4ed2-bec6-101fb03bb9ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Bastien Chong) #2

Not sure what's happening, I restarted both elasticsearch instance for my 2
nodes-cluster, and now it's working. But I'm sure the cluster was in good
state, I could PUT and GET.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/97485ca1-98ba-45bf-bc8c-d956a01e2241%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Patrick Proniewski) #3

Maybe you just discovered by yourself what Aaron described in a previous message (Message-Id: 1405173409302.ce95d5ef@Nodemailer on logstash mailing list):

[deleting a type of documents from an index] is a horrible idea in the same way that running DELETE FROM TABLE WHERE TYPE='cisco' AND DATE<... in a SQL database is a bad idea. You get thousands, if not millions of individual deletes to manage, and they are horrible in terms of disk I/O performance to the system and the cluster. Worse still, in elasticsearch, the deletes aren't immediate. They are only flagged for deletion. The actual delete takes place during the next segment merge, which is up to 30 minutes later. It could render you with horrible I/O during the seek as well as the subsequent delete.

On 14 juil. 2014, at 23:48, Bastien Chong wrote:

Not sure what's happening, I restarted both elasticsearch instance for my 2 nodes-cluster, and now it's working. But I'm sure the cluster was in good state, I could PUT and GET.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/97485ca1-98ba-45bf-bc8c-d956a01e2241%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/D78CC74D-6B54-412A-AFF0-2E54E593362C%40patpro.net.
For more options, visit https://groups.google.com/d/optout.


(system) #4