Deleting old Docs from particular index using curator plugin


(kish) #1

Hello -

I have installed curator plugin for deleting docs of particular index. So

  1. Can curator plugin can be used to delete a Doc's of a index/indices OR it is only used to delete the entire indices. If possible can you share any sample.

  2. To delete a indices also is it mandatory to have alias created for that indices.


(Christian Dahlqvist) #2

Curator deletes entire indices only.

No, aliases are not required nor expected.


(kish) #3

Then is there any way to delete the Docs alone for particular indices? Because my indices will be loaded every 1 minute and down the line it will a huge data.
i only need to keep the Docs say for past 10days, older i can delete. Could you suggest any way forward using curator plugin?

i tried using delete_by_query but vain.

termQuery := NewRangeQuery("date").Lte("now-2d/d")
		put1, err := elasticSearchClient.DeleteByQuery().
			Index(index).
			Type(index).
			Query(termQuery).
			Do(elasticSearchContext)

And mapping is as follow

const cscfMappaing = `
{
	"settings": {
	},
	"mapping": {
		"cscfcounter":{
			"properties":{
				"date": {
          			"type":   "date"
          		},
				"NodeId":{
					"type":"keyword"
				},
				"NodeType":{
					"type":"keyword"
				},
				"State":{
					"type":"keyword"
				},
				"HaGroupId":{
					"type":"keyword"
				},
				"LbGroupId":{
					"type":"keyword"
				},
				"CpuAverageLoad":{
					"type":"keyword"
				},
				"MemUsed":{
					"type":"keyword"
				},
				"MemFree":{
					"type":"keyword"
				},
				"Static_limit":{
					"type":"keyword"
				}
			}
		}}}`

(Christian Dahlqvist) #4

If you use time-based indices, e.g. an index per day or using the rollover API, you can simply drop complete indices using curator. This is much more efficient than deleting select documents from an index.

If you need to delete documents from within an index, delete-by-query is the way to go.


(kish) #5

Could you kindly explain it(i am new to ES). You mean to say like, we need to define a indices which can load only certain amount of data(size) post which old data will be deleted and new new data will stored?


(Christian Dahlqvist) #6

This blog post describes how the rollover index API works and the benefits of it. This blog post may also be useful.


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.