Deleting old Docs from particular index using curator plugin

Elastic Stack Elasticsearch
1

Hello -

I have installed curator plugin for deleting docs of particular index. So

  1. Can curator plugin can be used to delete a Doc's of a index/indices OR it is only used to delete the entire indices. If possible can you share any sample.

  2. To delete a indices also is it mandatory to have alias created for that indices.

2

Curator deletes entire indices only.

No, aliases are not required nor expected.

3

Then is there any way to delete the Docs alone for particular indices? Because my indices will be loaded every 1 minute and down the line it will a huge data.
i only need to keep the Docs say for past 10days, older i can delete. Could you suggest any way forward using curator plugin?

i tried using delete_by_query but vain.

termQuery := NewRangeQuery("date").Lte("now-2d/d")
		put1, err := elasticSearchClient.DeleteByQuery().
			Index(index).
			Type(index).
			Query(termQuery).
			Do(elasticSearchContext)

And mapping is as follow

const cscfMappaing = `
{
	"settings": {
	},
	"mapping": {
		"cscfcounter":{
			"properties":{
				"date": {
          			"type":   "date"
          		},
				"NodeId":{
					"type":"keyword"
				},
				"NodeType":{
					"type":"keyword"
				},
				"State":{
					"type":"keyword"
				},
				"HaGroupId":{
					"type":"keyword"
				},
				"LbGroupId":{
					"type":"keyword"
				},
				"CpuAverageLoad":{
					"type":"keyword"
				},
				"MemUsed":{
					"type":"keyword"
				},
				"MemFree":{
					"type":"keyword"
				},
				"Static_limit":{
					"type":"keyword"
				}
			}
		}}}`
4

If you use time-based indices, e.g. an index per day or using the rollover API, you can simply drop complete indices using curator. This is much more efficient than deleting select documents from an index.

If you need to delete documents from within an index, delete-by-query is the way to go.

5

Could you kindly explain it(i am new to ES). You mean to say like, we need to define a indices which can load only certain amount of data(size) post which old data will be deleted and new new data will stored?

6

This blog post describes how the rollover index API works and the benefits of it. This blog post may also be useful.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.