Documents not deleted when recreating same index

Elastic Stack Elasticsearch
1

Hello there,

I have a simple python script to create indices. This script checks if the index already exists, if so it first deletes the index and then recreates it. The problem I face is that, once I recreate this particular index, the document count does not go to 0.

I run the stat api on the index, as soon as I recreate it and this is what I get.

{
   "_shards": {
      "total": 10,
      "successful": 5,
      "failed": 0
   },
   "_all": {
      "primaries": {
         "docs": {
            "count": 22,
            "deleted": 168
         },
         "store": {
            "size_in_bytes": 54195,
            "throttle_time_in_millis": 0
         },
         "indexing": {
            "index_total": 1140,
            "index_time_in_millis": 107,
            "index_current": 0,
            "index_failed": 0,
            "delete_total": 0,
            "delete_time_in_millis": 0,
            "delete_current": 0,
            "noop_update_total": 0,
            "is_throttled": false,
            "throttle_time_in_millis": 0
         },
         "get": {

Does anyone an idea what's going on here?

2

How do you know it deletes and creates it, and there's not something else creating it?

3

My script returns the responses.

vinod@web4 ~/es> ./init.py
Enter tda_languagepair id: 1
Index already exists
deleting en-us_zh-hk index...
 response: {u'acknowledged': True}
creating en-us_zh-hk index...
 response: {u'acknowledged': True, u'shards_acknowledged': True}
4

Do you have any updates on this for me? I have added my mapping template below, incase you'd like to take a look.

{
	"template": "logstash-*",
	"settings" : {
		"number_of_shards" : 5,
	    "number_of_replicas" : 1,
	    "index.refresh_interval": "60s",
	    "index" : {}
	},
	"mappings": {
		"segment": {
	    	"_parent": {
	        	"type": "tm"
	        },
	        "_routing": {
	          "required": "true"
	        },
	        "properties": {
	        	"@timestamp": {
	        		"type": "date",
	            	"format": "strict_date_optional_time||epoch_millis"
	          	},
		        "@version": {
		        	"type": "string"
		        },
		        "id": {
		        	"type": "long"
		        },
		        "lastmodified": {
		        	"type": "date",
		            "format": "strict_date_optional_time||epoch_millis"
		        },
		        "source": {
		        	"type": "string",
		            "store": "true"
		        },
		       	"sourcewordcount": {
		        	"type": "long"
		        },
		        "target": {
		        	"type": "string",
		            "store": "true"
		        },
		        "targetwordcount": {
		            "type": "long"
		        },
		        "upload_id": {
		            "type": "long",
		            "store": "false"
		        },
		        "sourceDigest": {
		            "type": "string"
		        },
		        "targetDigest": {
		            "type": "string"
		        }
			}
		},
		"tm": {
	    	"properties": {
	        	"@timestamp": {
	            	"type": "date",
	            	"format": "strict_date_optional_time||epoch_millis"
	          	},
	          	"@version": {
	            	"type": "string"
	          	},
	          	"contenttype": {
	            	"type": "string"
	          	},
	          	"dataowner": {
	            	"type": "string"
	          	},
	          	"dataSource": {
	            	"type": "string"
	          	},
	          	"id": {
	            	"type": "long"
	          	},
	          	"industry": {
	            	"type": "string"
	          	},
	          	"sourcelanguage": {
	            	"type": "string"
	          	},
	          	"tags": {
	            	"type": "string"
	          	},
	          	"targettanguage": {
	            	"type": "string"
	          	}
	        }
		}
	}
}
5

And what's that python doing, can you post it?

6

Sure, here you go,

def delete_index(es, indexName):
    #delete if already exists
    if es.indices.exists(indexName):
        print("deleting %s index..." % indexName)
        res = es.indices.delete(index = indexName)
        print(" response: %s" % res)

def create_index(es, indexName):
    #create request body
    f  = open("http_log_logstash.json", "r")
    request_body = f.read()

    print("creating %s index..." % indexName)
    res = es.indices.create(index = indexName, body = request_body, ignore = 400)
    print(" response: %s " % res)
7

And what's in http_log_logstash.json.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.