Auto index creation with bulk API


(Marcus) #1

Hi,

I have these settings applied (ES version 1.7.5):

action.auto_create_index: false
index.mapper.dynamic: false

because I want to have full control over what indices/types are created.

But, still I get these in the logs:

[my_index] creating index, cause [auto(bulk api)], templates [], shards [6]/[1], mappings [mapping1]

which is problematic because now mapping1 is auto created based on the document indexed which is what I want to avoid.

Is there any more setting I can apply to get auto creation from bulk api disabled?


(Jymit Singh Khondhu) #2

Hi,
Please note that 1.7.x is now EOL and no particular code patches or bug fixes will be forthcoming. https://www.elastic.co/support/eol

Can you share how the above, mentioned modification was implemented? Please find this (v5.1.x) reference on if this is to be implemented on a per index or all indices basis.


(Marcus) #3

Can you share how the above, mentioned modification was implemented?

I don't really understand what you're after here, but if you mean how this problem is encountered is like the following:

  1. A bulk request is issued and started
  2. In the middle of the bulk request, a delete index request is made
  3. Next item in the bulk causes ES is auto creating the index with auto mapping <--error

(Jymit Singh Khondhu) #4

Where were these applied? In the elasticsearch.yml or other?


(Marcus) #5

Yes in elasticsearch.yml , on all nodes.


(Jymit Singh Khondhu) #6

Using POST /indexname is a manual request, not an auto-create.

The only way to prevent indexes being created manually is to make the whole cluster read only: https://www.elastic.co/guide/en/elasticsearch/reference/1.7/cluster-update-settings.html#_metadata


(Marcus) #7

I'm not issuing a POST /indexname request, i'm doing a bulk index request to an existing index and while doing it removing the index in question.


(Marcus) #8

My solution was to simply close the index first and wait 10 seconds and then delete index. This to fail any existing bulk requests and get around problem with autocreate.


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.