Auto index creation with bulk API

Elastic Stack Elasticsearch
1

Hi,

I have these settings applied (ES version 1.7.5):

action.auto_create_index: false
index.mapper.dynamic: false

because I want to have full control over what indices/types are created.

But, still I get these in the logs:

[my_index] creating index, cause [auto(bulk api)], templates [], shards [6]/[1], mappings [mapping1]

which is problematic because now mapping1 is auto created based on the document indexed which is what I want to avoid.

Is there any more setting I can apply to get auto creation from bulk api disabled?

2

Hi,
Please note that 1.7.x is now EOL and no particular code patches or bug fixes will be forthcoming. https://www.elastic.co/support/eol

Can you share how the above, mentioned modification was implemented? Please find this (v5.1.x) reference on if this is to be implemented on a per index or all indices basis.

3

Can you share how the above, mentioned modification was implemented?

I don't really understand what you're after here, but if you mean how this problem is encountered is like the following:

  1. A bulk request is issued and started
  2. In the middle of the bulk request, a delete index request is made
  3. Next item in the bulk causes ES is auto creating the index with auto mapping <--error
4

Where were these applied? In the elasticsearch.yml or other?

5

Yes in elasticsearch.yml , on all nodes.

6

Using POST /indexname is a manual request, not an auto-create.

The only way to prevent indexes being created manually is to make the whole cluster read only: https://www.elastic.co/guide/en/elasticsearch/reference/1.7/cluster-update-settings.html#_metadata

7

I'm not issuing a POST /indexname request, i'm doing a bulk index request to an existing index and while doing it removing the index in question.

8

My solution was to simply close the index first and wait 10 seconds and then delete index. This to fail any existing bulk requests and get around problem with autocreate.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.