Bulk insert with python

Elastic Stack Elasticsearch
1

Hi everyone,

just as a foreword: I'm new to elasticsearch (and this forum :slightly_smiling_face:) and still figuring out a lot.

So, I successfully tried to use bulk insertion from a file with the following content:

{ "index" : {"_type" : "_doc", "_id": "828ef1361dad4f289de8983e90f7ea96"} }
{"last_updated": "2020-02-03T14:15:01Z", "waiver": {"has_waiver": false}, "id": "828ef1361dad4f289de8983e90f7ea96"}

Then I tried to do the same with the python module and I wanted to use the existing "id" field as the "_id" for the elasticsearch document. So I did the following in python:

>>> from elasticsearch import Elasticsearch
>>> from elasticsearch.helpers import bulk
>>> client = Elasticsearch()
>>> bulk(client, ['{"_id": "828ef1361dad4f289de8983e90f7ea96","last_updated": "2020-02-03T14:15:01Z", "waiver": {"has_waiver": false}}'], index="test")

However, I got the following error:

elasticsearch.helpers.errors.BulkIndexError: ('1 document(s) failed to index.', [{'index': {'_index': 'test', '_type': '_doc', '_id': 'CTg9xngBlOHxyFjlu4JA', 'status': 400, 'error': {'type': 'mapper_parsing_exception', 'reason': "failed to parse field [_id] of type [_id] in document with id 'CTg9xngBlOHxyFjlu4JA'. Preview of field's value: '828ef1361dad4f289de8983e90f7ea96'", 'caused_by': {'type': 'mapper_parsing_exception', 'reason': 'Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters.'}}, 'data': '{"_id": "828ef1361dad4f289de8983e90f7ea96","last_updated": "2020-02-03T14:15:01Z", "waiver": {"has_waiver": false}}'}}])

Does anyone know how to get around the issue and use the ID field?

2

Figured it out. It should be a dict and not a string.

2 Likes
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.