Python Elasticsearch Bulk Conditional Upsert

Elastic Stack Elasticsearch
1

Using elasticsearch-py python helpers library for bulk conditional scripted update:

from elasticsearch import Elasticsearch
from elasticsearch import helpers
es = Elasticsearch(host=YOUR_HOST)
actions = []

document = {
    "field1": "test_script_value1",
    "date": "2017-01-13T16:24:00Z"
}
action = {
    "_op_type": "update",
    "_index": "YOUR_INDEX_NAME",
    "_type": "YOUR_DOCUMENT_TYPE",
    "_id": YOUR_DOCUMENT_ID,
    "_source": {
        "script": {
            "inline": "if ((ctx._source.doc) && (ctx._source.doc.date > doc.date)) { ctx.op='noop' } else { ctx._source.doc=doc }",
            "params": { "doc": document }
        },
        "upsert": { "doc": document }
    }
}
actions.append(action)
helpers.bulk(es, actions)

Also this requires enabling scripting (be careful, this is not safe if your users can access the elastic instance) in elasticsearch.yml configuration file:

script.engine.groovy.inline.update: on

Had a hard time finding the right documentation for this, so sharing in case it helps someone else (or me in the future).

Keywords: python, elastic, bulk, helpers, update, upsert, doc_as_upsert, script, conditional, date

4 Likes
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.