How do i use external versions when using pipelines?

Elastic Stack Elasticsearch
1

Im currently working on pushing logs to elasticsearch using filebeat. In order to index these docs to the preferred location, i use an ingester node.

{"name":"mydoc","myDocId":"myId1","myDocType":"someType","myIndex":"someIndex","myVersion":1}

With this sample log, i use a series of processors to structure it. In one of the pipeline processors i use a script processor and do:

...
ctx._index = ctx.myIndex; ctx._type=ctx.myType;
...

This works but uses the internal version of elasticsearch. Since the doc contains a version of itself (myVersion field), i would like to use it so that i can make sure that an older version of what filebeat picks up does not override the doc if its an older one.

ive tried setting the ctx._version and addex ctx._version_type=external but i get an error when i do so (dynamic mapping exception).

How do i use my own versioning sequence with the pipelines?

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.