Accessing Data in Pipelines

Hello,

I am having trouble accessing data that gets ingested into my data pipeline, that should transform data within each document that gets indexed into ES.

I understand that this data can be accessed via dot notation, and I have been successful in using the processors to change ONLY the fields associated with the explicit mapping (the outlined, single mapping) for all documents to be indexed into ES. For example, say that in each document that gets indexed, I have a field (an array) immediately under _source that says "stringArray" followed by a fields such as "name" "value" and "position". Say that for the three documents that get indexed, the positions are all different, like 1, 2, and 3. I want to set all these positions to say, 5.

Documentation says source document fields can be accessed directly or by using "_source" before the fields. That doesn't work for the source documents (docs containing the actual data). I have only been successful in manipulating fields only of the single mapping outlining the documents, by using something like "mappings._doc.properties.stringArray.name" which is clearly not what I want because that changes the explicit mapping and not fields within the document. I have a feeling the path I am specifying may need to be tweaked or incorrect. Ideas?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.