Storing the diff of two fields in a new field

I am trying to store some json data generated after running some api call, into elasticsearch, and my documents look like this.

{ "host" : "hostname", "time" : "timestamp", "command" : "command being run", "data_before" : "some json set of fields and thier values before running command", "data_after" : "same json set of fields with updated values after running command" }

My requirement is to loop through all the fields like data_before.* and compare them with their corresponding fields in data_after.* and store / ingest only those fields which are modified.

My use case is to see, which fields of my dataset are actually modified at a given point of time (one or two out of some 100+ fields will only be modified usually), storing/ingesting the entire dataset as data_before and data_after does not look feasible.

Any suggestions on how I can get this done at ingestNode level will be very helpful.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.