I have a json as input and am filtering the data later creating index in output block to push it into elastic
Here i want to split the data into two set and want them to send to two respective indices. like below example.
created query_details event object by combination of few fields from json input.
i want to send this quer_details columns alone to first index rest of the columns which is part of input json will be send to second index.
All my transaction has query_details so i can't use If else condition in my output field.
One way to accomplish this would be to utilize Ingest Pipelines along with two separate elasticsearch outputs to send the same document to two different indices.
You can use the clone filter to clone your event and a conditional to only apply the ruby filter to the cloned event and use the same conditional in the output.
You just need to check if you have pipeline.ecs_compatibility enabled or not as this will change the behavior of the clone filter as explained in the documentation.
I don't think so, if the conditional is correct it will index the cloned documents, with type equals to details on one index, and the other documents on the another index.
But since the other solution worked for you, there is no need to troubleshoot this further.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.