How to convert a json format string to json object using painless script kibana

[Aniket_Salvi]

This in my input:-

"bitstreamData": {
"bitstream": "{"Virtual Account Number Verification OUT":[{"Client Code":"LEND","Virtual Account Number":"","Transaction Amount":200000,"UTR number":"*******","Status":"Reject","Sender Name":"Test Remitter 9","Date":"10\/04\/2019","Remitter Account Number":"","Mode":"I","Reject Reason":"Invalid account number"}]}"
}

I am not able to convert above input to json object using painless script. Can anyone have solution to my problem.

Thank you.

[flash1293 (Joe Reuter)]

Hello @Aniket_Salvi,

no, there are no JSON parsing classes in painless, so this isn't possible.

But even if there were, it's not a good idea to use painless for this task. The way Kibana handles scripted fields is to calculate the value on the fly for each document you are querying for each request you are making. As JSON parsing is a relatively complex operation, this would put a lot of load on your cluster and make queries super slow, especially for larger data sets.

However you can do this kind of transformation during ingest, which means you only have to do the parsing once (when you are indexing the document), and all requests based on the parsed data will be very fast. This related question shows a way to do this with Logstash.

[Aniket_Salvi]

Hi Flash,
I am using elasticsearch api to ingest data in elasticsearch. How I can parse data before ingesting data to elasticsearch.

Thank you!

[flash1293 (Joe Reuter)]

Which client are you using? The simplest way would be to parse the JSON as part of your code before sending the request to Elasticsearch.

[Aniket_Salvi]

I am ingesting "IIB event logs" to elasticsearch using elasticsearch REST api.

[flash1293 (Joe Reuter)]

I meant in which programming language you are calling the elasticsearch API.

E.g. if you are using node.js you are probably doing something like this at one point (using an imaginary api, just for illustrative purposes):

httpClient.post('my_index/_doc', { bitstreamData: myData });

This is the point where you can do your preprocessing:

myData.bitstream = JSON.parse(myData.bitstream);
httpClient.post('my_index/_doc', { bitstreamData: myData });