Using data from a json document field in a Visualization

[relentless (Aaron Warner)]

Hi Guys

I have a log which produces a field called relatedData. This field contains a small json document.
e.g
"relatedData": "{"method":"Put","StartDateTime":"2019-06-20T07:57:08.9337817+12:00","EndDateTime":"2019-06-20T07:57:08.9494062+12:00","TimeTakenInMilliSeconds":15.624500000000001}",

I'm wanting to extract the value for TimeTakenInMelliSeconds and use that number in a visualization. But not sure how to go about it.
Any ideas on how to approach this with kibana? I'm thinking it could be a scripted field but didn't see any examples of other people doing it.

[lukas (Lukas Olson)]

Does relatedData contain the same fields in each document, or is it dynamic?

[relentless (Aaron Warner)]

It contains different data types (sometimes json objects, sometimes strings), we have api's dumping logs and any extra information (like an exception message or a user friendly message etc) is placed in this. But we could create a new field that specifically logs this particular data object if that would make it much easier.

[lukas (Lukas Olson)]

Creating a separate field specifically for the timestamp would definitely make this task easier. If that's not possible, however, you could probably accomplish this through a scripted field that gets the value for relatedData, parses the JSON and then extracts the TimeTakenInMilliseconds.

[relentless (Aaron Warner)]

We do have the default @timestamp field in each document and bunch of other fields. Sorry I just provided the particular field I was interested in trying to parse. Sounds like I should be using a scripted field with a regex to pull this information out. Is that about right?

[relentless (Aaron Warner)]

Actually now that I'm looking through the painless docs they say that regex is disabled by default for performance reasons.