Kibana version:
7.5.0
Elasticsearch version:
7.5.0
Browser version:
Brave latest version
Howdy, y'all!
Description:
I implemented an ETL pipeline service and a forecasting service using Clojure base on OpenJDK8. I started to work with a profiling library: Tufte by Taoensso. It collects the number of calls to a function and returns running time statistics of the profiled form. Having this data I plan to create a Canvas displaying the information as a table and perhaps graphs. However, I am positive using text as datatype would not make my life easier if I would like to make graphs. Anyway, the following is an example of how the library prints to stdout with System.out.println():
pId nCalls Min 50% ≤ 90% ≤ 95% ≤ 99% ≤ Max Mean MAD Clock Total
:get-s3-event 1 161.00ns 161.00ns 161.00ns 161.00ns 161.00ns 161.00ns 161.00ns ±0% 161.00ns 0%
Accounted 161.00ns 0%
Clock 49.35μs 100%
The data looks like the following:
{:min 161,
:event-name :get-s3-event,
:mean 161.0,
:p75 161,
:mad-sum 0.0,
:p99 161, :n 1,
:time "2020-02-19T15:53:35Z",
:p25 161,
:p90 161,
:max 161,
:mad 0.0,
:p50 161,
:sum 161,
:p95 161}
To begin with, I am ignoring the Clock and Account values because I already have Metricbeat and APM doing that for me. However, these numbers aren't detailed enough because it doesn't tell the time measurement. I am like: "161 what?!". Tufte does handle the numbers in its own logic, so if the value is 9e-6, then it is a nanosecond, if the value is 9e-9, then it is ms, etc. Therefore, I pass this hash map over a function that will apply a regex on the string representation (stdout output value) searching for the time measure (m, s, ms, ns, us). Then, finding the matching value, the function will update the values to a string, looking like the following example:
{:min "161ns", :event-name :get-s3-event, :mean "161.0ns", :p75 "161ns", :mad-sum 0.0, :p99 "161ns", :n 1, :time "2020-02-19T15:53:35Z", :p25 "161ns", :p90 "161ns", :max "161ns", :mad 0.0, :p50 "161ns", :sum "161ns", :p95 "161ns"}
I am indexing the modified hash map with string values to Elasticsearch using the mappings data type of text. However, maybe I could use a better data type that could help me represent the time duration of nCalls and create a Canvas with graphs using duration. What do y'all think?
I am using the data type documentation as reference. I haven't seen anything that could help me or maybe I missed something