Kibana visualization for creating custom bar graph

Elastic Stack Kibana
1

Hi all,

I have a log file containing a series of jobs along with start and end time for each job. I have parsed the log file using logstash and converted the start and end times into timestamp fields. In order to calculate the time for each job, I created a scripted field named job_time defined as:
doc['end_time'].value - doc['start_time'].value

I need to create a bar graph with job_name and job_time as the axes. The vertical bar chart in Kibana's visualize tab only provides aggregate operations like count, sum, average as the Y-axis. I am unable to set job_time as the Y-axis. Could you please help me out?

2

Hey @GautamP, is the job_type not showing up in the y-axis Field dropdown after you select an aggregation?

3

Hey @Brandon_Kobel

The fields in the logstash index along with their data types are as follows:

job_start_time     date
job_end_time       date
job_name           string

job_time is a scripted field: (doc['job_end_time'].value - doc['job_start_time'].value) / 1000.0

job_name doesn't show up for any aggregation.

These are the only fields that are availabe:

361×729 17.3 KB

I would like to have job_name on Y-axis and its corresponding job_time on x-axis or vice versa. I don't think I need any aggregation for this but the visualize tab insists on selecting an aggregation

4

I think the closest that you're going to get is having the job_name on the x-axis and the job_time on the y-axis. However, you'll have to choose an aggregation to use for the y-axis. It might be beneficial for us to take a step back, as I'm not sure if this is going to give you exactly what you're looking for.

So, you have jobs that are running that have a job_name and a job_time, are you looking to get an entire history of the jobs running and their time, or are you looking for an average/sum of the job_time?

5

@Brandon_Kobel
Each job has a set of sub jobs, each of which may again have a set of sub jobs. I'm looking to get the entire history of jobs running such that I can drill down or roll up to different jobs on different levels to get the the corresponding job times on the various levels.

Here is my discussion with @Mark_Harwood on the same.

My final intention is to obtain something like JDiskReport but for figuring out where time goes, not disk space.

6

@GautamP thank you for that explanation.

For your specific use-case, I'd recommend using an x-axis terms aggregation on job_name and a y-axis sum aggregation on job_time, this will give you a chart with the total time spent grouped by job.

7

@Brandon_Kobel
Thanks for the suggestion. It's working now.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.