Display Percentage Work Completed

[chivas]

Hi,
Below is my data from log file

Task,1.1-29
Task,1.1-30
Task,1.1-31
TaskStarted,1.1-29
TaskMidway,1.1-29
TaskCompleted,1.1-29

The line starting with Task,1.1-29 is the definition of a task 1.1 and subtask 29. The last 3 lines (TaskStarted, TaskMidway, and TaskCompleted) mention the various stages of the subtask 1.1-29 upto completion.
Can I display this in Kibana showing that 1 subtask (1.1-29) has been completed, and the remaining 2 subtasks are still pending. So an overall completion of task 1.1 at 33.33%, and incomplete work at 66.66%?
I tried with a pie chart, but was not successful. Any pointers how I can get this done?

Thanks in advance!

[lukeelmers]

Hi @chivas,

I have a couple questions about this, just to make sure I understand: is the entire task (with subtask) a single field? (i.e. 1.1.-29), or is it split into two (i.e. 1.1 and 29)?

Also, are you only wanting to see progress at the top level? i.e.
1.1 > 33.33%
1.x > 0%

Or are you also wanting to see the breakdown of subtasks within each larger task? i.e.
1.1 > 33.33%
1.1-29 > 100%
1.1.30 > 0%

Thanks,

Luke

[chivas]

Hi @lukeelmers ,

@lukeelmers

I have a couple questions about this, just to make sure I understand: is the entire task (with subtask) a single field? (i.e. 1.1.-29 ), or is it split into two (i.e. 1.1 and 29 )?

It is a single field, i.e 1.1-29. However I have control over it and can be split into two (i.e. 1.1 and 29) if required.

@lukeelmers

Also, are you only wanting to see progress at the top level? i.e.
1.1 > 33.33%
1.x > 0%

Right now, only top level progress.

@lukeelmers

Or are you also wanting to see the breakdown of subtasks within each larger task? i.e.
1.1 > 33.33%
1.1-29 > 100%
1.1.30 > 0%

Did not think of this, but since you mentioned, it would be good to have sometime in the future, so any prep work required to achive this (from a data perpective)?

Thanks!

[lukeelmers]

Thanks for clarifying!

I've thought about this from a few different angles, and I'm not sure there's a way to do exactly what you're looking for without bucket script aggregations, which unfortunately are not yet supported in Kibana.

Bucket script aggregations let you define a custom metric by using a script to perform per-bucket computations. So in your case, if you split the task and subtask IDs into separate fields, you could probably do something roughly along the lines of: aggregate on the task ID, take the results of each bucket, aggregate on the subtasks within each bucket, and use a bucket script to calculate # TaskCompleted / TotalSubtasks as its own metric, which would give you the overall completion percentage for that task.

I wish I could provide more helpful advice here on your particular use case, but if you're interested, you can follow along with the relevant Github issue to be notified of any updates related to the development of this feature.

[chivas]

Thanks @lukeelmers for the advice regarding bucket script aggregations.

So I was thinking, if I load my individual task data lines into elasticsearch (whenever a new task is created) as follows:

Task,1.1-29,TaskStarted:N,TaskMidway:N,TaskCompleted:N
Task,1.1-30,TaskStarted:N,TaskMidway:N,TaskCompleted:N
Task,1.1-31,TaskStarted:N,TaskMidway:N,TaskCompleted:N

Now, whenever any update about the task comes in, (e.g. TaskStarted,1.1-29), then update the flag for TaskStarted field for Task,1.1-29 from N to Y in elasticsearch document from logstash output block based on the document id (say, 1.1-29). So the document will change to

Task,1.1-29,TaskStarted:Y,TaskMidway:N,TaskCompleted:N

I think it will then be easy to display this data as percentage via a pie chart.
Would this be a prudent approach, because I read someplace to avoid updating elasticsearch documents from logstash?

Thanks!