Help Creating a Visualization

[CommanderTso (Scott Macmillan)]

Hi folks,

I'm working my way through learning Kibana, but was hoping someone could give me a leg-up on a visualization I've been asked to create in the short term.

I have an index of filebeat documents harvested from /var/log/messages on a Lustre storage system. For the records in there that I want to visualize, each will have a value for error.type and error.additional_occurances. The second field there represents the number of additional identical messages that Lustre squashed down.

I'd like to create a visualization that will show, overtime, the occurrances of each error.type and augment its magnitude on the visualization by the number of error.additional_occurances. For example, if I had, say an error of type "bulk READ error" with an additional_occurances of 400, the magnitude for that error at that point in time would be 400 (see note at bottom).

I'm open to bucketing this stuff however is helpful, and open to building it with Timeline, Visual Builder - whatever works. Any pointers you folks might have for me would be very welcome.

Thanks!

Note: To be exact, it would be 401 (the record itself plus the additional_occurances), but I don't need to get hung up on that.

[CommanderTso (Scott Macmillan)]

Playing with it more, I tried:

.es(index=filebeat-storage*,split=error.type:10, metric=sum:error.additional_occurances)

My concern here is that I'm pretty sure I'm dropping events that don't have a value for additional_occurances, which won't be present if my grok parser didn't find any corresponding value in the log entry.

[flash1293 (Joe Reuter)]

Hi @CommanderTso,

that looks pretty good already. You can add a static value of one (the implicit original error that didn't get squashed) to each value like this:

.es(index=filebeat-storage*,split=error.type:10, metric=sum:error.additional_occurances).add(.static(1))

This should make sure you are not dropping empty buckets

[CommanderTso (Scott Macmillan)]

Awesome - thanks! I'm also working through the Kibana on-demand course, and the introduction of aggregations fills a big hole in my understanding of how the Visualizations can work.