Display number of messages in a queue

[mladen (Mladen Stankovic)]

Hello,

I want to make visualization for number of messages in a queue. I am trying various options, but I feel it should not be that complicated and I am miss something. I have 34 different queues and check every minute how much messages I got in every queue. My goal is to display every queue that is not empty with the number of messages in time.

This is my output of one queue from Elasticsearch:

qmraz2:true queue_manager:QMRAZ2 prospector.type:log beat.version:6.2.2 beat.name:localhost beat.hostname:localhost source:/logs/monitoring/QMRAZ2/QMRAZ2_monitoring_localhost.log message:12-04-18;20:44:11,OHSA.SOT.MAPPING.FAULT,4,QMRAZ2 @timestamp:April 12th 2018, 20:44:11.000 queue_name:OHSA.SOT.MAPPING.FAULT curdepth:4 @version:1 offset:35,023,933 host:localhost date_time:12-04-18;20:44:11 tags:beats_input_codec_plain_applied _id:CVEsu2IB9DvZCDLxMIyW _type:doc _index:qmraz2 _score:

Can you please tell me how can I do this?

[thomasneirynck]

@mladen,

You could do a terms aggregation (= Aggregation) on the identifier of a queue, and then count the number of documents (=Metric) for each (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html).

You could nest this under a date-range aggregation (now - 1m), and Elasticsearch will omit from the result every date-range that has no terms (ie. queues). (https://www.elastic.co/guide/en/elasticsearch/reference/6.2/search-aggregations-bucket-daterange-aggregation.html)

[mladen (Mladen Stankovic)]

Hello @thomasneirynck, thank you for links. I will try to setup on this way.