Aggregate multiple rows fields data in single data

[vinay_garg]

Hi,

Environment: Elasticsearch, Logstash, Kibana(5.6)

I have multiple Fields data. Each field is related to JMX.
JMX configuration is:

"host":"localhost",
"port":1099,
"url" : "service:jmx:rmi:///jndi/rmi://localhost:1099/karaf-root",
"username" : "admin",
"password": "admin",
"alias" : "JMX",
"queries" : [
{
"object_name" : "org.apache.activemq:type=Broker,brokerName=amq,destinationType=Queue,destinationName=*",
"object_alias" : "ActiveMQ",
"attributes" : ["QueueSize","EnqueueCount","DispatchCount","DequeueCount","Name"]
}
]
}

All fields data is properly indexing. As available in screenshot

file:///home/vinay/Pictures/Screenshot%20from%202017-09-13%2018-26-37.png

Now each row have different timestamp but each column field are related to each other. So i have to aggregate all these fields (DispatchCount, DequeueCount,EnqueueCount,QueueName) data into single table.

Can anyone suggest how can i do it?

[ppisljar (Peter Pisljar)]

i am not sure i completely understand your problem, but you could try to create a table visualization and define the following metrics:

• sum of DispatchCount
• sum of DequeCount
• sum of Enqueue Count
• top 5 of QueueName

then for split rows select Date Histogram, with appropriate period (minute?)
you could also split rows by term, if there is another field that is common to all your rows that you would want to join, like 'host' or sth

let me know if this helps

[vinay_garg]

Hi,

I have this data "QueueSize","EnqueueCount","DispatchCount","DequeueCount","Name"".

My requirement is to create a Table which can show information in this way:

QueueName | QueueSize | EnqueueCount | DispatchCount | DequeueCount

MyQueue1 | 100 | 10 | 5 | 5

MyQueue2 | 500 | 10 | 5 | 5

[ppisljar (Peter Pisljar)]

so just as i described above, but split rows with term aggregation on QueueName field

[vinay_garg]

I'm using the following metric:
Unique count of DispatchCount
Unique count of DequeCount
Unique count of Enqueue Count

I think split will not work because each field are not part of same row.

How can i create table "MyQueue1 | 100 | 10 | 5 | 5"

[ppisljar (Peter Pisljar)]

i think you shouldn't be using unique count but sum .... but i don't know your data ..

so just as i said, split rows by term aggregation on queuename field, add metrics for all the other fields.

[vinay_garg]

Let me clear it.

We are having original data as mention in below screenshot. One metric_path having multiple fields(ActiveMQ.ActiveMQ.Name,ActiveMQ.ActiveMQ.Name, etc..).

We set different field name for each value by using filters
if ("DispatchCount" in [metric_path] ) {
ruby {
code => "event.set('DispatchCount' , event.get('metric_value_number') )"
}
}
We have created fields frm metric_path as mentioned in screenshot:

Each field is behave as different row rather than same row with other fields.

Now i have to visualise the multiple row data into single. Split will work that case if we have data only in single row.