I am collecting wait statistics from an RDBMS and would like to visualize them in Kibana. Once a minute I am getting the list of wait type and the duration of the wait. From one interval to the next the type of waits happening can vary and there are dozens of possible wait types.
I started out with an index like this:
put /mssql/
{
"mappings": {
"waits" : {
"properties": {
"timestamp" : { "type" : "date", "format": "yyyy-MM-dd HH:mm:ss" },
"sql_instance_name" : { "type" : "text" },
"waits_ms": { "type" : "object" }
}
}
}
}
And then indexes data like this:
POST mssql/waits
{
"timestamp" : "2017-11-10 14:56:05",
"sql_instance" : "MYSERVER\\INTERNAL",
"waits_ms": [
{
"SOS_SCHEDULER_YIELD" : 100
},
{
"WRITELOG" : 284
},
{
"PAGEIOLATCH_SH" : 11
},
{
"PREEMPTIVE_OS_AUTHENTICATIONOPS" : 446
},
{
"PAGEIOLATCH_EX" : 109
}
]
}
The problem is that when I visualize it in Kibana I need to select the field names and the field names could be different on the next document. For example it could look like:
POST mssql/waits
{
"timestamp" : "2017-11-10 14:57:05",
"sql_instance" : "MYSERVER\\INTERNAL",
"waits_ms": [
{
"ASYNC_NETWORK_IO" : 971
},
{
"CMEMTHREAD" : 663
},
{
"SOS_SCHEDULER_YIELD" : 400
},
{
"ASYNC_IO_COMPLETION" : 389
},
{
"WRITELOG" : 277
}
]
}
So my field choices in Kibana show up as waits_ms.SOS_SCHEDULER_YIELD
, waits_ms.ASYNC_IO_COMPLETION
, etc. I don't see a way to say ALL waits_ms
fields, something like waits_ms.*
.
Is there a better approach here?
Thanks