Hi guys,
I know that time series builder is marked as experimental in 6.5.4, but was hoping that it is me not the builder and someone could steer me in right direction.
My environment: rhel 7.3, postgresql 11.2, elk 6.5.4 and I am running a bunch of monitoring queries to collect postgres cluster metrics.
one of the query produces io metrics and outputs in a csv format file.
schemaname, heap_read, heap_hit,idx_read,idx_hit,etc ...
I use filebeat to read the file and ship it to logstash that parses data, creates and index and inserts data.
here is logstash.yml
input {
beats {
port => 5044
}
}
filter {
csv {
columns => [
"qry_timestamp", "schemaname",
"heap_read", "heap_hit", "idx_read", "idx_hit", "toast_read",
"toast_hit", "tidx_read", "tidx_hit"
]
separator => ","
remove_field => ["message"]
}
date {
target => "@timestamp"
match => ["qry_timestamp", "ISO8601"]
}
mutate {
convert => {
"heap_read" => "integer"
"heap_hit" => "integer"
"idx_read" => "integer"
"idx_hit" => "integer"
"toast_read" => "integer"
"toast_hit" => "integer"
"tidx_read" => "integer"
"tidx_hit" => "integer"
}
}
}
output {
elasticsearch {
hosts => [ "myhost:7200" ]
action => "index"
index => ["diskio_index-%{+YYYY.MM.dd}"]
document_type => ["csv"]
}
# output dots while processing
#stdout { codec => rubydebug }
stdout { codec => "dots" }
# if date parse failure, dump it to screen for review
if "_dataparsefailure" in [tags] {
stdout {codec => "rubydebug"}
}
it does create index and inserts data. I can see it in kibana discover, but when I tried to visualize it (my search query saved in discover ) in time series builder following Disk IO sample visualization provided with Kibana I could not really do it.
whenever I tried to select a field in Data section of Time Series, for example, I could not see any fields from my index. All I saw in drop-down list was number and offset, and number is grayed out, so I can't even select it. Choosing offset renders one of the value, I believe the first integer on the first record and that is it.
I can't touch second metric no matter what I do. I tried to visualize index itself instead of saved query with the same result.
What am I doing wrong? did I build the index wrong? how then it should be build?
Why I cannot choose/see field of my index?
any help would be greatly appreciated.
Thank You.