Can't select actual index fields in kibana time series builder


#1

Hi guys,
I know that time series builder is marked as experimental in 6.5.4, but was hoping that it is me not the builder :slight_smile: and someone could steer me in right direction.
My environment: rhel 7.3, postgresql 11.2, elk 6.5.4 and I am running a bunch of monitoring queries to collect postgres cluster metrics.
one of the query produces io metrics and outputs in a csv format file.
schemaname, heap_read, heap_hit,idx_read,idx_hit,etc ...
I use filebeat to read the file and ship it to logstash that parses data, creates and index and inserts data.
here is logstash.yml
input {
beats {
port => 5044
}
}

filter {
csv {
columns => [
"qry_timestamp", "schemaname",
"heap_read", "heap_hit", "idx_read", "idx_hit", "toast_read",
"toast_hit", "tidx_read", "tidx_hit"
]

       separator => ","
       remove_field => ["message"]

    }
    date {

target => "@timestamp"

            match => ["qry_timestamp", "ISO8601"]
    }
    mutate {
            convert => {
                    "heap_read" => "integer"
                    "heap_hit" => "integer"
                    "idx_read" => "integer"
                    "idx_hit" => "integer"
                    "toast_read" => "integer"
                    "toast_hit" => "integer"
                    "tidx_read" => "integer"
                    "tidx_hit" => "integer"
            }
    }

}
output {
elasticsearch {
hosts => [ "myhost:7200" ]
action => "index"
index => ["diskio_index-%{+YYYY.MM.dd}"]

document_type => ["csv"]

    }

    # output dots while processing
    #stdout { codec => rubydebug }
    stdout { codec => "dots" }
    # if date parse failure, dump it to screen for review
    if "_dataparsefailure" in [tags] {
            stdout {codec => "rubydebug"}
    }

it does create index and inserts data. I can see it in kibana discover, but when I tried to visualize it (my search query saved in discover ) in time series builder following Disk IO sample visualization provided with Kibana I could not really do it.
whenever I tried to select a field in Data section of Time Series, for example, I could not see any fields from my index. All I saw in drop-down list was number and offset, and number is grayed out, so I can't even select it. Choosing offset renders one of the value, I believe the first integer on the first record and that is it.
I can't touch second metric no matter what I do. I tried to visualize index itself instead of saved query with the same result.
What am I doing wrong? did I build the index wrong? how then it should be build?
Why I cannot choose/see field of my index?
any help would be greatly appreciated.
Thank You.


#2

BTW, I tried to visualize the same index/query in line, area and bar charts with the same problem - when I am trying to choose field for a metric I can choose only "offset". the second field is "number" and it is grayed out. I cannot see any of my diskio_index-* fields.