Want to split vertical bars based on my log fields


(Murat) #1

I have an application log file consists of following log levels: INFO, WARN, ERROR, DEBUG. Following filter criteria works fine in logstash config file:

filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} [%{DATA:thread_name}]?-[%{DATA:class}] %{GREEDYDATA:message}" }
}
date {
match => ["timestamp", "yyyy-MM-dd HH:mm:ss,SSS"]
target => "@timestamp"
}
}

output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

I can see log-level field in "Discover" view of Kibana. However, I would like to visualize my app log as following: Split a vertical bar at a given moment to show how many ERROR logs, how many INFO logs, etc. are hit at a given moment.
When I go to "Visualize" tab and try to do "Add sub-buckets", "split bars" on X-axis, sub-aggregation="Terms"; I cannot see the field: "log-level" under the selectable "Field" options. Could you please help me to split the bars based on log-level?
Thanks.


(Jon Budzenski) #2

Which version of kibana/elasticsearch? Can you share your mappings for the log-level field? I'm guessing either the mappings need to be updated on the Kibana end (on the index pattern page, clicking the orange refresh button), or the field is mapped as text, which will cause it to be filtered out from the terms list.


(Murat) #3

Thanks for the reply!
I am using the version 5.1.1.
As I am quite new in ELK, I have no idea where to find the mappings for my log fields. Could you please describe where can I find the mappings of the fields (for future purposes)?
Btw, I clicked on the refresh button on the index pattern page and I can now split bars based on log-level field.
Thanks a lot for your help. Would be nice to hear where can I find the mapping of a field though :slight_smile:


(Jon Budzenski) #4

Mappings are defined in elasticsearch, and you can find them by appending _mapping to your request. An example would be http://localhost:9200/my_index_name/_mapping. More details in the docs, https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.