Logstash Metrics Using Dots - problems with ES 2.x?


(Ninjada) #1

hello,

just going through all the potential problems with my upgrade to latest elk stack versions (logstash 2.x / elasticsearch 2.x / kibana 4.x)

received a number of warnings about dots in field names:

Dots in field names lead to ambiguous field resolution, in fields: logs:http_%{response}.count, logs:http_%{response}.rate_15m, logs:http_%{response}.rate_1m, logs:http_%{response}.rate_5m, logs:http_200.count, logs:http_200.rate_15m, logs:http_200.rate_1m, logs:http_200.rate_5m,

with the following reason - elasticsearch 2.x changes for field names:

https://www.elastic.co/guide/en/elasticsearch/reference/2.0/breaking_20_mapping_changes.html#_field_names_may_not_contain_dots

Noticed this was coming from my use of metrics for http_%{response} codes in my log filters. This is what I have in my cfg:

            metrics {
                    meter => [ "http_%{response}" ]
                    add_tag => "metric"
                    flush_interval => "60"
            }

Seems for each metric logstash puts out .rate1m .rate5m & .rate15m etc. but the latest documentation says it should be outputting with underscores:

https://www.elastic.co/guide/en/logstash/current/plugins-filters-metrics.html

I'd assume that should cover it, but that doesn't appear to be the case when im using 'metrics' and dots are still all over the shop.

running latest logstash 2.1.0 & ES 2.1.0


(system) #2