Fields not being categorized properly

Elastic Stack Kibana
1

I'm new to the elastic stack. Trying to visualize some data from our squid proxies and am having trouble. I've set a field in the grok filter to specifically set the type to number like so:

%{NUMBER:reply_size_include_header}

But when I view the fields in the Kibana index patterns page, it's classifying it as a string and not a number so I can't do generate some graphs. The data itself is parsed correctly. I don't know enough about these products to know where to look. Anyone seen this kind of issue before?

2

When you say you set the field in the grok filter, where did you do this exactly?

If you goto Management > Index Patterns and select your index pattern. Then, click the refresh button in the top right-hand corner of the screen. After doing that, what does the row includes for this column?

3

So in /etc/logstash/conf.d I created a .conf file for the squid logs and within the grok filter I specifically set for the reply_size_include_header field to be a number like shown in my first post. If I go to Management -> Index Patterns and refresh it shows string for both the reply_size_include_header and reply_size_include_header.keyword fields. The value itself in the log is a number so it’s not like it needs to do any conversions.

4

That is not setting it as a number, try;

%{NUMBER:reply_size_include_header:int}
5

I tried this, restarted logstash, and it doesn't categorize this field as a number, unfortunately.

6

Now I'm noticing in Kibana that there's a conflict on the field type. The two choices that I have is URL and String which doesn't really make much sense at all.

7

You may want to use stdout+rubydebug to see what the structure looks like.

8

ok so I've configured the following output

output {
        stdout { codec => rubydebug }
        if [type] == "SQUID" {
            elasticsearch {
                                hosts=> "localhost:9200"
                                index => "logstash-squid-%{+YYYY.MM.dd}"
                        }
        }
}

where can I see the outputs? I figured they'd be in logstash-plain.log but I'm not seeing anything there.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.