COMBINEDAPACHELOG grok definition creating all fields as type string

Elastic Stack Logstash
1

Hi all,

I'm using the %{COMBINEDAPACHELOG} grok pattern for my Apache logs and parsing to Elasticsearch service (v2.3) in AWS and all the fields are created as string type. I have tried a number of custom patterns and I'm still having the same issue. I also have a custom IIS grok pattern running and it's working fine.

Here's my grok definition:

filter {
  if [type] == "apache-access" {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }

    geoip {
        source => "clientip"
    }
  }

output {
  if [type] == "apache-access"  {
elasticsearch {
  hosts => ["my-elasticsearch-url:80"]
  index => "apache-%{+YYYY.MM.dd}"
  manage_template => false
}
  }
 stdout {codec => rubydebug}
}

Here's my elasticsearch index/mapping:

Pasted image384×1007 22.4 KB

Has anyone had the same experience? Any feedback, input or help would be much appreciated!

2

Yes, this is arguably a bug in the grok pattern(s). You should use a mutate filter's convert option to convert e.g. the bytes field to an integer. This issue is relevant:

3

Thanks Magnus, I've created a mutate filter and converted the relevant fields to integer and float and it now works.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.