Pattern not applied

Hi,
I'm rather new to Elastic and now I'm running into a strange problem I do not understand and did not find the right pointer to fix it.

I think everyone worked fine in the beginning but then I added a new input and output definition to logstash which most likely broke something but to me it does not make much sense.
So the current scenario:
I'm collecting via logstash from two different inputs. One is filebeat and the other http json.
I want both to end up in different indexes and apply different filters.
So I tag the JSON input manually with a certain tag ("wforce"):

input {
    http {
         port => 8080
         codec => json
         type => wforce_report
         add_field => { "input" => "http" }
         tags => ["wforce"]
    }
}

input {
    beats {
      port => 5044
      tags => ["beats"]
    }
}

I skip the filter definitions since I do not think they are relevant here.

Output definition looks like this:

output {
    if "wforce" in [tags] {
      elasticsearch {
           hosts => "elasticsearch:9200"
           index => "logstash-wforce-%{+YYYY.MM.dd}"
           template => "/tmp/templates/wforce_template.json"
           template_name => "wforce"
           template_overwrite => true
           user => elastic
           password => changeme
      }
    } else {
      elasticsearch {
           hosts => "elasticsearch:9200"
           index => "logstash-mail-%{+YYYY.MM.dd}"
           user => elastic
           password => changeme
      }
    }
}

So the index definition works totally fine but something is strange with the template.

{
    "index_patterns" : ["logstash-wforce*"],
    "settings" : { "index.refresh_interval" : "5s"},
    "mappings" : {
            "dynamic_templates" :
            [
                {
                    "minor_fields" : {
                        "match" : "*minor",
                        "mapping" : { "type" : "integer", "index" : true }
                    }
                },
                {
                    "major_fields" : {
                        "match" : "*major",
                        "mapping" : { "type" : "integer", "index" : true }
                    }
                },
                {
                    "string_fields" : {
                        "match_mapping_type" : "string",
                        "mapping" : { "type" : "keyword", "index" : true }
                    }
                }
            ],
            "properties" : {
                "geoip"  : {
                    "dynamic": true,
                    "properties" : {
                        "ip": { "type": "ip" },
                        "location" : { "type" : "geo_point" },
                        "latitude" : { "type" : "half_float" },
                        "longitude" : { "type" : "half_float" }
                    }
                },
                "policy_reject": { "type": "boolean" },
                "success": { "type": "boolean"},
                "tls": { "type": "boolean" },
                "t": { "type": "float" }
            }
    }
}

Now what happens is that I certainly see in elastic in the index management's mapping tab that everything seems to be there. Especially also the minor_fields and major_fields definitions which should make sure these are saved as integers.

But the index pattern fields does not have these as numbers but strings. E.g. the policy_reject boolean works but that just might be because ES recognizes it itself like this.

Any pointer?

Thanks,
Wolfgang

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.