Question about mapping types in 6.X (and next versions)

[leandrojmp (Leandro Pereira)]

Hello,

I recently updated from 5.6.2 to 6.2.1 and one the problems I had was related to the mapping types.

I was using filebeat to send data and setting the 'type' field in filebeat.yml, in Logstash I use the type value in filters and output, when I updated Logstash and Elasticsearch to 6.2.1, the type field from filebeat was ignored and the pipeline was broken, changing the name of the field from 'type' to a custom field named 'doctype' in filebeat.yml and in the logstash configuration solved the problem.

Since I heavily depend on the type value to use filters and outputs on my logstash configurations, my question is: will I be able to set the type field on other inputs like tcp or udp on the future versions?

Right now, besides filebeat, in which I cannot set the type field anymore, I have a 5.4 logstash instance with tcp and udp inputs where I set the type field like below:

tcp {
        host => "ip-address"
        port => 2514
        type => "custom-type"
}

udp {
        host => "ip-address"
        port => 2514
        type => "custom-type"
}

Reading the documentation it shows that for version 6.2 you still can set the type field for those inputs, but it will be supported in future versions?

Should I start to migrate from using the type field for using tags?

[magnusbaeck (Magnus Bäck)]

You'll always be able to have a field named type. It's how that string is used that's about to change. Maybe type => "foo" will stop working at some point but then you can just transform that into add_field => { "type" => "foo" }.

[leandrojmp (Leandro Pereira)]

Thanks @magnusbaeck,

I'm going to review my logstash pipelines and start using add_field in the inputs and add a custom field to use in the filters and in the outputs.

I think something like this will solve my problems now and avoid future problems.

input {
    tcp {
        add_field => { "doctype" => "custom-type" }
    }
}

filter {
    if [doctype] == "custom-type"  { }
}