Logstash File Input not setting 'type'

Elastic Stack Logstash
1

Hi,

I am using a file input filter in logstash, and setting the 'type' to a custom value, however when logs are sent to elasticsearch the type is displaying as "logs" and not the value that I am setting.

This in turn is breaking my filter plugin as I am running grok filters based on the 'type' value (within IF statements). For example, my file input plugin would look like:

input {
  file {
    type => "my-log"
    add_field => {
        "program" => "my-program"
        "server" => "my-server"
    }
   path => [ "/usr/local/etc/logstash/my-file.log" ]
   start_position => "beginning"
   sincedb_path => "/tmp/sincedb_ls"
   codec => multiline {
      pattern => "^\["
      what => "previous"
      negate => true
    }
  }
}

I set type => "my-log" in my file input, however this is not being saved. Instead I get a type of "logs" in Elasticsearch (excerpt taken from Marvel Sense _search API):

{
        "_index": "logstash-my-server-2015.09.14",
        "_type": "logs",
        "_id": "AU_LV8iKTwurq_6KlASy",
        "_score": null,
        "_source": {
           "message":

etc etc......

Has anyone had similar issues?

Thanks
Luke

2

That's because you need to set document_type in the ES output.
You can set that to whatever you want, including %{type} if you want it to be dynamic.