Logstash2.4 "if" subsentence doesnt work

Elastic Stack Logstash
1

I'm using logstash-2.4.1 and elasticsearch-2.4.6.
Depending following config,logstash can write the log data into EL and build the index

input {
            file {
     type => "toll"
    path => ["/path/to/file4/*.log"]
    ignore_older => 0
    start_position => beginning
    sincedb_path => "/path/to/file4/sincedb.txt"
    codec => json {
    charset => ["UTF-8"]
 }
}
}

    output{
     elasticsearch {        
    			hosts => [ "192.168.4.56:9200","192.168.15.145:9200","192.168.15.41:9200" ]        
    			index => "data_%{application}"    }

    }

But when I use "if" condition to chose the different output,logstash can't write anything into EL.
following is the config.

 input {

            file {
                type => "toll"
                path => ["/path/to/file4/*.log"]
                ignore_older => 0
                start_position => beginning
                sincedb_path => "/path/to/file4/sincedb.txt"
                codec => json {
              charset => ["UTF-8"]
            }
             }
            }

    output{
    if [type] == "toll"{
     elasticsearch {        
    			hosts => [ "192.168.4.56:9200","192.168.15.145:9200","192.168.15.41:9200" ]        
    			index => "data_%{application}"    }
      }

    }

Are there any problems in my config file,or Is the version issue?
thanks a lot;

2

Use a stdout { codec => rubydebug } output to dump the raw event. Does the type field really contain "toll"? Are you getting any events at all?

1 Like
3

thank you reply.

I output the data,like this:
{
"application" => "smap-special",
"applicationVersion" => "0.0.1",
"code" => "",
"date" => "2015-05-18T09:56:33.996Z",
"description" => nil,
"host" => "0:0:0:0:0:0:0:1",
"level" => "INFO",
"loggerName" => "smap_scenic.controller.ScenicController",
"message" => "smap tollcost service",
"requestUID" => nil,
"threadName" => "http-nio-8097-exec-4",
"throwable" => nil,
"type" => "response",
"user" => nil,
"@version" => "1",
"@timestamp" => "2018-03-16T01:11:16.612Z",
"path" => "/path/to/file4/smap_toll_allpath.2018-03-11.log"
}

So,I changed the config file like this:
output{
if [type] == "response"{
elasticsearch {
hosts => ["192.168.15.145:9200"]
index => "data_%{application}" }
}
}

But it still doesn't work,no index is created.

I thought the "type" is from "file" that I have set not from the json field.

I deleted the sincedb and remove the "if condition",the index was created and the data flowed into EL.

Hope to get your help,thanks.

4

@magnusbaeck thank you very much.

I change the config,like this:
input {

   file {
        type => "response"
        path => ["/path/to/file4/*.log"]
        ignore_older => 0
        start_position => beginning
        sincedb_path => "/path/to/file4/sincedb.txt"
        codec => json {
      charset => ["UTF-8"]
  }
  }
}

#output{stdout { codec => rubydebug }}


output{
    if [type] == "response"{
     elasticsearch {
                        hosts => ["192.168.15.145:9200"]
                        index => "data_%{application}"
                                }
      }

        }

It works!
input "type" must be same with output "type" and must exsits in log json.
I thought that "the type" has nothing to do with the log data.
thanks a lot!

5

I thought the "type" is from "file" that I have set not from the json field.

Apparently the file input doesn't overwrite the type field read from the file, so if it's present in the file it'll get used and otherwise the file input's type option will prevail.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.