[SOLVED] Logstash: No output, no logs

Elastic Stack Logstash
1

Hi all,

My architecture:

Equipments --> syslog-ng --> redis <-- logstash --> elasticsearch <--kibana

I made a lot of tests and I'm sure that syslog-ng received correctly logs, that redis put the logs in a list (key: test), that logstash pop each elements on the list (when logstash is start my list is all the time empty, when I stop logstash I can LPOP manually the redis list and get elements).

When logstash is down:

127.0.0.1:6379> LLEN test
(integer) 22

When logstash is up:

127.0.0.1:6379> LLEN test
(integer) 0

For the first time, I get an issue when I used default codec (JSON) in logstash input redis plugin because of "-" character inside the log:

{:timestamp=>"2016-04-07T08:54:34.567000+0200", :message=>"JSON parse failure. Falling back to plain-text", :error=>#<LogStash::Json::ParserError: Unexpected character ('-' (code 45)): Expected space separating root-level values
 at [Source: [B@526fbdeb; line: 1, column: 6]>, :data=>"2016-04-07T08:53:34+02:00 m010gbigip1-prod notice icrd:  5957,13693,             RestRequestSender,   INFO,Connection idle too long fd:15 cached.", :level=>:error}

And this one as output from logstash:

"tags":["_jsonparsefailure"]

So, I changed the codec input by "line" and now I don't have any issue. But I don't have any logstash output too.

Logstash input configuration:

input {
      redis {
          host => "127.0.0.1"
          data_type => "list"
          key => "test"
          codec => "line"
      }
 }

Logstash filter configuration:
(The grok patterns work fine: tested)

> filter {
>       grok {
>           match => {"message" => [ "%{F5_TMM}", "%{F5_TMM_BIS}", "%{F5_TMM_TIER}", "%{F5_TMM1}", "%{F5_TMM1_BIS}", "%{F5_TMM1_TIER}", "%{F
>           remove_field => [ "to_delete" ]
>       }
>       mutate {
>           add_field => { "@source_host" => "%{client}" }
>       }
>       dns {
>           nameserver => "152.9.102.222"
>           reverse => [ "@source_host" ]
>           action => "replace"
>      }
>  }

Logstash output configuration:

output {
      file {
          path => "/etc/logstash/test_f5_output.txt"
      }
      stdout {
      }
 }

Do you have any idea to retrieve the message from redis without json failure (and if I use "line" codec I get nothing) ? Do I need to use another codec ?

Tanks in advance,
Alex

2

Just to share the output when I started logstash in debug mode, I get the log below (with codec "line"), but I don't have any logstash output:

> config LogStash::Filters::DNS/@reverse = ["@source_host"] {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Filters::DNS/@action = "replace" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Filters::DNS/@add_tag = [] {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Filters::DNS/@remove_tag = [] {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Filters::DNS/@add_field = {} {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Filters::DNS/@remove_field = [] {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Filters::DNS/@periodic_flush = false {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Filters::DNS/@timeout = 2 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> Plugin not defined in namespace, checking for plugin file {:type=>"output", :name=>"file", :path=>"logstash/outputs/file", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}
> Plugin not defined in namespace, checking for plugin file {:type=>"codec", :name=>"json_lines", :path=>"logstash/codecs/json_lines", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}
> config LogStash::Codecs::JSONLines/@charset = "UTF-8" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Codecs::JSONLines/@delimiter = "\n" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@path = "/etc/logstash/test_f5_output.txt" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@codec = <LogStash::Codecs::JSONLines charset=>"UTF-8", delimiter=>"\n"> {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@workers = 1 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@flush_interval = 2 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@gzip = false {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@filename_failure = "_filepath_failures" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@create_if_deleted = true {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@dir_mode = -1 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::File/@file_mode = -1 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> Will start workers for output {:worker_count=>1, :class=>LogStash::Outputs::File, :level=>:debug, :file=>"logstash/output_delegator.rb", :line=>"34", :method=>"initialize"}
> Plugin not defined in namespace, checking for plugin file {:type=>"output", :name=>"stdout", :path=>"logstash/outputs/stdout", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}
> config LogStash::Codecs::Line/@charset = "UTF-8" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Codecs::Line/@delimiter = "\n" {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::Stdout/@codec = <LogStash::Codecs::Line charset=>"UTF-8", delimiter=>"\n"> {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> config LogStash::Outputs::Stdout/@workers = 1 {:level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
> Will start workers for output {:worker_count=>1, :class=>LogStash::Outputs::Stdout, :level=>:debug, :file=>"logstash/output_delegator.rb", :line=>"34", :method=>"initialize"}
3

I found the solution.

I put the codec "plain" in redis input plugin.