Logstash lost eventlog from winlogbeat


(Pavel Kozlov) #1

Hi
i dont undestend where i wrong (.
on my server (Ubuntu) install ELK
logstash recive syslog and netflow messages from network devices - all work all right.
Install on some windows server Winlogbeat
use send log to elasticsearch direct - all work right
use send log to logstash - event can not send to ES
errors from LS - none.

LS config

    input {
            # netflow v9
            udp {
                    type => "mtnfv9"
                    port => 9995
                    codec => netflow {
                    versions => [9]
                    }

            }
            #tcp syslog stream via 5140
            tcp {
                    type => "mtsl"
                    port => 5141

                    }

            #udp syslogs tream via 5140
            udp {
                    type => "mtsl"
                    port => 5141
 
                    }
            # windows eventlog from winlogbeats
            beats {
                    type => msel
                    port => 5044
            }
    }
    filter {

            if [type] == "mtnfv9" { some filters}
    else if [type] == "msel" {none filters}
    else if [type] == "mtsl" {some filters}

    }

    output {
                      elasticsearch {
                            hosts => ["192.168.1.42"]
                            manage_template => false
                            index => "%{[@metadata][type]}-%{+YYYY.MM.dd}"
                            document_type => "%{[@metadata][type]}"
                           }   
    }

see packet thow tcpdump - packet sent and recive
plugin - logstash-input-beats - installed


(Andrew Kroh) #2

Setting the type => msel on the beats input will not take effect because type is already set in the event sent by winlogbeat. See the beats input documentation here.


(Andrew Kroh) #3

Try using the following output so you can see the full event from Winlogbeat then adjust your filters appropriately.

stdout { codec => rubydebug { metadata => true } }


(Pavel Kozlov) #4

HI andrewkroh,
thank for your reply
i did it your recomendation.
no effekt (

input {
        beats {
                port => 5044
        }
}

output {
        stdout { codec => rubydebug }
        elasticsearch {
                hosts => ["192.168.1.42"]
                manage_template => false
                index => "%{[@metadata][type]}-%{+YYYY.MM.dd}"
                document_type => "%{[@metadata][type]}"
        }
}

root@it-logmon01:~# /opt/logstash/bin/logstash-plugin list | grep "beats"
logstash-input-beats


(Pavel Kozlov) #5

New information
error

Beats input: unhandled exception {:exception=>#<SystemCallError: Unknown error - No message available>, :backtrace=>["org/jrub y/RubyIO.java:3020:insysread'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-2.2.9/lib/lumberjack/beats/ server.rb:463:in read_socket'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-2.2.9/lib/lumberjack/beats/s erver.rb:443:inrun'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-2.2.9/lib/logstash/inputs/beats_suppo rt/connection_handler.rb:34:in accept'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-2.2.9/lib/logstash/ inputs/beats.rb:211:inhandle_new_connection'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-2.2.9/lib/lo gstash/inputs/beats_support/circuit_breaker.rb:42:in execute'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-be ats-2.2.9/lib/logstash/inputs/beats.rb:211:inhandle_new_connection'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-i nput-beats-2.2.9/lib/logstash/inputs/beats.rb:167:in run'"], :level=>:error}

may be i most manual update it plugin from gem?


(Andrew Kroh) #6

You could try updating the plugin with these commands.


(Pavel Kozlov) #7

Andrey thank for your reply.
then i run update plugins - process freeze (

/opt/logstash# ./bin/logstash-plugin update logstash-input-beats
You are updating logstash-input-beats to a new version 3.0.3, which may not be compatible with 2.2.9. are you sure you want to proceed (Y/N)?
Y
Updating logstash-input-beats

i most update plugin for it corect work?


(Pavel Kozlov) #8

anybody help me with my problem?
any idea?


(Andrew Kroh) #9

Based on that warning, I'd say the version is incompatible and my suggestion of making sure you have the latest plugin version wasn't so good. I'd revert to a clean Logstash install.

You'll probably get better responses regarding the Logstash exception you encountered if you post in the Logstash category. Include your OS, Logstash version, config, and the exception when you post.


(Pavel Kozlov) #10

Andrey thank you
at this time i write message in logstash category )


(system) #11

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.