Logstash not receiving logs from Winlogbeat 5.0 alpha

Elastic Stack Beats
1

upgraded winlogbeat to v 5.a and not I'm not able to receive any data to logstash see below error on log stash

D:\Elastic\logstash\bin>logstash -f logstash.conf
io/console not supported; tty will not be manipulated
Settings: Default pipeline workers: 2
←[31mInvalid setting for elasticsearch output plugin:

output {
elasticsearch {
# This setting must be a boolean
# Expected boolean 'true' or 'false', got "True"
manage_template => "True"
...
}
} {:level=>:error}←[0m←[31mPipeline aborted due to error {:exception=>#<LogStash::ConfigurationError: Something is wrong with your configuration.>, :backtrace=>["D:/Elastic/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/config/mixin.rb:134:in config_init'", "D:/Elastic/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/outputs/base.rb:63:ininitialize'", "D:/Elastic/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:74:in register'", "D:/Elastic/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:instart_workers'", "org/jruby/RubyArray.java:1613:in each'", "D:/Elastic/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:instart_workers'", "D:/Elastic/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136:in run'", "D:/Elastic/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb:473:instart_pipeline'"], :level=>:error}←[0m
stopping pipeline {:id=>"main"}
The signal HUP is in use by the JVM and will not work correctly on this platform

2

this is my logstash.conf

input {
  beats {
    port => 5044
  }
 }
 output {
  elasticsearch {
    hosts => "192.168.110.100:9200"
    manage_template => True
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}
3

this is the winlog beat 5a config

winlogbeat.event_logs:
    - name: Application
    - name: Security
    - name: System
    - name: Active Directory
    - name: DFS Replication
    - name: Directory Service
    - name: DNS Server
    - name: File Replication Service
    - name: Key Management Service

output.logstash:
  # The Logstash hosts
  hosts: ["192.168.110.100:5044"]
  
    # Template name. By default the template name is winlogbeat.
  template.name: "winlogbeat"
  
    # Overwrite existing template
  template.overwrite: false

    # Path to template file
  template.path: C:\winlogbeat5\winlogbeat.template.json


logging.level: debug
4

This page contains a working example of what your Logstash configuration should look like for use with Beats.

As for your Winlogbeat configuration, those template options are not available when using the logstash output (they only work for the elasticsearch output). You must manually install the index template to Elasticsearch.

5

this is the result when I try to install the plugin.. it was previously working... should I remove log stash completely ?

D:\Elastic\logstash\bin>logstash-plugin install logstah-imput-beats
io/console not supported; tty will not be manipulated
Validating logstah-imput-beats
Plugin logstah-imput-beats does not exist
ERROR: Installation aborted, verification failed for logstah-imput-beats

6

For newer versions the plugin should already be included with Logstash. But in any case, you have a typo causing your problem: "imput" should be "input"

7

note I had loaded the template manually with no problem earlier

8

loaded that just now
D:\Elastic\logstash\bin>logstash-plugin install logstash-input-beats
io/console not supported; tty will not be manipulated
Validating logstash-input-beats
Installing logstash-input-beats
Installation successful

will start beats and see if its fixed

9

ok I restarted the logstash and winlog beat and no index for winlogbeat

yellow open test 1 1 0 0 159b 159b
yellow open .kibana 1 1 24 6 56kb 56kb

10

FYI this was working with the current release of Winlogbeat

11

Did you delete the registry file used by Winlogbeat to persist it's read location? Try deleting C:/ProgramData/winlogbeat/.winlogbeat.yml to ensure there is data for Winlogbeat to read and send to Logstash.

Are there any errors in the Winlogbeat log file?

12

I'm new to elastic so if you don't mind where is the log for the beat located. ?

13

Check in C:/ProgramData/winlogbeat/logs/

14

No log file for today and the service is running

15

2016-08-04T17:32:08-04:00 INFO EventLog[System] Stop processing.
2016-08-04T17:32:08-04:00 DBG WinEventLog[System] Closing handle
2016-08-04T17:32:08-04:00 INFO EventLog[File Replication Service] Successfully published 100 events
2016-08-04T17:32:08-04:00 INFO EventLog[File Replication Service] Stop processing.
2016-08-04T17:32:08-04:00 DBG WinEventLog[File Replication Service] Closing handle
2016-08-04T17:32:08-04:00 INFO EventLog[DNS Server] Stop processing.
2016-08-04T17:32:08-04:00 DBG WinEventLog[DNS Server] Closing handle
2016-08-04T17:32:08-04:00 INFO EventLog[Directory Service] Successfully published 100 events
2016-08-04T17:32:08-04:00 INFO EventLog[Directory Service] Stop processing.
2016-08-04T17:32:08-04:00 DBG WinEventLog[Directory Service] Closing handle
2016-08-04T17:32:08-04:00 INFO EventLog[Application] Stop processing.
2016-08-04T17:32:08-04:00 DBG WinEventLog[Application] Closing handle
2016-08-04T17:32:08-04:00 INFO EventLog[DFS Replication] Stop processing.
2016-08-04T17:32:08-04:00 DBG WinEventLog[DFS Replication] Closing handle
2016-08-04T17:32:08-04:00 INFO EventLog[Security] Successfully published 100 events
2016-08-04T17:32:08-04:00 INFO EventLog[Security] Stop processing.
2016-08-04T17:32:08-04:00 DBG WinEventLog[Security] Closing handle
2016-08-04T17:32:09-04:00 INFO EventLog[Key Management Service] Stop processing.
2016-08-04T17:32:09-04:00 DBG WinEventLog[Key Management Service] Closing handle
2016-08-04T17:32:09-04:00 DBG Checkpoint saved to disk. numUpdates=3
2016-08-04T17:32:09-04:00 INFO winlogbeat cleanup
2016-08-04T17:32:09-04:00 INFO Dumping runtime metrics...
2016-08-04T17:32:09-04:00 INFO cmdline=["C:\winlogbeat5\\winlogbeat.exe","-c","C:\winlogbeat5\\winlogbeat.yml","-path.home","C:\winlogbeat5","-path.data","C:\\ProgramData\\winlogbeat"]
2016-08-04T17:32:09-04:00 INFO dropReasons={}
2016-08-04T17:32:09-04:00 INFO ignoredEvents={"Active Directory": 0, "Application": 0, "DFS Replication": 0, "DNS Server": 0, "Directory Service": 0, "File Replication Service": 0, "Key Management Service": 0, "Security": 0, "System": 0, "total": 0}
2016-08-04T17:32:09-04:00 INFO libbeatEsPublishEventsCallCou

16

Based on those logs, it appears to be successfully publishing events to the output. The end of the log that provides metrics about each output is cut off. There should be a metric tell you how many events were ACK'ed by Logstash.

Next step is to check Logstash's log files. You could also start Logstash in the foreground and run it with the debug flag to get some addition information logged to stdout.

17

This topic was automatically closed after 21 days. New replies are no longer allowed.