Logstash does not forward logs

Hi all.
I'm facing a strange issue with logstash 7.2.0.
I have two winlogbeats with a similar config sending logs to logstash (elk-1). Logstash then sends the logs to elasticsearch node (elk-2).
I can confirm that logs from both winlogbeat hosts ("wef1" and "wef2") are reaching logstash because I have setup a file output and I do see logs from both hosts in that file. I differentiate events from these two hosts by the field "hostname" which I see in the output file.
That is I get either events with "hostname":"wef1" and "hostname":"wef2".

Next I have an output config to send the logs from logstash to elasticsearch. However when I perform search across the index in Kibana I see only events from host "wef1".

"elk-1" is the logstash node. "elk-2" is the elasticsearch node. They stand on the same network so there is no firewalling in between.

Here is my logstash config:

input {
  beats {
    port => 5045
    id => "winlogbeat-wef"
    tags => ["winlogbeat-wef"]

output {
  if "Winlogbeat-Security" in [tags] {
    elasticsearch {
      hosts => ["elk-2:9200"]
      manage_template => false
      index => "%{[@metadata][beat]}-security-%{[@metadata][version]}-%{+YYYY.MM.dd}"}
    file {
      path => "/tmp/win_wef.log"

Here is the beats config on both nodes:

- name: ForwardedSecurityLog
    forwarded: true
    tags: ["Winlogbeat-Security"]
    ignore_older: 24h
  hosts: ["elk-1:5045"]

Please advise how to debug the issue.
I've turned on debug log in logstash and elasticsearch but have not discovered anything specific.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.