Logstash logs does not appears in the index created by Filebeat

(Peter Szemesy) #1

Hi All,

I have installed the filebeat component for log/monitor purposes for Logstash activities as it was described under the Logs menu item at Kibana.
Maybe I have miss-configured something, but only the file opening and file closing activities are recorded in the filebeat-* indices (no pipeline related activities, nor start or stop related ones).

I use the standard filebeat configuration (filebeat modules enable logstash - obviously).
I did not modified anything in fileds.yml, and I have configured the elasticsearch connection and the kibana connection in filebeat .yml only.

(Pier-Hugues Pellerin) #2

Hello @pszemesy , if you look at the Logstash log are these message present? By default the logstash Filebeat module should read anything that is added to the log.

(Peter Szemesy) #3

Yes, in the Logstash log everything appears as it should, so what I do not understand why these log entries does not in index.

(Pier-Hugues Pellerin) #4

Can you share your configuration?

(Peter Szemesy) #5

Please find it below (I have deleted the commented lines):

- type: log
  enabled: false
    - /var/log/*.log
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
  index.number_of_shards: 3
  host: ""
  username: "elastic"
  password: "********"
  protocol: "http"
  hosts: ["","",""]

  protocol: "http"
  username: "elastic"
  password: "********"

  - add_host_metadata: ~
  - add_cloud_metadata: ~


  - module: logstash
     enabled: true
     enabled: false

The others (fileds.yml and filebeat_reference.yml) are the standards from the installation.

(Pier-Hugues Pellerin) #6

Looking at your configuration everything appears normal on my side, The logstash module, should be take by default log in /var/log/logstash/logstash-plain*.log, Is there any error in the Filebeat log?