Logstash logs does not appears in the index created by Filebeat


(Peter Szemesy) #1

Hi All,

I have installed the filebeat component for log/monitor purposes for Logstash activities as it was described under the Logs menu item at Kibana.
Maybe I have miss-configured something, but only the file opening and file closing activities are recorded in the filebeat-* indices (no pipeline related activities, nor start or stop related ones).

I use the standard filebeat configuration (filebeat modules enable logstash - obviously).
I did not modified anything in fileds.yml, and I have configured the elasticsearch connection and the kibana connection in filebeat .yml only.


(Pier-Hugues Pellerin) #2

Hello @pszemesy , if you look at the Logstash log are these message present? By default the logstash Filebeat module should read anything that is added to the log.


(Peter Szemesy) #3

Hi,
Yes, in the Logstash log everything appears as it should, so what I do not understand why these log entries does not in index.


(Pier-Hugues Pellerin) #4

Can you share your configuration?


(Peter Szemesy) #5

Please find it below (I have deleted the commented lines):
filebeat.yml:

filebeat.inputs:
- type: log
  enabled: false
  paths:
    - /var/log/*.log
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
setup.kibana:
  host: "http://100.78.196.213:5601"
  username: "elastic"
  password: "********"
  protocol: "http"
output.elasticsearch:
  hosts: ["100.78.196.210:9200","100.78.196.211:9200","100.78.196.212:9200"]

  protocol: "http"
  username: "elastic"
  password: "********"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

modules.d/logstash.yml:

  - module: logstash
    log:
     enabled: true
    slowlog:
     enabled: false

The others (fileds.yml and filebeat_reference.yml) are the standards from the installation.


(Pier-Hugues Pellerin) #6

Looking at your configuration everything appears normal on my side, The logstash module, should be take by default log in /var/log/logstash/logstash-plain*.log, Is there any error in the Filebeat log?