Hi,
I do see a lot of posts on this, I tried everything but I am not able to get this working. Can you please help?
This is my filebeat log:
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/staff_secure_access.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/staff_secure_error.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/www_error.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/idpdev-secure-access.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/wwwdev-secure-error.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/wwwdev-secure-access.log
2018-01-04T12:30:24-05:00 INFO Non-zero metrics in the last 30s: filebeat.harvester.open_files=12 filebeat.harvester.running=12 filebeat.harvester.started=12 libbeat.logstash.call_count.PublishEvents=24 libbeat.logstash.publish.read_bytes=2606 libbeat.logstash.publish.write_bytes=860507 libbeat.logstash.published_and_acked_events=23540 libbeat.publisher.published_events=24564 publish.events=22528 registrar.states.current=12 registrar.states.update=22528 registrar.writes=12
2018-01-04T12:30:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=62 libbeat.logstash.publish.read_bytes=2170 libbeat.logstash.publish.write_bytes=2117892 libbeat.logstash.published_and_acked_events=63488 libbeat.publisher.published_events=63488 publish.events=63488 registrar.states.update=63488 registrar.writes=31
2018-01-04T12:31:24-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=37 libbeat.logstash.publish.read_bytes=1295 libbeat.logstash.publish.write_bytes=1233413 libbeat.logstash.published_and_acked_events=37888 libbeat.publisher.published_events=38912 publish.events=38912 registrar.states.update=38912 registrar.writes=19
2018-01-04T12:31:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=10 libbeat.logstash.publish.read_bytes=350 libbeat.logstash.publish.write_bytes=342363 libbeat.logstash.published_and_acked_events=10240 libbeat.publisher.published_events=10240 publish.events=10240 registrar.states.update=10240 registrar.writes=5
2018-01-04T12:32:24-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=26 libbeat.logstash.publish.read_bytes=910 libbeat.logstash.publish.write_bytes=860988 libbeat.logstash.published_and_acked_events=26624 libbeat.publisher.published_events=26624 publish.events=26624 registrar.states.update=26624 registrar.writes=13
2018-01-04T12:32:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=64 libbeat.logstash.publish.read_bytes=2240 libbeat.logstash.publish.write_bytes=1987332 libbeat.logstash.published_and_acked_events=65536 libbeat.publisher.published_events=65536 publish.events=65536 registrar.states.update=65536 registrar.writes=32
2018-01-04T12:33:24-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=69 libbeat.logstash.publish.read_bytes=2415 libbeat.logstash.publish.write_bytes=1971220 libbeat.logstash.published_and_acked_events=70656 libbeat.publisher.published_events=69632 publish.events=69632 registrar.states.update=69632 registrar.writes=34
2018-01-04T12:33:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=22 libbeat.logstash.publish.read_bytes=770 libbeat.logstash.publish.write_bytes=485487 libbeat.logstash.published_and_acked_events=22528 libbeat.publisher.published_events=22528 publish.events=22528 registrar.states.update=22528 registrar.writes=11
2018-01-04T12:34:24-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=25 libbeat.logstash.publish.read_bytes=875 libbeat.logstash.publish.write_bytes=553281 libbeat.logstash.published_and_acked_events=25600 libbeat.publisher.published_events=26624 publish.events=26624 registrar.states.update=26624 registrar.writes=13
2018-01-04T12:34:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=10 libbeat.logstash.publish.read_bytes=385 libbeat.logstash.publish.write_bytes=211627 libbeat.logstash.published_and_acked_events=10357 libbeat.publisher.published_events=8309 publish.events=10357 registrar.states.update=10357 registrar.writes=6
2018-01-04T12:34:59-05:00 INFO File is inactive: /logs/idpdev_error.log. Closing because close_inactive of 5m0s reached.
2018-01-04T12:34:59-05:00 INFO File is inactive: /logs/idpdev-secure-error.log. Closing because close_inactive of 5m0s reached.
2018-01-04T12:34:59-05:00 INFO File is inactive: /logs/staff_secure_error.log. Closing because close_inactive of 5m0s reached.
2018-01-04T12:34:59-05:00 INFO File is inactive: /logs/idpdev_access.log. Closing because close_inactive of 5m0s reached.
Logs on my logstash is not showing any movement. Thank you.
My filebeat.yml file:
filebeat.prospectors:
- input_type: log
paths:
- /logs/*.log
harvester_limit: 12
output:
logstash:
hosts: ["logstaship:5443"]
bulk_max_size: 1024
ssl:
certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
filbeat conf file on my logstash server:
input {
beats {
port => 5443
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}