Closing because close_inactive of 5m0s reached

Elastic Stack Beats
1

Hi,

I do see a lot of posts on this, I tried everything but I am not able to get this working. Can you please help?

This is my filebeat log:

2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/staff_secure_access.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/staff_secure_error.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/www_error.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/idpdev-secure-access.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/wwwdev-secure-error.log
2018-01-04T12:29:54-05:00 INFO Harvester started for file: /logs/wwwdev-secure-access.log
2018-01-04T12:30:24-05:00 INFO Non-zero metrics in the last 30s: filebeat.harvester.open_files=12 filebeat.harvester.running=12 filebeat.harvester.started=12 libbeat.logstash.call_count.PublishEvents=24 libbeat.logstash.publish.read_bytes=2606 libbeat.logstash.publish.write_bytes=860507 libbeat.logstash.published_and_acked_events=23540 libbeat.publisher.published_events=24564 publish.events=22528 registrar.states.current=12 registrar.states.update=22528 registrar.writes=12
2018-01-04T12:30:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=62 libbeat.logstash.publish.read_bytes=2170 libbeat.logstash.publish.write_bytes=2117892 libbeat.logstash.published_and_acked_events=63488 libbeat.publisher.published_events=63488 publish.events=63488 registrar.states.update=63488 registrar.writes=31
2018-01-04T12:31:24-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=37 libbeat.logstash.publish.read_bytes=1295 libbeat.logstash.publish.write_bytes=1233413 libbeat.logstash.published_and_acked_events=37888 libbeat.publisher.published_events=38912 publish.events=38912 registrar.states.update=38912 registrar.writes=19
2018-01-04T12:31:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=10 libbeat.logstash.publish.read_bytes=350 libbeat.logstash.publish.write_bytes=342363 libbeat.logstash.published_and_acked_events=10240 libbeat.publisher.published_events=10240 publish.events=10240 registrar.states.update=10240 registrar.writes=5
2018-01-04T12:32:24-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=26 libbeat.logstash.publish.read_bytes=910 libbeat.logstash.publish.write_bytes=860988 libbeat.logstash.published_and_acked_events=26624 libbeat.publisher.published_events=26624 publish.events=26624 registrar.states.update=26624 registrar.writes=13
2018-01-04T12:32:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=64 libbeat.logstash.publish.read_bytes=2240 libbeat.logstash.publish.write_bytes=1987332 libbeat.logstash.published_and_acked_events=65536 libbeat.publisher.published_events=65536 publish.events=65536 registrar.states.update=65536 registrar.writes=32
2018-01-04T12:33:24-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=69 libbeat.logstash.publish.read_bytes=2415 libbeat.logstash.publish.write_bytes=1971220 libbeat.logstash.published_and_acked_events=70656 libbeat.publisher.published_events=69632 publish.events=69632 registrar.states.update=69632 registrar.writes=34
2018-01-04T12:33:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=22 libbeat.logstash.publish.read_bytes=770 libbeat.logstash.publish.write_bytes=485487 libbeat.logstash.published_and_acked_events=22528 libbeat.publisher.published_events=22528 publish.events=22528 registrar.states.update=22528 registrar.writes=11
2018-01-04T12:34:24-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=25 libbeat.logstash.publish.read_bytes=875 libbeat.logstash.publish.write_bytes=553281 libbeat.logstash.published_and_acked_events=25600 libbeat.publisher.published_events=26624 publish.events=26624 registrar.states.update=26624 registrar.writes=13
2018-01-04T12:34:54-05:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.call_count.PublishEvents=10 libbeat.logstash.publish.read_bytes=385 libbeat.logstash.publish.write_bytes=211627 libbeat.logstash.published_and_acked_events=10357 libbeat.publisher.published_events=8309 publish.events=10357 registrar.states.update=10357 registrar.writes=6
2018-01-04T12:34:59-05:00 INFO File is inactive: /logs/idpdev_error.log. Closing because close_inactive of 5m0s reached.
2018-01-04T12:34:59-05:00 INFO File is inactive: /logs/idpdev-secure-error.log. Closing because close_inactive of 5m0s reached.
2018-01-04T12:34:59-05:00 INFO File is inactive: /logs/staff_secure_error.log. Closing because close_inactive of 5m0s reached.
2018-01-04T12:34:59-05:00 INFO File is inactive: /logs/idpdev_access.log. Closing because close_inactive of 5m0s reached.

Logs on my logstash is not showing any movement. Thank you.
My filebeat.yml file:

filebeat.prospectors:
- input_type: log
  paths:
    - /logs/*.log
  harvester_limit: 12

output:
  logstash:
    hosts: ["logstaship:5443"]
    bulk_max_size: 1024
    ssl:
      certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

filbeat conf file on my logstash server:

input {
   beats {
       port => 5443
       ssl => true
       ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
       ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
    }
}
1 Like
2

Why is timing out after 5ms? Can I increase it. Thanks.

3

You can increase close_timeout by setting it in your prospector.

E.g. you can increase it to 10 minutes instead of the default 5 minutes.

filebeat.prospectors:
 - input_type: log
   close_inactive: 10m

See more on this option: https://www.elastic.co/guide/en/beats/filebeat/master/configuration-filebeat-options.html#close-inactive

2 Likes
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.