# Fresh installation apparently imports logs for few seconds and then stops

**URL:** <https://discuss.elastic.co/t/fresh-installation-apparently-imports-logs-for-few-seconds-and-then-stops/158877>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 30, 2018, 9:47am UTC](https://discuss.elastic.co/t/fresh-installation-apparently-imports-logs-for-few-seconds-and-then-stops/158877 "2018-11-30T09:47:10Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pastrufazio](https://avatars.discourse-cdn.com/v4/letter/p/f9ae1b/32.png) [@Pastrufazio](https://discuss.elastic.co/u/Pastrufazio)\
**Post date:** [November 30, 2018, 9:47am UTC](https://discuss.elastic.co/t/fresh-installation-apparently-imports-logs-for-few-seconds-and-then-stops/158877/1 "2018-11-30T09:47:10Z")

</div>

Hi all,

I installed this new environment:

**SERVER (loststash+elasticsearch+kibana)**

- logstash 6.5.1
- elasticsearch 6.5.1
- kibana 6.5.1

**REMOTE MACHINE (generating logs)**

- filebeat 5.0.1

I passed to filebeat our old logs, restarted the service and this is the result:  
2018-11-30T09:53:53+01:00 DBG Try to publish 835 events to logstash with window size 608  
2018-11-30T09:53:54+01:00 DBG update max ok window size: 405 \< 608  
2018-11-30T09:53:54+01:00 DBG increase window size to: 912  
2018-11-30T09:53:54+01:00 DBG set to batchSize: 835  
2018-11-30T09:53:54+01:00 DBG 608 events out of 835 events sent to logstash. Continue sending  
2018-11-30T09:53:54+01:00 DBG Try to publish 227 events to logstash with window size 835  
2018-11-30T09:53:54+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:53:54+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:53:54+01:00 DBG 227 events out of 227 events sent to logstash. Continue sending  
2018-11-30T09:53:54+01:00 DBG send completed  
2018-11-30T09:53:58+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:53:58+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:54:06+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:54:06+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:54:16+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:54:16+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:54:21+01:00 INFO Non-zero metrics in the last 30s: libbeat.logstash.publish.write\_bytes=84278 filebeat.harvester.started=82 filebeat.harvester.running=82 libbeat.logstash.published\_and\_acked\_events=2026 libbeat.logstash.publish.read\_bytes=72 libbeat.publisher.published\_events=2026 filebeat.harvester.open\_files=82 libbeat.logstash.call\_count.PublishEvents=1  
2018-11-30T09:54:26+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:54:26+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:54:36+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:54:36+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:54:46+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:54:46+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:54:51+01:00 INFO No non-zero metrics in the last 30s  
2018-11-30T09:54:56+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:54:56+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:55:06+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:55:06+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:55:16+01:00 DBG End of file reached: /var/log/app.20181129221005.log; Backoff now.  
2018-11-30T09:55:16+01:00 DBG End of file reached: /var/log/app.20180418221219.log; Backoff now.  
2018-11-30T09:55:21+01:00 INFO No non-zero metrics in the last 30s

Kibana is indexing but only for few minus:

[[Album] Imgur: The magic of the Internet ![](https://i.imgur.com/iqgFR3P.jpg?fb "            
    

    Imgur: The magic of the Internet") ](https://imgur.com/a/n4OrwzU)

How can i let it process all my logs?

This is my filebeat config file:

```
filebeat:

  prospectors:

    -
      paths:
        - /var/log/app.*.log

      ignore_older: 0

output:
  logstash:
    hosts: ["XXX:XXX:XXX:XXX:5044"]
  console: 
    pretty: true

logging:

  to_syslog: true
  to_files: true

  files:

    path: /var/log/foo/filebeat-log
    name: filebeat.log
    rotateeverybytes: 10485760    
    keepfiles: 7

  level: debug
```

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [December 3, 2018, 10:24pm UTC](https://discuss.elastic.co/t/fresh-installation-apparently-imports-logs-for-few-seconds-and-then-stops/158877/2 "2018-12-03T22:24:44Z")

</div>

According to logs filebeat did reach the end of file. Is this all logs?

Why such an old filebeat? I think 5.0 is EOL for quite some time.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 31, 2018, 10:24pm UTC](https://discuss.elastic.co/t/fresh-installation-apparently-imports-logs-for-few-seconds-and-then-stops/158877/3 "2018-12-31T22:24:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.