# Filebeat unable to ship non modified files to logstash

**URL:** <https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 6, 2019, 7:59am UTC](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955 "2019-08-06T07:59:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 6, 2019, 7:59am UTC](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955/1 "2019-08-06T07:59:46Z")

</div>

Hello,

I configured **filebeat.yml** to send data to logstash, I just specified where my data are stored and the ip address of my logstash.

I have a python script to generate data, if I don't change any line in my data, filebeat don't send them to logstash. And after a modification filebeat succeed to send data to logstash.

I would like send these files to logstash without being constrained to modify them by hand each time.

Thank you for your help.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [August 6, 2019, 11:59am UTC](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955/2 "2019-08-06T11:59:58Z")

</div>

Please share the debug logs of Filebeat (output of `./filebeat -e -d "*"`).

---

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 6, 2019, 12:16pm UTC](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955/3 "2019-08-06T12:16:02Z")

</div>

> [@kvch](#):
>
> `./filebeat -e -d "*"`

```
2019-08-06T14:11:00.693+0200 DEBUG [acker] beater/acker.go:64 stateful ack {"count": 1}
2019-08-06T14:11:00.713+0200 DEBUG [registrar] registrar/registrar.go:404 Registry file updated. 172 states written.
2019-08-06T14:11:00.713+0200 DEBUG [registrar] registrar/registrar.go:356 Processing 1 events
2019-08-06T14:11:00.713+0200 DEBUG [registrar] registrar/registrar.go:326 Registrar state updates processed. Count: 1
2019-08-06T14:11:00.713+0200 DEBUG [registrar] registrar/registrar.go:346 Registrar states cleaned up. Before: 172, After: 172, Pending: 0
2019-08-06T14:11:00.713+0200 DEBUG [registrar] registrar/registrar.go:411 Write registry file: /usr/share/filebeat/bin/data/registry/filebeat/data.json (172)
2019-08-06T14:11:00.713+0200 DEBUG [acker] beater/acker.go:64 stateful ack {"count": 1}
2019-08-06T14:11:00.713+0200 DEBUG [publisher] pipeline/client.go:200 Pipeline client receives callback 'onFilteredOut' for event: %+v{0001-01-01 00:00:00 +0000 UTC null null {698022-65025 true <nil> /home/test/Documents/PRODUCERS/Data/HC/54563ec9-d582-4641-8b14-6edafae9c65e.json 0 2019-08-06 14:08:59.312651456 +0200 CEST -1ns log map[] 698022-65025} false}
2019-08-06T14:11:00.713+0200 DEBUG [input] file/states.go:68 New state added for /home/test/Documents/PRODUCERS/Data/HC/616fc149-9cae-4932-9691-e0b7ec2c521c.json
2019-08-06T14:11:00.733+0200 DEBUG [registrar] registrar/registrar.go:404 Registry file updated. 172 states written.
2019-08-06T14:11:00.733+0200 DEBUG [registrar] registrar/registrar.go:356 Processing 1 events
2019-08-06T14:11:00.733+0200 DEBUG [registrar] registrar/registrar.go:326 Registrar state updates processed. Count: 1
2019-08-06T14:11:00.733+0200 DEBUG [registrar] registrar/registrar.go:346 Registrar states cleaned up. Before: 172, After: 172, Pending: 0
2019-08-06T14:11:00.733+0200 DEBUG [registrar] registrar/registrar.go:411 Write registry file: /usr/share/filebeat/bin/data/registry/filebeat/data.json (172)
2019-08-06T14:11:00.733+0200 DEBUG [publisher] pipeline/client.go:200 Pipeline client receives callback 'onFilteredOut' for event: %+v{0001-01-01 00:00:00 +0000 UTC null null {697952-65025 true <nil> /home/test/Documents/PRODUCERS/Data/HC/616fc149-9cae-4932-9691-e0b7ec2c521c.json 0 2019-08-06 14:08:59.365041842 +0200 CEST -1ns log map[] 697952-65025} false}
2019-08-06T14:11:00.733+0200 DEBUG [input] file/states.go:68 New state added for /home/test/Documents/PRODUCERS/Data/HC/6aba6cf8-377c-422a-8ca7-91e2b9995d87.json
2019-08-06T14:11:00.733+0200 DEBUG [acker] beater/acker.go:64 stateful ack {"count": 1}
2019-08-06T14:11:00.757+0200 DEBUG [registrar] registrar/registrar.go:404 Registry file updated. 172 states written.
2019-08-06T14:11:00.757+0200 DEBUG [registrar] registrar/registrar.go:356 Processing 1 events
2019-08-06T14:11:00.758+0200 DEBUG [registrar] registrar/registrar.go:326 Registrar state updates processed. Count: 1
2019-08-06T14:11:00.758+0200 DEBUG [registrar] registrar/registrar.go:346 Registrar states cleaned up. Before: 172, After: 172, Pending: 0
2019-08-06T14:11:00.758+0200 DEBUG [registrar] registrar/registrar.go:411 Write registry file: /usr/share/filebeat/bin/data/registry/filebeat/data.json (172)
2019-08-06T14:11:00.758+0200 DEBUG [publisher] pipeline/client.go:200 Pipeline client receives callback 'onFilteredOut' for event: %+v{0001-01-01 00:00:00 +0000 UTC null null {697970-65025 true <nil> /home/test/Documents/PRODUCERS/Data/HC/6aba6cf8-377c-422a-8ca7-91e2b9995d87.json 0 2019-08-06 14:08:59.387379355 +0200 CEST -1ns log map[] 697970-65025} false}
2019-08-06T14:11:00.758+0200 DEBUG [input] file/states.go:68 New state added for /home/test/Documents/PRODUCERS/Data/HC/7c80e9f7-f1a0-4485-90db-65dc27eb0d65.json

```

---

<div class="post-metadata">

**Author:** ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)\
**Post date:** [August 7, 2019, 8:19am UTC](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955/4 "2019-08-07T08:19:21Z")

</div>

hello, you want me to provide more log ?

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [August 9, 2019, 3:30pm UTC](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955/5 "2019-08-09T15:30:17Z")

</div>

Does your script add a newline to the end of log lines it generates? Filebeat needs a newline at the end of a line in order to detect a finished log line.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 6, 2019, 3:30pm UTC](https://discuss.elastic.co/t/filebeat-unable-to-ship-non-modified-files-to-logstash/193955/6 "2019-09-06T15:30:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.