# \[Filebeat\] Newly added lines don't send to a server

**URL:** <https://discuss.elastic.co/t/filebeat-newly-added-lines-dont-send-to-a-server/180716>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 12, 2019, 6:41pm UTC](https://discuss.elastic.co/t/filebeat-newly-added-lines-dont-send-to-a-server/180716 "2019-05-12T18:41:00Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![0x00dec0de](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/0x00dec0de/32/46049_2.png) [@0x00dec0de](https://discuss.elastic.co/u/0x00dec0de)\
**Post date:** [May 12, 2019, 6:41pm UTC](https://discuss.elastic.co/t/filebeat-newly-added-lines-dont-send-to-a-server/180716/1 "2019-05-12T18:41:01Z")

</div>

Hi there,  
newly added lines to a log don't send to a server.

If I add a new log file to the Filebeat config and restart then.  
A Filebeat sends to a server all content of log file, but if then a new line added to this file, nothing happens.

Thanks for any info!

Some more info below

System:  
FreeBSD 11.0 amd64

Filesystem:  
ufs (atime, ctime, mtime)

Filebeat:  
latest, but I had also tried to use v6.6.0

Filebeat log:  
ERROR [publisher] pipeline/client.go:90 Failed to publish event: unimplemented

Follow the error we get this code on Github:

> <https://github.com/elastic/beats/blob/master/libbeat/publisher/pipeline/client.go#L88-L90>

Logstash log without any output.

Filebeat config:  
[https://pastebin.com/Mb5F7vmk](https://pastebin.com/Mb5F7vmk)

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [May 14, 2019, 2:07pm UTC](https://discuss.elastic.co/t/filebeat-newly-added-lines-dont-send-to-a-server/180716/2 "2019-05-14T14:07:06Z")

</div>

Do you have some sample log with messages that trigger this error? Looks like a bug. Never seen this before and we need to debug this.

---

<div class="post-metadata">

**Author:** ![0x00dec0de](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/0x00dec0de/32/46049_2.png) [@0x00dec0de](https://discuss.elastic.co/u/0x00dec0de)\
**Post date:** [May 15, 2019, 7:53am UTC](https://discuss.elastic.co/t/filebeat-newly-added-lines-dont-send-to-a-server/180716/3 "2019-05-15T07:53:02Z")

</div>

Thanks for the reply!  
It is my mistake and I have fixed it.

For history:

```auto
[2019-05-14T16:28:59,965][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"access_log-prod-2019.05.14", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x1852523>], :response=>{"index"=>{"_index"=>"access_log-prod-2019.05.14", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}

```

Be warning! It is not a fully correct way

More in the docs [https://fossies.org/linux/elasticsearch/docs/reference/modules.asciidoc](https://fossies.org/linux/elasticsearch/docs/reference/modules.asciidoc)

Fix:

```auto
curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'{ "transient": { "cluster.max_shards_per_node": 1500 }}'

```

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 12, 2019, 7:53am UTC](https://discuss.elastic.co/t/filebeat-newly-added-lines-dont-send-to-a-server/180716/4 "2019-06-12T07:53:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.