[Filebeat] Newly added lines don't send to a server

Elastic Stack Beats
1

Hi there,
newly added lines to a log don't send to a server.

If I add a new log file to the Filebeat config and restart then.
A Filebeat sends to a server all content of log file, but if then a new line added to this file, nothing happens.

Thanks for any info!

Some more info below

System:
FreeBSD 11.0 amd64

Filesystem:
ufs (atime, ctime, mtime)

Filebeat:
latest, but I had also tried to use v6.6.0

Filebeat log:
ERROR [publisher] pipeline/client.go:90 Failed to publish event: unimplemented

Follow the error we get this code on Github:

Logstash log without any output.

Filebeat config:
https://pastebin.com/Mb5F7vmk

2

Do you have some sample log with messages that trigger this error? Looks like a bug. Never seen this before and we need to debug this.

3

Thanks for the reply!
It is my mistake and I have fixed it.

For history:

[2019-05-14T16:28:59,965][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"access_log-prod-2019.05.14", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x1852523>], :response=>{"index"=>{"_index"=>"access_log-prod-2019.05.14", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}

Be warning! It is not a fully correct way

More in the docs https://fossies.org/linux/elasticsearch/docs/reference/modules.asciidoc

Fix:

curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'{ "transient": { "cluster.max_shards_per_node": 1500 }}'

Thanks!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.