# ERR Failed to publish events: temporary bulk send failure **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 12, 2017, 8:05am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-temporary-bulk-send-failure/111265 "2017-12-12T08:05:29Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Pan\_Wang](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pan\_Wang](https://discuss.elastic.co/u/Pan_Wang)\ **Post date:** [December 12, 2017, 8:05am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-temporary-bulk-send-failure/111265/1 "2017-12-12T08:05:29Z")
filebeat version 6.0.1 Elasticsearch version 5.6.2 Filebeat has 40，Elasticsearch has 4. filebeat.yml: filebeat.registry\_file: /home/work/data/registry ``` filebeat.config.prospectors: enabled: true path: configs/*.yml reload.enabled: true reload.period: 30s output.elasticsearch: hosts: ["IP1:9200","IP2:9200","IP3:9200","IP4:9200"] ``` filebeat log ``` 2017-12-12T15:45:03+08:00 ERR Failed to publish events: temporary bulk send failure 2017-12-12T15:45:03+08:00 ERR Failed to publish events: temporary bulk send failure 2017-12-12T15:45:03+08:00 INFO Connected to Elasticsearch version 5.6.2 2017-12-12T15:45:04+08:00 INFO Connected to Elasticsearch version 5.6.2 2017-12-12T15:45:04+08:00 INFO Template already exists and will not be overwritten. 2017-12-12T15:45:04+08:00 INFO Template already exists and will not be overwritten. 2017-12-12T15:45:08+08:00 ERR Failed to publish events: temporary bulk send failure 2017-12-12T15:45:08+08:00 INFO Connected to Elasticsearch version 5.6.2 2017-12-12T15:45:08+08:00 INFO Template already exists and will not be overwritten. 2017-12-12T15:45:09+08:00 ERR Failed to publish events: temporary bulk send failure 2017-12-12T15:45:09+08:00 ERR Failed to publish events: temporary bulk send failure 2017-12-12T15:45:09+08:00 INFO Connected to Elasticsearch version 5.6.2 2017-12-12T15:45:09+08:00 INFO Template already exists and will not be overwritten. 2017-12-12T15:45:09+08:00 ERR Failed to publish events: temporary bulk send failure 2017-12-12T15:45:09+08:00 INFO Connected to Elasticsearch version 5.6.2 2017-12-12T15:45:09+08:00 INFO Template already exists and will not be overwritten. 2017-12-12T15:45:12+08:00 INFO Connected to Elasticsearch version 5.6.2 ``` There is no ERR when I reduce filebeat to 20。 I have modified filebeat.yml: ``` filebeat.registry_file: /home/work/data/registry queue.mem: events: 1000000 flush.timeout: 30s filebeat.config.prospectors: enabled: true path: configs/*.yml reload.enabled: true reload.period: 30s output.elasticsearch: hosts: ["IP1:9200","IP2:9200","IP3:9200","IP4:9200"] bulk_max_size: 100000 ``` But it doesn't work. Anybody encountered this problem? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 12, 2017, 8:07am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-temporary-bulk-send-failure/111265/2 "2017-12-12T08:07:41Z")
> [@Pan\_Wang](#): > > bulk\_max\_size: 100000 Why would you set it this large? Larger size does not necessarily equal improved performance, but will use up more memory. Quoting the docs: > Specifying a larger batch size can improve performance by lowering the overhead of sending events. However big batch sizes can also increase processing times, which might result in API errors, killed connections, timed-out publishing requests, and, ultimately, lower throughput. ---
**Author:** ![Pan\_Wang](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pan\_Wang](https://discuss.elastic.co/u/Pan_Wang)\ **Post date:** [December 12, 2017, 8:25am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-temporary-bulk-send-failure/111265/3 "2017-12-12T08:25:06Z")
I want to reduce connection by increase maximum number of events in a single Elasticsearch bulk API index request. It's just an attempt. ``` ERR Failed to publish events: temporary bulk send failure Connected to Elasticsearch version 5.6.2 ``` What causes this situation? thank you. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 12, 2017, 9:10am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-temporary-bulk-send-failure/111265/4 "2017-12-12T09:10:13Z")
I do not know. Is there anything in the logs? Have you tried with a smaller bulk size, e.g. 1000 documents? ---
**Author:** ![Pan\_Wang](https://avatars.discourse-cdn.com/v4/letter/p/13edae/32.png) [@Pan\_Wang](https://discuss.elastic.co/u/Pan_Wang)\ **Post date:** [December 12, 2017, 9:30am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-temporary-bulk-send-failure/111265/5 "2017-12-12T09:30:02Z")
Yes, I have tried with a smaller bulk size, the other logs no abnormalities. Perhaps because bulk is full. [Elasticsearch](https://www.elastic.co/guide/en/beats/filebeat/master/elasticsearch-output.html) was disconnected. ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/b/3bbf2e538b3dcffb00be4cb9329274d443c2272f.png) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 12, 2017, 9:40am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-temporary-bulk-send-failure/111265/6 "2017-12-12T09:40:04Z")
How much data are you receiving each day? Might it be worthwhile reducing the number of primary shards to [reduce the risk of bulk rejections](https://www.elastic.co/blog/why-am-i-seeing-bulk-rejections-in-my-elasticsearch-cluster)? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 9, 2018, 9:40am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-temporary-bulk-send-failure/111265/7 "2018-01-09T09:40:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.