# ERROR Failed to publish events: temporary bulk send failure

**URL:** <https://discuss.elastic.co/t/error-failed-to-publish-events-temporary-bulk-send-failure/235175>\
**Category:** APM\
**Tags:** nodejs, server, php\
**Created:** [June 1, 2020, 1:38pm UTC](https://discuss.elastic.co/t/error-failed-to-publish-events-temporary-bulk-send-failure/235175 "2020-06-01T13:38:09Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![afonsoavr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/afonsoavr/32/65059_2.png) [@afonsoavr](https://discuss.elastic.co/u/afonsoavr)\
**Post date:** [June 1, 2020, 1:38pm UTC](https://discuss.elastic.co/t/error-failed-to-publish-events-temporary-bulk-send-failure/235175/1 "2020-06-01T13:38:09Z")

</div>

I create the my apm server in ECS on AWS.  
This APM Server is with 256 CPU and 1024 MiB per task with 5 tasks.

And eventualy I receive this errors:

```
2020-05-27T01:38:37.929Z ERROR elasticsearch/client.go:333 Failed to perform any bulk index operations: Post http://elastic.local:9200/_bulk: EOF

2020-05-27T01:38:39.183Z ERROR pipeline/output.go:121 Failed to publish events: Post http://elastic.local:9200/_bulk: EOF

2020-05-27T01:38:41.558Z ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure

```

or only this log:

```
ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure

```

**Kibana version** : 6.8.1

**Elasticsearch version** : 6.8.1

**APM Server version** : 6.8.1 (6 tasks in AWS ECS with loadbalancer)

**APM Agent language and version** :  
nodejs -\> 3.5.0  
nodejs -\> 3.6.1  
elasticapm-php -\> 7.0.0-rc2

**Original install method (e.g. download page, yum, deb, from source, etc.) and version**: The install with:

```
curl -L -O https://artifacts.elastic.co/downloads/apm-server/apm-server-6.8.1-x86_64.rpm && \
    rpm -vi apm-server-6.8.1-x86_64.rpm

```

**Is there anything special in your setup?**

In my file apm-server.yml has the customer parameters:  
BULK\_MAX\_SIZE -\> 5120  
WORKERS -\> 35  
queue\_mem\_events -\> workers \* bulk\_max\_size (179200)

**observation elk config:**  
My Elasticsearch is 9 nodes with 3 Master Ingestion and 6 Data Nodes.

---

<div class="post-metadata">

**Author:** ![gil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gil/32/41911_2.png) [@gil](https://discuss.elastic.co/u/gil)\
**Post date:** [June 1, 2020, 4:38pm UTC](https://discuss.elastic.co/t/error-failed-to-publish-events-temporary-bulk-send-failure/235175/2 "2020-06-01T16:38:43Z")

</div>

Welcome @afonsoavr! I don't see anything obvious given the information you've provided. Is there anything logged by elasticsearch at this time? Have you tried enabling [debug logging](https://www.elastic.co/guide/en/apm/server/6.8/enable-apm-server-debugging.html) in apm-server?

Also, have any events been ingested? Does the same occur when the PHP agent is not in use?

---

<div class="post-metadata">

**Author:** ![afonsoavr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/afonsoavr/32/65059_2.png) [@afonsoavr](https://discuss.elastic.co/u/afonsoavr)\
**Post date:** [June 2, 2020, 11:19am UTC](https://discuss.elastic.co/t/error-failed-to-publish-events-temporary-bulk-send-failure/235175/3 "2020-06-02T11:19:34Z")

</div>

Hi @gil,  
Thank's for your answer.

My config for APM Server is "logging.level: warning" and "logging.to\_stderr: true".  
And trying differents configurations for combination BULK\_MAX\_SIZE and WORKERS.

And, I increase the configuration the 256 CPU for 512 CPU because in 256 CPU the logging increase errors in other moments.

About the configurations CPU, 256 ou 512 is a 0.25 Core and 0.5 Core.

In my last log is: _ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure_

In my case, dont has option the not run applications with PHP.  
This error is normally associated with PHP agent?

---

<div class="post-metadata">

**Author:** ![gil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gil/32/41911_2.png) [@gil](https://discuss.elastic.co/u/gil)\
**Post date:** [June 2, 2020, 6:59pm UTC](https://discuss.elastic.co/t/error-failed-to-publish-events-temporary-bulk-send-failure/235175/4 "2020-06-02T18:59:32Z")

</div>

> [@afonsoavr](#):
>
> This error is normally associated with PHP agent?

I had asked about the PHP agent because the one you're using is not officially supported and wondered if it may submit data that is accepted by APM Server but not by Elasticsearch - that shouldn't happen but a lot work has been done to improve intake validation since 6.8.1. I would expect this information would be exposed by the Elasticsearch logs, are you able to access those?

> [@afonsoavr](#):
>
> My config for APM Server is "logging.level: warning" and "logging.to\_stderr: true".

You may try `logging.level: debug` if the Elasticsearch logs don't provide enough insight.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 23, 2020, 2:59pm UTC](https://discuss.elastic.co/t/error-failed-to-publish-events-temporary-bulk-send-failure/235175/5 "2020-06-23T14:59:42Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
