# CSV exporting reports fails almost all the time

**URL:** <https://discuss.elastic.co/t/csv-exporting-reports-fails-almost-all-the-time/289087>\
**Category:** Kibana\
**Tags:** docker\
**Created:** [November 12, 2021, 4:30pm UTC](https://discuss.elastic.co/t/csv-exporting-reports-fails-almost-all-the-time/289087 "2021-11-12T16:30:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mike\_Williams](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_williams/32/81420_2.png) [@Mike\_Williams](https://discuss.elastic.co/u/Mike_Williams)\
**Post date:** [November 12, 2021, 4:30pm UTC](https://discuss.elastic.co/t/csv-exporting-reports-fails-almost-all-the-time/289087/1 "2021-11-12T16:30:51Z")

</div>

In very similar vanes as [7.5 [reporting] fails randomly](https://discuss.elastic.co/t/7-5-reporting-fails-randomly/215160) and [Kibana Reporting -csv download fails frequently with error 'Unable to generate report Max attempts reached (3)'](https://discuss.elastic.co/t/kibana-reporting-csv-download-fails-frequently-with-error-unable-to-generate-report-max-attempts-reached-3/278405) we're currently unable to do CSV exporting most of the time.  
We're on kibana 7.13.4 with Elasticsearch 7.13.3.  
Outside of server.port, server.host and the Elasticsearch connectivity options our entire kibana.yml is;

```auto
monitoring.ui.ccs.enabled: false
xpack.encryptedSavedObjects.encryptionKey: aijeeb8eiYoongo7ooy1uquah3aiz1ha
xpack.reporting.csv.maxSizeBytes: 512mb
xpack.reporting.encryptionKey: aijeeb8eiYoongo7ooy1uquah3aiz1ha
xpack.reporting.queue.timeout: 15m
xpack.security.encryptionKey: aijeeb8eiYoongo7ooy1uquah3aiz1ha

```

We do have kibana load balanced across multiple machines. No docker.  
All have exactly the same configuration.

In the kibana log we get things like this.

```auto
kvtt491u0ekd2fccc5714yfm - _claimPendingJobs encountered a version conflict on updating pending job kvtt5lzq0ekd2fccc5e41dfi: ResponseError: version_conflict_engine_exception
    at onBody (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
    at IncomingMessage.onEnd (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
    at IncomingMessage.emit (events.js:387:35)
    at endReadableNT (internal/streams/readable.js:1317:12)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)

```

sometimes kibana logs a HTTP 409 from Elasticsearch.  
All the kibana instances log exactly the same \_claimPendingJobs error each time a report fails.

On the kibana reporting page we get told

```auto
Error: Failed to decrypt report job data. Please ensure that xpack.reporting.encryptionKey is set and re-generate this report. Error: Unsupported state or unable to authenticate data

```

or

```auto
Max attempts reached (3)

```

I've been ramping `xpack.reporting.queue.timeout` up based on [Kibana Reporting -csv download fails frequently with error 'Unable to generate report Max attempts reached (3)'](https://discuss.elastic.co/t/kibana-reporting-csv-download-fails-frequently-with-error-unable-to-generate-report-max-attempts-reached-3/278405) but the report is failing long before the timeout is exceeded.  
As in all 3 attempts fail within 2 or 3 minutes for large reports, or seconds for tiny reports.

`http.max_content_length` for Elasticsearch is 576mb.

To begin with we didn't have `xpack.reporting.encryptionKey` set. I don't know if that matters. Maybe there is some data structure somewhere that doesn't get updated?

The exports are both large and tiny.  
I've managed to get exports of 3.3mb, 33mb, and one at 65mb. Any longer time frames than those and it fails every time, sometimes the report will fail at the 33mb or 65mb size time frames too.  
I've also managed to have exports containing just 13 rows (of a timestamp and the single character "-" due to a failure on my part) fail, but then immediate afterwards work fine (it was 465 bytes long).

---

<div class="post-metadata">

**Author:** ![tsullivan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsullivan/32/31077_2.png) [@tsullivan](https://discuss.elastic.co/u/tsullivan)\
**Post date:** [December 2, 2021, 7:09pm UTC](https://discuss.elastic.co/t/csv-exporting-reports-fails-almost-all-the-time/289087/2 "2021-12-02T19:09:17Z")

</div>

> [@Mike\_Williams](#):
>
> sometimes kibana logs a HTTP 409 from Elasticsearch.

This warning can be ignored. See: [Reporting troubleshooting | Kibana Guide [8.11] | Elastic](https://www.elastic.co/guide/en/kibana/current/reporting-troubleshooting.html#_statuscodeerror_version_conflict_engine_exception)

> [@Mike\_Williams](#):
>
> ```auto
> Error: Failed to decrypt report job data. Please ensure that xpack.reporting.encryptionKey is set and re-generate this report. Error: Unsupported state or unable to authenticate data
> 
> ```

This means a request to generate a CSV came to one Kibana instance, and the job was claimed by a different instance that has a different encryption key. Find the instance that claimed the job in the Stack Management \> Reporting screen. Each report has an info panel with metadata about the report: it includes the UUID of the Kibana instance that processed the job.

> [@Mike\_Williams](#):
>
> I've managed to get exports of 3.3mb, 33mb, and one at 65mb. Any longer time frames than those and it fails every time, sometimes the report will fail at the 33mb or 65mb size time frames too.

What is the amount of RAM on the Kibana instance that processed the job? It may be too low.

> [@Mike\_Williams](#):
>
> To begin with we didn't have `xpack.reporting.encryptionKey` set. I don't know if that matters. Maybe there is some data structure somewhere that doesn't get updated?

There should have been messages in the server logs stating that a random encryption key was being generated for the instance. Reports that were created with that encryption key can not be recovered since the key was ephemeral.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 30, 2021, 7:09pm UTC](https://discuss.elastic.co/t/csv-exporting-reports-fails-almost-all-the-time/289087/3 "2021-12-30T19:09:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
