Max CSV export elastic cloud 50MB, how to increase it

Elastic Stack Kibana
1

Hello, I need to generate large CSV reports so I must set 'xpack.reporting.csv.maxSizeBytes' to a high value but I'm being capped at 50MB, how can I go beyond that?

The error when trying to save the changes is:

Kibana - 'xpack.reporting.csv.maxSizeBytes': has to be a number between [0, 52428800] inclusive

Thanks

2

This could be a restriction that isn't necessarily needed in newer versions, or just a bug in the application. You can notify the support team of this issue at https://cloud.elastic.co/support

3

Thank you @tsullivan I think the same after the change with the chunked requests. Also, Elasticsearch body size can not be changed in elastic cloud. Do you think the default value would accept a high csv export size?

4

Which version are you using? The dev team worked on an optimization change to save CSV export data into Elasticsearch by partitioning chunks of the CSV data across multiple ES documents in v7.15. Is that what you are referring to with "the change with the chunked requests" ?

In that optimization, Kibana detects the max body size that ES will accept, and uses that as the size for each chunk in the partitioned data.

5

Yes, I read that PR and was very excited about that change. We are using Elastic Cloud 8.1 and just could bump from 10MB to 50MB :frowning:

6

Sorry to hear that. I do just want to emphasize that the only way to see improvement in this area will be to file an issue in https://cloud.elastic.co/support.

The docs on User Settings explain there is a maximum to this value: Edit Kibana user settings | Elasticsearch Service Documentation | Elastic. Let's consider that a bug as of 7.15.

7

Just curious, what value do you need it set to?

8

The previous platform allowed CSV dumps for 1mm rows - 2.5gb , so we want to get as close as possible to that number.

Do you think that cap will be removed soon? (this year?)

1 Like
9

We made a custom request and the limit was increased by Elastic Cloud Support. We started at 500MB and we could get 200K rows report in 4min so it is a good start. Kibana node has 4GB. Indeed the cap removal was approved by you guys, so hopefully we have it removed in future versions.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.