Downloading large amount of logs as CSV using Kibana/Eland

Elastic Stack Kibana
4

Hi @xynobob,

Sorry to hear that Elastic Support were not of much help. We're happy to do our best efforts here!

There is a 10MB limit by default which can be configured via the xpack.reporting.csv.maxSizeBytes setting in kibana.yml for your cluster as per this thread. You'll see in the docs that we recommend if you need to export more than 250MB you might want to consider exporting in smaller batches, which you could do by splitting your requests across multiple timeframes.

For the first option of Python, are you using the Eland ML client or the Elasticsearch Python client? Your original message mentioned the Eland client, but the thread is referencing the latter.

That aside, having a look at the docs I see a couple of other options you could try:

  1. Export via the Point in time API.
  2. SQL with the CSV response format

Let us know if any of those options, potentially in batches, help you get the data you need.