Export CSV limited to 500 records

[russellpetrichsophos]

Hello Kibana Team,

I'm using Kibana 6.3.2 and I'm trying to export the results of a saved search with over 200k hits.

However, when I click "Export" only the first 500 hits are included in the exported CSV:Kibana_ExportCsvLimitation|690x284

How do I export all 200k+ results to the exported CSV?

-Russell

[mikecote (Mike)]

Hi @russellpetrichsophos,

Is your exported file size around 10MB? If so, I think the export has reached a file size limit that defaults to 10MB. There would be server logs about max Size Reached.

You can change this limit by changing the xpack.reporting.csv.maxSizeBytes setting in Kibana and restarting the server. That limit exists to prevent large exports from causing performance and storage issues.

Mike.

[russellpetrichsophos]

Hey @mikecote,

Thanks for responding. No it's only 562kb - not even close to 10 MB.

-Russell

P.S. I didn't receive an email notifying me of your reply to this post. I wouldn't have noticed the update if I hadn't logged back into my Elastic account to check on this. I'm "watching" this topic.

[tsullivan (Tim Sullivan)]

Hi Russel, the problem might be that you aren't using the official Kibana distributable from Elastic. I pulled down version 6.3.2 of the Elastic stack from Docker and CSV Export from Discover. This is what you would see if you had the official code:

I believe the reason you are not seeing the Reporting tab and have Export instead is because you cluster is provided by Logz.io and they can not lease the Elastic commercial features (X-Pack) to you per license restrictions.

Kibana Reporting with X-Pack does scan & scroll search to the Elasticsearch back-end, and concatenates the results into a CSV. As @mikecote mentioned, that potentially creates very large CSV files.

[russellpetrichsophos]

Hello @tsullivan,

Thank you for your response. I'll take this information back to my team to discuss options.

Thanks,

-Russell