# Csv file export with 1 million records(approx 1 gb size) in kibana reporting

**URL:** https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892
**Category:** Kibana
**Created:** [March 10, 2020, 9:17am UTC](https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892 "2020-03-10T09:17:34Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![sarvendras](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarvendras/32/46287_2.png) [@sarvendras](https://discuss.elastic.co/u/sarvendras)
#### Post date: [March 10, 2020, 9:17am UTC](https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892/1 "2020-03-10T09:17:34Z")

</div>

Hi All,  
I am trying to export csv file with 1 million records(approx 1 gb size) from kibana reporting.  
I had tried different combinations of below configs but getting different errors.Also increased kibana heap size to 4 gb.  
Is it practially possible to export such large file

kibana.yml :  
xpack.monitoring.enabled: true  
server.maxPayloadBytes: 10485760  
xpack.reporting.queue.timeout: 3600000  
xpack.reporting.csv.maxSizeBytes: 104857600

Error recieved in Kibana reporting tab:  
Unable to generate report  
Max attempts reached (3)

Unable to generate report  
Request Entity Too Large

---

<div class="post-metadata">

### Author: ![rashmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rashmi/32/16391_2.png) [@rashmi](https://discuss.elastic.co/u/rashmi)
#### Post date: [March 11, 2020, 5:54am UTC](https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892/2 "2020-03-11T05:54:44Z")

</div>

@joelgriffith is this something which you can shed some light on ? Only thing which came to my mind was to increase the limit with the `server.maxPayloadBytes` setting, but are there more settings to try ?

Also the Kibana logs, screenshots would help  
Thanks  
Rashmi

---

<div class="post-metadata">

### Author: ![sarvendras](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarvendras/32/46287_2.png) [@sarvendras](https://discuss.elastic.co/u/sarvendras)
#### Post date: [March 13, 2020, 5:18am UTC](https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892/3 "2020-03-13T05:18:10Z")

</div>

@joelgriffith and ELK team,

Please suggest on this...

---

<div class="post-metadata">

### Author: ![joelgriffith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joelgriffith/32/53738_2.png) [@joelgriffith](https://discuss.elastic.co/u/joelgriffith)
#### Post date: [March 13, 2020, 6:41pm UTC](https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892/4 "2020-03-13T18:41:28Z")

</div>

Yes: you'll need to update your ElasticSearch cluster to support the large file-size: `http.max_content_length` should be set to the same size as Reportings size ([https://www.elastic.co/guide/en/kibana/6.8/reporting-settings-kb.html#reporting-csv-settings](https://www.elastic.co/guide/en/kibana/6.8/reporting-settings-kb.html#reporting-csv-settings)).

I'll note that a 1GB/1M records is outside the bounds of a normal CSV export, and I'm not sure how ElasticSearch/Kibana will handle that since it's a large file-size.

---

<div class="post-metadata">

### Author: ![ElasticLearner](https://avatars.discourse-cdn.com/v4/letter/e/977dab/32.png) [@ElasticLearner](https://discuss.elastic.co/u/ElasticLearner)
#### Post date: [March 13, 2020, 7:12pm UTC](https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892/5 "2020-03-13T19:12:01Z")

</div>

Hey Sarvendras,

I had encounter a similar road block and I had figured out a workaround with some help on me end.

If you are utilizing Logstash, you can use the "elasticsearch" input with the "query" option and have the output use the "csv" plugin.

Make sure you have permissions to write where ever you have the path pointed to.

I hope this helps, its not a a real solution, but I believe it'll help you get your reports.

([https://www.elastic.co/guide/en/logstash/6.8/plugins-inputs-elasticsearch.html](https://www.elastic.co/guide/en/logstash/6.8/plugins-inputs-elasticsearch.html))  
([https://www.elastic.co/guide/en/logstash/6.8/plugins-outputs-csv.html](https://www.elastic.co/guide/en/logstash/6.8/plugins-outputs-csv.html))

---

<div class="post-metadata">

### Author: ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [March 13, 2020, 7:21pm UTC](https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892/6 "2020-03-13T19:21:15Z")

</div>

```
input {
   elasticsearch {
      hosts => ["host1:9200", "host2:9200"]
      index => "index-2020"
      user => "elastic"
      password => "xxxx"
   }
}

filter {
mutate { remove_field => ["field1","field2","@timestamp","@version"] }
}

output {
   csv {
      fields => ["myfield1","myfield2","myfield3"]
      csv_options => { "col_sep" => "," }
      path => "/data01/csv_files/test.csv"
    }
  stdout { codec => rubydebug }
}

```

Pretty much same what previous use said

[https://discuss.elastic.co/t/how-to-export-csv-in-kibana-7-5-with-more-then-1-million-row/214526/4](https://discuss.elastic.co/t/how-to-export-csv-in-kibana-7-5-with-more-then-1-million-row/214526/4)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 10, 2020, 7:21pm UTC](https://discuss.elastic.co/t/csv-file-export-with-1-million-records-approx-1-gb-size-in-kibana-reporting/222892/7 "2020-04-10T19:21:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
