# How to speed up indexing of csv file via logstash

**URL:** <https://discuss.elastic.co/t/how-to-speed-up-indexing-of-csv-file-via-logstash/197785>\
**Category:** Logstash\
**Created:** [September 3, 2019, 8:31am UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-of-csv-file-via-logstash/197785 "2019-09-03T08:31:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ludaca](https://avatars.discourse-cdn.com/v4/letter/l/ac8455/32.png) [@ludaca](https://discuss.elastic.co/u/ludaca)\
**Post date:** [September 3, 2019, 8:31am UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-of-csv-file-via-logstash/197785/1 "2019-09-03T08:31:01Z")

</div>

Hello,  
ES 7.3 cluster on k8s, 6 data nodes  
csv file: 4500000 rows, 90 fields  
Indexing takes an hour and a half approximately.  
\</\> logstash conf file:  
input { stdin {  
type =\> "stdin-type"  
}  
}  
filter {  
csv {  
separator =\> ","  
skip\_header =\> "true"  
columns =\> [.....]  
}  
date {  
match =\> ["my\_date", "YYYYMMddHHmmss"]  
target =\> "my\_date"  
}  
mutate {  
convert =\> {  
"Speed\_Min" =\> "integer"  
"Speed\_Max" =\> "integer"  
"Speed\_Avg" =\> "integer"  
}  
}  
}  
output {  
elasticsearch {  
hosts =\> ["elasticsearch:9200"]  
index =\> "index-%{+YYYY-MM-dd}"  
}  
}  
\</\>  
What can be done to improve indexing time?  
Thanks in advance

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 3, 2019, 1:15pm UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-of-csv-file-via-logstash/197785/2 "2019-09-03T13:15:47Z")

</div>

> [@ludaca](#):
>
> What can be done to improve indexing time?

To find that out you need to identify where the bottleneck is. It could be logstash, it could be elasticsearch. It could be CPU, it could (in elasticsearch) be disk throughput. It could be several other things. You need to monitor the components of your system and see which one is limiting throughput. Nobody can predict that without visibility into your servers.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 1, 2019, 1:20pm UTC](https://discuss.elastic.co/t/how-to-speed-up-indexing-of-csv-file-via-logstash/197785/3 "2019-10-01T13:20:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.