# Logstash Huge data import + Fast import **URL:** \ **Category:** Logstash\ **Created:** [July 7, 2016, 7:31am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909 "2016-07-07T07:31:59Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Pierro](https://avatars.discourse-cdn.com/v4/letter/p/ba8739/32.png) [@Pierro](https://discuss.elastic.co/u/Pierro)\ **Post date:** [July 7, 2016, 7:31am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909/1 "2016-07-07T07:31:59Z")
Hi, I'm a newbie with ELK and I've got some questions, of course. Let's try this one : My config: 3 servers: 1 master with Elasticsearch and logstash, 2 nodes with Elasticsearch. All are data store. The Logstash output filter hosts the 2 nodes. I'm doing data import from a CSV file read by Logstash to Elasticsearch, until now it's working. But now I have to import a huge csv file with 56.000.000 items in one time. Logstash seems OK and try to do the job but it seems that I'll be an old man when it'll be done. I've tried ton divide the file, hoping Logstash threads import faster but it don't (strange ascertainment, the store size is bigger after multiple-files import than after a single file import, for the same amount of data). So you see it coming : What can I do to faster the import ? Thanks ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 7, 2016, 8:11am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909/2 "2016-07-07T08:11:22Z")
> I've tried ton divide the file, hoping Logstash threads import faster but it don't Did you use a single file input listing multiple paths or multiple file inputs listing a single path each? ---
**Author:** ![Pierro](https://avatars.discourse-cdn.com/v4/letter/p/ba8739/32.png) [@Pierro](https://discuss.elastic.co/u/Pierro)\ **Post date:** [July 7, 2016, 8:25am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909/3 "2016-07-07T08:25:22Z")
I've shared items in multiple files (file1.csv, file2.csv, ...), put them in the same directory, and change the Logstath input filter like that : input{ file { path =\> "/opt/logstash-2.3.3/input/file\*.csv" ... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 7, 2016, 8:31am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909/4 "2016-07-07T08:31:38Z")
I believe each file input will be processed in a single thread. To increase parallelism you have to split it into multiple file inputs. ---
**Author:** ![Pierro](https://avatars.discourse-cdn.com/v4/letter/p/ba8739/32.png) [@Pierro](https://discuss.elastic.co/u/Pierro)\ **Post date:** [July 7, 2016, 8:38am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909/5 "2016-07-07T08:38:58Z")
So I have to add multiple input filter ? Something like that : input{ file { path =\> "/opt/logstash-2.3.3/input/file\*.csv" ... } file { path =\> "/opt/logstash-2.3.3/input/file\*.csv" ... } file { path =\> "/opt/logstash-2.3.3/input/file\*.csv" ... } ... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 7, 2016, 8:41am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909/6 "2016-07-07T08:41:43Z")
As I said, "multiple file inputs listing a single path each". ```nohighlight input { file { path => "file1" } file { path => "file2" } ... } ``` ---
**Author:** ![Pierro](https://avatars.discourse-cdn.com/v4/letter/p/ba8739/32.png) [@Pierro](https://discuss.elastic.co/u/Pierro)\ **Post date:** [July 7, 2016, 8:44am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909/7 "2016-07-07T08:44:23Z")
OK, got it ! Thanks for your help 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:49am UTC](https://discuss.elastic.co/t/logstash-huge-data-import-fast-import/54909/8 "2017-07-06T04:49:09Z")