# Logstash file input very slow with lots of files

**URL:** <https://discuss.elastic.co/t/logstash-file-input-very-slow-with-lots-of-files/233598>\
**Category:** Logstash\
**Created:** [May 20, 2020, 7:20pm UTC](https://discuss.elastic.co/t/logstash-file-input-very-slow-with-lots-of-files/233598 "2020-05-20T19:20:56Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ByronTN](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/byrontn/32/68756_2.png) [@ByronTN](https://discuss.elastic.co/u/ByronTN)\
**Post date:** [May 20, 2020, 7:20pm UTC](https://discuss.elastic.co/t/logstash-file-input-very-slow-with-lots-of-files/233598/1 "2020-05-20T19:20:56Z")

</div>

Hello,

I am working to import a ton of old logs into our elastic cache cluster. These are cloudtrail files so there are millions of very small files I am trying to import. When starting Logstash with an input to glob of files ~100k the cpu spins for hours w/o doing anything. Is there anyway to speed this up so the what seems like the file discovery is faster?

```auto
    input {
       file {
            add_field => ["import_account" , "xxx"]
            path => "/data/xxx/CloudTrail/us-east-1/2020/01/**/*.json.gz"
            mode => "read"
            type => "cloudtrail"
            codec => "cloudtrail"
            file_completed_action => "log_and_delete"
            file_completed_log_path => "/data/xxx/proccessed.log"
            max_open_files => 10
       }
    }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 17, 2020, 7:20pm UTC](https://discuss.elastic.co/t/logstash-file-input-very-slow-with-lots-of-files/233598/2 "2020-06-17T19:20:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.