# Logstash processing of files in subfolder's not working when having a large amount of subfolders

**URL:** <https://discuss.elastic.co/t/logstash-processing-of-files-in-subfolders-not-working-when-having-a-large-amount-of-subfolders/365266>\
**Category:** Logstash\
**Created:** [August 21, 2024, 10:01am UTC](https://discuss.elastic.co/t/logstash-processing-of-files-in-subfolders-not-working-when-having-a-large-amount-of-subfolders/365266 "2024-08-21T10:01:22Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Maretti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maretti/32/118976_2.png) [@Maretti](https://discuss.elastic.co/u/Maretti)\
**Post date:** [August 21, 2024, 10:01am UTC](https://discuss.elastic.co/t/logstash-processing-of-files-in-subfolders-not-working-when-having-a-large-amount-of-subfolders/365266/1 "2024-08-21T10:01:22Z")

</div>

Hi

Context: I am processing a dataset that contains a lot of subfolder's (2359 in total), when I run my logstash config on for example 5 subfolder's everything is getting processed correctly but on the full dataset this doesn't work

Input config:

```auto
    file {
        # xml logs
        path => "/var/log/logstash/**/*.paldata"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        codec => multiline {
            pattern => "</data>"
            negate => true
            what => "next"
            auto_flush_interval => 1
        }
        mode => "read"
        type => "xml"
        file_completed_action => "log"
        file_completed_log_path => "/var/log/logstash/file_completed_audit"
    }

```

when I check on the file\_completed\_log\_path, I see no files that have been processed so it looks like it doesn't even read it

I checked in the folder and the subfolders are present

I increased my memory to give it more room but this didn't work either

Any ideas what could be the cause of this?