I have a logstash configuration to ingest large number of json files from a directory.
The ingestion works perfectly fine, But for a particular number of json files.
For example i pointed the input for a folder with 400000 json files and it gets ingested without a problem. But when I point the input for a folder with 1000000 json files, it does not ingest.
Any idea of why this could happen ?