# Mutliple file input not processed (or may be erased in elasticsearch)

**URL:** <https://discuss.elastic.co/t/mutliple-file-input-not-processed-or-may-be-erased-in-elasticsearch/234147>\
**Category:** Logstash\
**Created:** [May 25, 2020, 12:01pm UTC](https://discuss.elastic.co/t/mutliple-file-input-not-processed-or-may-be-erased-in-elasticsearch/234147 "2020-05-25T12:01:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pmo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pmo/32/79080_2.png) [@pmo](https://discuss.elastic.co/u/pmo)\
**Post date:** [May 25, 2020, 12:01pm UTC](https://discuss.elastic.co/t/mutliple-file-input-not-processed-or-may-be-erased-in-elasticsearch/234147/1 "2020-05-25T12:01:59Z")

</div>

Hi,  
I' ve to treat many json files. When I try to import file by file via logstash by putting the name of each file, it works. But when I use "\*", it seams that only 1 file is treated. Or may be there all but only the last one is added to elastic index.  
Could you please help me ? I would like all the data in the json files be inserted in the elastic index.

**File to read**

```auto
-rw-rw-r-- 1 aiqdata aiqdata 59 25 mai 12:11 gpon-onu-NE:OLT37-035RNSPATTON-20200525.json
-rw-rw-r-- 1 aiqdata aiqdata 2149 25 mai 12:11 gpon-onu-NE:OLT38-035RNSPATTON-20200525.json
-rw-rw-r-- 1 aiqdata aiqdata 556 25 mai 12:11 gpon-onu-NE:OLT-CLRX6-20200525.json

```

**logstash conf file**

```auto
input {
  file {
    path => "/home/aiqdata/logstash/gpon/data/gpon-onu-NE:OLT*.json"
    mode => "read"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => json_lines
    type => ONU
    #type => DEBUG
    exit_after_read => true
  }
}

output {
  if [type] == "OLT" {
    elasticsearch {
    hosts => "http://localhost:9200"
    index => "gpon-olt-%{+YYYY.MM.dd}"
    ssl => "false"
    }
  }
  if [type] == "ONU" {
    elasticsearch {
    hosts => "http://localhost:9200"
    index => "gpon-onu-%{+YYYY.MM.dd}"
    ssl => "false"
    }
  }
  if [type] == "DEBUG" {
    stdout {
      codec => rubydebug {
      metadata => true
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 26, 2020, 12:33am UTC](https://discuss.elastic.co/t/mutliple-file-input-not-processed-or-may-be-erased-in-elasticsearch/234147/2 "2020-05-26T00:33:00Z")

</div>

What if you tried;

```auto
path => "/home/aiqdata/logstash/gpon/data/gpon-onu-NE*.json"

```

---

<div class="post-metadata">

**Author:** ![pmo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pmo/32/79080_2.png) [@pmo](https://discuss.elastic.co/u/pmo)\
**Post date:** [May 26, 2020, 7:18am UTC](https://discuss.elastic.co/t/mutliple-file-input-not-processed-or-may-be-erased-in-elasticsearch/234147/3 "2020-05-26T07:18:17Z")

</div>

Hi @warkolm,  
Thanks for your help, but same result.  
I think that all files are parsed because there are deleted, but only one is treated.  
I try to keep only one file to see if all can be treated and it works. But When all the 3 files are present, only one is inserted.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 23, 2020, 7:18am UTC](https://discuss.elastic.co/t/mutliple-file-input-not-processed-or-may-be-erased-in-elasticsearch/234147/4 "2020-06-23T07:18:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.