Mutliple file input not processed (or may be erased in elasticsearch)

Elastic Stack Logstash
1

Hi,
I' ve to treat many json files. When I try to import file by file via logstash by putting the name of each file, it works. But when I use "*", it seams that only 1 file is treated. Or may be there all but only the last one is added to elastic index.
Could you please help me ? I would like all the data in the json files be inserted in the elastic index.

File to read

-rw-rw-r-- 1 aiqdata aiqdata   59 25 mai   12:11 gpon-onu-NE:OLT37-035RNSPATTON-20200525.json
-rw-rw-r-- 1 aiqdata aiqdata 2149 25 mai   12:11 gpon-onu-NE:OLT38-035RNSPATTON-20200525.json
-rw-rw-r-- 1 aiqdata aiqdata  556 25 mai   12:11 gpon-onu-NE:OLT-CLRX6-20200525.json

logstash conf file

input {
  file {
    path => "/home/aiqdata/logstash/gpon/data/gpon-onu-NE:OLT*.json"
    mode => "read"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => json_lines
    type => ONU
    #type => DEBUG
    exit_after_read => true
  }
}

output {
  if [type] == "OLT" {
    elasticsearch {
    hosts => "http://localhost:9200"
    index => "gpon-olt-%{+YYYY.MM.dd}"
    ssl => "false"
    }
  }
  if [type] == "ONU" {
    elasticsearch {
    hosts => "http://localhost:9200"
    index => "gpon-onu-%{+YYYY.MM.dd}"
    ssl => "false"
    }
  }
  if [type] == "DEBUG" {
    stdout {
      codec => rubydebug {
      metadata => true
      }
    }
  }
}
2

What if you tried;

path => "/home/aiqdata/logstash/gpon/data/gpon-onu-NE*.json"
3

Hi @warkolm,
Thanks for your help, but same result.
I think that all files are parsed because there are deleted, but only one is treated.
I try to keep only one file to see if all can be treated and it works. But When all the 3 files are present, only one is inserted.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.