Conf files getting mixed when uploading

[cjb312 (Chris)]

I'm trying to ingest a number of csv files to view on Kibana but am encountering an issue of fields carrying over from the previous file. I have multiple .conf files formatted like so:

    input {
  file {
    path => "/media/network-data/mzeal_pcap_nsa_data/log_data/LogDatacsv/2015-01-24/byte_count*.csv"
    start_position => "beginning"
   sincedb_path => "/dev/null"
  }
}

filter {
  csv {
      separator => ","
      columns => ["ts","ts_delta","orig_h","uniq_hosts","flows","bytes","flow_avg","host_avg"]
  }
}

output {
  elasticsearch {
    hosts => "centos7"
    index => "network-data-import"
  }
}

The only difference between each .conf is the path and the columns. For some reason if I was to upload this .conf first then another proceeding it the second upload will contain some fields from the first one like "flow_avg" and "host_avg" when those aren't in the second .conf's columns/fields.

Any idea as to why this is happening?

[Badger]

If path.config points to a directory containing multiple files, then they are concatenated to form a single configuration. Events are read from all of the inputs, sent through all of the filters, and written to all of the outputs. The files are not independent unless you are using pipelines.yml with path.config on each pipeline pointing to a separate file.