File_completed_action => "delete" - Works intermittently

Elastic Stack Logstash
1

Hi There,
I worked with logstash - 6.7.0 and ES - 6.7.1

I tried to use file_completed_action with " delete" and sometimes this action was work well and sometimes not.

Does anyone know how "file_completed_action" works?
Or how can I take care of it that will always be deleted?

Thanks

my logstash conf look like this:

input {
  file {
    path => ["/mnt/extracted/*/GeneralInfo.txt", "/mnt/extracted/*/system_overview.txt"]
    start_position => beginning
    codec => multiline{
      pattern => "%{WORD}(%{SPACE}?):(%{SPACE}?)(%{GREEDYDATA}?)"
      negate => true
      what => "previous"
      auto_flush_interval => 1
        }
mode => "read"
file_completed_action => "delete"
type => "general_info"
}
}

filter {
if [type] == "general_info" {
  grok {
    match => { "message" => "(?<key_info>[^:]*):\s?%{GREEDYDATA:value_info}" }
  }
  grok {
    match => { "path" => "%{GREEDYDATA}/extracted/%{DATA:uuid}/" }
  }
}
}

output {
if [type] == "general_info" {
elasticsearch {
  hosts => ["10.208.XX.XX", "10.208.XX.XXX", "10.208.XX.XXX"]
  index => "general_info-%{+YYYY.MM.dd}"
}
}
# stdout {}
}

And my pipeline:

- pipeline.id: GeneralInfo
  path.config: "/etc/logstash/conf.d/GeneralInfo.txt.conf"
  pipeline.workers: 3
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.