# Logstash and delete of gz files

**URL:** <https://discuss.elastic.co/t/logstash-and-delete-of-gz-files/324514>\
**Category:** Logstash\
**Created:** [February 2, 2023, 8:47am UTC](https://discuss.elastic.co/t/logstash-and-delete-of-gz-files/324514 "2023-02-02T08:47:13Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rhh](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@Rhh](https://discuss.elastic.co/u/Rhh)\
**Post date:** [February 2, 2023, 8:47am UTC](https://discuss.elastic.co/t/logstash-and-delete-of-gz-files/324514/1 "2023-02-02T08:47:13Z")

</div>

Hi

I have the following my conf file ...

input {  
file {  
path =\> "C:/TDS.Extra/ConsoleApp13/ConsoleApp13/bin/Debug/test.gz"  
sincedb\_path =\> "nul"  
mode =\> "read"  
file\_completed\_action =\> "delete"  
codec =\> "json"  
}  
}

Running logstash on a windows 10 machine. Parse the files fine, BUT the file is not deleted after it is parsed.

logstash run under administrator account and the administrator has write permissions to the folder where test.tz resides.

When I try the same with a txt file it works fine. The file is deleted.

input {  
file {  
path =\> "C:/TDS.Extra/ConsoleApp13/ConsoleApp13/bin/Debug/test.txt"  
sincedb\_path =\> "nul"  
mode =\> "read"  
file\_completed\_action =\> "delete"  
codec =\> "json"  
}  
}

SO IT HAS TO DO WITH GZ FILES.

What am I missing here ?

Is this a known limitation / bug ?

Any help is apprishitated

Regards

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 2, 2023, 5:36pm UTC](https://discuss.elastic.co/t/logstash-and-delete-of-gz-files/324514/2 "2023-02-02T17:36:51Z")

</div>

> [@Rhh](#):
>
> Is this a known limitation / bug ?

Read through to the end of [this](https://github.com/logstash-plugins/logstash-input-file/issues/256) issue.

---

<div class="post-metadata">

**Author:** ![Rhh](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@Rhh](https://discuss.elastic.co/u/Rhh)\
**Post date:** [February 6, 2023, 10:07am UTC](https://discuss.elastic.co/t/logstash-and-delete-of-gz-files/324514/3 "2023-02-06T10:07:58Z")

</div>

I have compared the logs using a txt file and a gz file.

Using a txt file, the file is deleted as expected.

using a gz file the file is not deleted as expected.

The only line different in the logs ( from the gz file log ) is this ...

[2023-02-06T10:40:14,945][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu

I read something about auto\_flush\_interval, but do not understand how this should be used...

ANY HELP ???

This line does not come if I use a txt file.

Here is my conf file for txt files :

```auto
input {
  file {
    path => "C:/Debug/debug-*.txt"
    sincedb_path => "nul"
    mode => "read"
    file_completed_action => "delete"
    codec => "json"
  }       
}

filter {   

    grok { match => { "path" => "(?<filename>debug-[A-Z][A-Z]-[0-9]+-[0-9]+.txt)" } }

    date {
      match => ["timestamp", "ISO8601"]
      timezone => "Europe/Oslo"
      locale => "no"
      target => "@timestamp"
    }

    mutate { 
      remove_field => ["timestamp", "path","port", "@version"] 
      replace => { "source" => "DEBUG" }
      replace => { "logger" => "LOGSTASH.FTP" }	
      replace => { "host" => "%{host}" }
      uppercase => ["host"]		
    }
}

output {
   stdout {
    codec => rubydebug
   }
   
   elasticsearch {
      hosts => "127.0.0.1:9200"
      manage_template => false
      index => "tds-logs-debug-%{+YYYY.MM.dd}"
   }
  
   file {
     'path' => 'c:/logs/tds-logs-debug-%{+YYYY.MM.dd}.log'
   }
}

```

Here is my conf file for gz files :

```auto
input {
  file {
    path => "C:/Debug/debug-*.gz"
    sincedb_path => "nul"
    mode => "read"
    file_completed_action => "delete"
    codec => "json"
  }       
}

filter {   

    grok { match => { "path" => "(?<filename>debug-[A-Z][A-Z]-[0-9]+-[0-9]+.gz)" } }

    date {
      match => ["timestamp", "ISO8601"]
      timezone => "Europe/Oslo"
      locale => "no"
      target => "@timestamp"
    }

    mutate { 
      remove_field => ["timestamp", "path","port", "@version"] 
      replace => { "source" => "DEBUG" }
      replace => { "logger" => "LOGSTASH.FTP" }	
      replace => { "host" => "%{host}" }
      uppercase => ["host"]		
    }
}

output {
   stdout {
    codec => rubydebug
   }
   
   elasticsearch {
      hosts => "127.0.0.1:9200"
      manage_template => false
      index => "tds-logs-debug-%{+YYYY.MM.dd}"
   }
  
   file {
     'path' => 'c:/logs/tds-logs-debug-%{+YYYY.MM.dd}.log'
   }
}

```

Regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 6, 2023, 10:08am UTC](https://discuss.elastic.co/t/logstash-and-delete-of-gz-files/324514/4 "2023-03-06T10:08:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.