# LOSING OUTPUT DELETE TAKES HOURS

**URL:** <https://discuss.elastic.co/t/losing-output-delete-takes-hours/253305>\
**Category:** Logstash\
**Created:** [October 26, 2020, 12:37pm UTC](https://discuss.elastic.co/t/losing-output-delete-takes-hours/253305 "2020-10-26T12:37:06Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel\_Lopez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_lopez/32/50639_2.png) [@Daniel\_Lopez](https://discuss.elastic.co/u/Daniel_Lopez)\
**Post date:** [October 26, 2020, 12:37pm UTC](https://discuss.elastic.co/t/losing-output-delete-takes-hours/253305/1 "2020-10-26T12:37:06Z")

</div>

HI to all

I will try to explain this issue:

I have these lines in log

**08:41:39.691 new resources port:6215 trunk:1**  
**08:41:40.128 Release resources port:6215**

The First line is index at 08:46, so late i think, but with the second line is worst, it is a delete for the doc and it takes sometimes hours to dissapear from elastic

My conf:

```auto
input {
	file {
		mode => "tail"
		path => "/logs/*.log" 
		start_position => "beginning"
		

    }
}

filter {

if [message] =~ "new resources" {

grok {
  match=>["message","%{DATESTAMP:timestart}.*?\port:%{NUMBER:port}.*?\trunk:%{NUMBER:trunk}"]
      }

date
  {
locale => "es"
match => ["timestart","dd/MM/YY HH:mm:ss.SSS", "ISO8601"]
timezone => "Europe/Vienna"
target => "timestart"
}
 

mutate {
			
			remove_field => ["@version"]
			remove_field => ["host"]
			remove_field => ["message"]
      

  }
  }
 else if [message] =~ "Release resources" {
 grok {
match=>["message","%{DATESTAMP:timeend}.*?\port:%{NUMBER:port}"]
}

 

mutate {
			
			
			remove_field => ["@version"]
			remove_field => ["host"]
			remove_field => ["message"]
			

  }
  }else{
  drop{}}
   }
   output {
stdout {}

if [trunk]
{
		elasticsearch { 
		 hosts => ["bbdd01.local:9200","bbdd02.local:9200","bbdd03.local:9200"]
		 index => "ipm_ory" 
		 
      user => "logstash"
     password => "pass"
		 document_id => "%{port}"
		 doc_as_upsert => true
		 action => index
      retry_on_conflict => 5
		}
}
if [timeend]
{
		elasticsearch { 
		 hosts => ["bbdd01.local:9200","bbdd02.local:9200","bbdd03.local:9200"]
		 index => "ipm_ory"
		 
      user => "logstash"
     password => "pass"
		 document_id => "%{port}"
		 doc_as_upsert => true
		 action => delete
      retry_on_conflict => 5
		}
}

		}

```

Maybe, is there some config in elastic to improve delete from logstash, or is there i way to improve my conf?

BR

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 23, 2020, 12:37pm UTC](https://discuss.elastic.co/t/losing-output-delete-takes-hours/253305/2 "2020-11-23T12:37:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.