# How to stop logstash when ruby filter encountered error

**URL:** https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899
**Category:** Logstash
**Created:** [March 21, 2021, 7:50am UTC](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899 "2021-03-21T07:50:47Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![nameisnotimportant](https://avatars.discourse-cdn.com/v4/letter/n/ea5d25/32.png) [@nameisnotimportant](https://discuss.elastic.co/u/nameisnotimportant)
#### Post date: [March 21, 2021, 7:50am UTC](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899/1 "2021-03-21T07:50:47Z")

</div>

Hi,

May i know how to stop logstash from indexing when error found inside ruby filter?

i have following config file which will call external python script to process data and insert the records into elasticsearch.

It has no issue, except that when the ruby filter encounter error, logstash will not stop but continue to index empty records into elasticsearch. How to stop the process?

sample code:

```
input {
	file {
		path => "path/logfiledata.txt" 
		start_position => "beginning"
		sincedb_path => "NUL"
	}
}
filter {
    if [message] =~ /.+/ {
       ruby {
            code => 'require "open3"
                data = "\"" + event.get("message").strip + "\""
                cmd = "python /path/pythonscritp.py #{data}"
                Open3.popen3(cmd) do |stdin, stdout, stderr, wait_thr|
                  begin
                    event.set("process_result", stdout.readline)
                  rescue StandardError => e
                    logger.error("Exception: " + "class name: " + e.class.name + " | " + "error message: " + stderr.read)
                  end
                end'
       }
     }
    mutate {
        # to remove carrige return, new line, tab, double quote, square/curly brackets
        gsub => ["process_result","[\t\n\r]", ""]
        gsub => ['process_result', '"', ""]
        gsub => ["process_result", "[\{\}\[\]]", "" ]
    }
    kv {
        source => "process_result"
        field_split => ","
        value_split => ":"
    }
    mutate {
       remove_field => ["process_result"]
       remove_field => ["message"]
       remove_field => ["path"]
       remove_field => ["@version"]
       remove_field => ["host"]
 }
 
}

output{ 
  elasticsearch {
      hosts => "localhost:9200"
      index => "redemption_reason_%{+YYYY_MM_dd}"
      user => "elastic" 
      password => "changeme"
      ssl => true
      ssl_certificate_verification => true
      cacert => "certificate.crt"
  }
  stdout{}
}
```

---

<div class="post-metadata">

### Author: ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)
#### Post date: [March 21, 2021, 10:21am UTC](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899/2 "2021-03-21T10:21:27Z")

</div>

you can try to add a field or a tag everytime ruby filter throws error, then on your output set a condition to only send the output to ES if the field or tag doesn’t exist. you can direct the failed event to a logfile for example.

this won’t stop logstash (i believe filter failure won’t stop logstash), but it will prevent documents with error to be sent to ES

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 21, 2021, 3:56pm UTC](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899/3 "2021-03-21T15:56:25Z")

</div>

If you actually want to exit the logstash process then see [here](https://discuss.elastic.co/t/stop-logstash-with-a-filter-or-ruby-code/259000/4). If you want to drop the events when you get an error you can call event.cancel in the ruby filter.

---

<div class="post-metadata">

### Author: ![nameisnotimportant](https://avatars.discourse-cdn.com/v4/letter/n/ea5d25/32.png) [@nameisnotimportant](https://discuss.elastic.co/u/nameisnotimportant)
#### Post date: [March 22, 2021, 1:55am UTC](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899/4 "2021-03-22T01:55:55Z")

</div>

Thanks ptamba.  
Although this is not what i am after, but your suggestion gives me an idea to think about it, whether i should exit logstash process or let it go so that other good records can be inserted.

---

<div class="post-metadata">

### Author: ![nameisnotimportant](https://avatars.discourse-cdn.com/v4/letter/n/ea5d25/32.png) [@nameisnotimportant](https://discuss.elastic.co/u/nameisnotimportant)
#### Post date: [March 22, 2021, 2:02am UTC](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899/5 "2021-03-22T02:02:57Z")

</div>

Thanks Badger.

I saw your reply in the other post earlier. And i did try that solution but what i get is the following error message:  
"Ruby exception occurred: this signal not yet implemented in windows"

This is how i apply it.

> ruby {  
> code =\> 'require "open3"  
> data = """ + event.get("message").strip + """  
> cmd = "python D:/tklim/Documents/ProgrammingFolder/pythonFiles/python\_get\_parameters\_from\_logstas.py #{data}"  
> Open3.popen3(cmd) do |stdin, stdout, stderr, wait\_thr|  
> begin  
> event.set("process\_result", stdout.readline)  
> rescue StandardError =\> e  
> logger.error("Exception: " + "class name: " + e.class.name + " | " + "error message: " + stderr.read)  
> Process.kill(15, Process.pid)  
> end  
> end'  
> }

But today, i changed the number from 15 to 9, it works. I think, perhaps it is my pc setting that caused the 'signal not yet implemented in windows' error.

> Process.kill(9, Process.pid)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 19, 2021, 2:03am UTC](https://discuss.elastic.co/t/how-to-stop-logstash-when-ruby-filter-encountered-error/267899/6 "2021-04-19T02:03:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
