Stop Logstash with a Filter or Ruby-Code

[cwiechmann (Chris)]

Hi All,

I'm using Logstash to prepare/enrich documents before they are send to Elasticsearch. The enrichment is not really optional and will have an impact later, when the data is queried. Therefore, I need to make sure that the process works and that thousands of documents are not indexed without the enrichment.
The documents are sent to Logstash via Filebeat and an HTTP service is used for the data lookup.

The problem is that I haven't found a way to stop event processing when, for example, when the HTTP service is not available.
I would prefer to just stop Logstash to prevent Filebeat from continuing to send events, which are then incompletely indexed.

Is there any way to stop Logstash with a filter or Ruby code?

Thanks!
Chris

[Badger]

@cwiechmann

Is there any way to stop Logstash with a filter or Ruby code?

In a ruby filter you can call pretty much any code you want. That would include Process::exit. However, that would not shutdown inputs and persist things like a sincedb.

What you really want to do is call shutdown, but I am not sure if you can cons a reference to that, and I am not sure if it would hang waiting for the shutdown to complete whilst shutdown waits for the ruby filter to complete.

[cwiechmann (Chris)]

Thanks for the response. Today I had the chance (finally) to test your approach, but unfortunately it doesn't work. Perhaps as expected. On top of that, I'm not experienced at all with Ruby.

I tried: Process:exit in different ways, but no luck. Can you or someone else help with more examples how this would be possible?

Cheers,
Chris

[Badger]

I just tried it and it is not going to work. The ruby exit function just raises a SystemExit exception. logstash catches that and logs a 'Ruby exception occurred' message.

That said, what you can do is

ruby { code => 'Process.kill(15, Process.pid)' }

and the really good news is that logstash catches the SIGTERM and goes through normal shutdown processing. Double cash back bonus!