Check service is running or not on remote machine before run Logstash exec input plugin

Gireesh_Deepak_R · September 27, 2018, 8:27pm

Hello Team,

I need help to find a work around for my use case.

I am using logstash exec input plugin to execute some bash scripts, to process data and index them into ES. And, exec was scheduled to run daily once at 12AM. That part was implemented and it has been running absolutely fine.

Now, I want to add some intelligence to this exec input. I want to check the health of the external service which is running on another machine (not on the same machine where logstash is running).

Can I implement such kind of conditional exec input plugin, check service on remote machine is running or not before run 'exec input plugin' ?

Any help would be appreciated.
Thanks in advance!

-Gireesh

magnusbaeck · September 27, 2018, 9:35pm

The exec plugin itself can't do that, but why not perform the check from the program run by the exec plugin?

Gireesh_Deepak_R · September 28, 2018, 2:14pm

Thanks Magnus for the reply, yup, that would be one option. But, that option has some limitation in my use case, when I scale. I need validation at Logstash layer, I see heartbeat can monitor health of the external service.

So, Could you please tell me, Is it possible in logstash for below use case?

If healthbeat(or anybeat) monitor is successful, then execute the one exec input plugin or else execute the other exec input plugin?

magnusbaeck · October 3, 2018, 5:36pm

What you're saying doesn't make sense. If you're worried about scalability, running a bash for each event is crazy. Unless the heartbeat check fails in the majority of the cases so that a check on the Logstash side would avoid invoking the bash script in the first place, but that sounds unlikely.

My previous answer still applies, but I can add that it would be easy to write a custom plugin that does what the exec plugin does but additionally performs the heartbeat check.

Gireesh_Deepak_R · October 3, 2018, 6:38pm

"Running a bash for each event is crazy"
Re: I'm running a bash job daily once, just want to check health of the external service before run, not sure why it looks crazy. As you suggested I can do the health check in bash job itself, but, I'm just checking is there any possibility with any other beat that can be used along with exec.

No need to run health beat continuously, just want to run before exec input plugin run (scheduled daily once).

Tried all this, but I wasn't successful.

Yup, I'll look into writing a custom plugin, that expertise still I need to improve.

Thanks

magnusbaeck · October 3, 2018, 6:43pm

Re: I'm running a bash job daily once, just want to check health of the external service before run, not sure why it looks crazy.

If you only run it once a day I don't understand your concern about scalability. Why does it matter whether you check the health of the other service in the bash script or in Logstash?

Gireesh_Deepak_R · October 3, 2018, 9:07pm

I understand Magnus, It doesn't matter if I check the health of the service in bash script, as I said, I can and I'll if I can't at logstash.

Scalability - Let's say If I have 100 bash jobs runs thru 100 exec input plugins, then every bash should have a health check run. So, If I can check the health at logstash, only once before run 100 exec input plugins, that would be nice to have, right?

All exec scheduled to run daily once but all nearly at the same time.

Then you may ask, why I need 100 bash/exec, because I have different unique log formats which require different filters to process and finally index them into ES.

Thanks

system · October 31, 2018, 9:13pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.