Invalid cron expression for schedule field of elasticsearch input plugin

Elastic Stack Logstash
1

I have a pipeline whereby the input section looks like this:

input {
    elasticsearch {
        hosts => ["elasticsearch.my.host.here:port"]
        index => 'my_index_name_pattern'
        query => '{ "query": {  "match_all": {} } }'
    }
}

I have logstash running via docker and it's being orchestrated by Marathon/Mesos. I'm trying to add a "schedule" field to the input plugin section so that the container stays up and the pipeline just runs once a day. Otherwise, the container closes itself once the pipeline completes which then causes Marathon to endlessly attempt to redeploy it. From my understanding of the documentation, I thought that adding the following to the elasticsearch input section would be valid:

schedule => "0 0 0 * * ?"

This should have the pipeline get triggered once a day, every day, at midnight. I verified via elasticsearch-croneval that elasticsearch deems this as a valid expression and will create the intended interval.

However, the stdout logs are complaining that this is an invalid cron expression that cannot be handled. Can someone point out where I'm going wrong here?

Note: I've already tried using '' instead of "", but nothing changed.

Thanks!

2

I don't think this is valid, the third column represents the day and it needs to be a valid day between 1 and 31, 0 is not valid.

If you want to run it every day at midnight you need this:

0 0 * * *

This will run at 00:00 every day on UTC, if you need it to run on another timezone you need to specify it using the canonical name

Something like:

0 0 * * * America/Chicago
3

Hi leandrojmp,

Thanks for the reply!

bin/elasticsearch-croneval throws an exception when verifying your expression of:

0 0 * * *
4

This tool is unrelated to Logstash, it is used to validate expressions used by Elasticsearch, not sure exactly where because I never used it.

The expression as shared is similar to the ones on the documentation.

5

To add to this, I believe the third column is for hour unless I'm mistaken.

From my understanding, it's seconds, minutes, hours, day-of-month, month, day-of-week. So my version of the expression has it triggered on the 0 second, 0 minute, 0 hour, for every possible day-of-month, for every possible month, and it doesn't care about the day-of-week (this has to be a ? or else it would conflict with day-of-month I believe)

6

Sorry, I added my last reply in before yours appeared!

That's interesting how they use different evaluators. It appears like your expression did end up working (although Marathon is still annoyingly stuck in a re-deploy loop :frowning: I guess it's waiting on some kind of health indicator to be seen so that its container status can get updated)!

thank you so much!

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.